When the Structural Engineers Leave the Construction Site – Why the Current Development of AI Concerns Me
You close the construction site, check the plans and the structure, and ask yourself:
What do the people who are leaving know that we may not see yet?
I am an architect and urban planner. Before I moved into the digital world almost three decades ago, I worked in architecture and urban planning. Maybe that is why I still think a lot in images of spaces, structures, foundations and statics. And when I look at what is happening in artificial intelligence, this image feels more and more appropriate to me.
Because something is happening in the global AI industry that increasingly concerns me as a designer and former architect:
People who work on the very foundations of these systems are leaving their companies or going public with serious warnings.
Just a few days ago, I wrote here about AI researcher Jacob Coxon leaving Anthropic. I also looked back at my own almost three decades in the digital industry – from my first web design projects, to intranets and extranets, information architecture, and eventually the design of complex digital spaces used by millions of people. My conclusion at the time was simple: The ghosts we created are increasingly moving beyond our control. But building trust and taking responsibility remains a human task. ... I have continued reading, researching and trying to understand what is happening.
And the more closely I look, the less Coxon's warning feels like an isolated voice shouting into the desert. A pattern is starting to emerge.
When the Structural Engineers Start Running
Bilal Chughtai is another AI researcher who recently left his employer and has now spoken publicly about his concerns. On September 14, 2026, the former Google DeepMind research engineer wrote on X and LinkedIn:
“I earnestly believe that AI has the potential to kill us all.”
Bilal Chughtai, September 14, 2026
He also wrote that he was “extremely concerned by the default trajectory of this technology” and that we might be running out of time to avoid such an outcome. At the same time, Chughtai said he believes it is still possible to navigate AI safely, but that this will require cooperation rather than a race between AI companies.
When people who have worked on the inner workings and safety of these systems start pulling the emergency brake, I don't think the right response is simply to call it panic. For me, it is the equivalent of structural engineers running out of a half-finished building and shouting that something may be wrong with the structure. So what do the builders do?
Beijing has dismissed calls for slowing AI development as “fearmongering” and part of a “Cold War playbook.” In Washington, President Donald Trump has also pushed back against calls for slowing AI development, arguing that the United States needs to stay ahead of China. That creates a difficult situation. If political leaders and technology companies treat warnings from their own researchers as background noise, the pressure to keep moving can become stronger than the pressure to understand where we are going. And that is what worries me.
The Russell Problem: A Wrong Blueprint
This is not a new concern. AI pioneer and UC Berkeley professor Stuart Russell has warned for years about what he sees as a fundamental problem in the way we think about intelligent machines.
His basic argument is surprisingly simple:
If we give a machine a fixed objective and make it extremely capable of achieving that objective, we should not assume that the machine will automatically understand what humans actually mean or want.
Russell has proposed a different foundation for AI. Machines should be designed around human preferences, should remain uncertain about what those preferences really are, and should be willing to defer to humans. In other words, uncertainty is not necessarily a weakness. It can be a safety feature.
Russell's approach is very different from the idea of building increasingly autonomous systems that simply become better and better at achieving whatever objective we give them. His work on this problem goes back years, including his book Human Compatible and earlier academic work.
For me, this is again an architectural question. What exactly are we building into the foundation?
The Architecture of Blind Trust
As a UX designer, I have spent much of my career trying to organize complexity. We design digital paths, navigation and information architectures so that people can use technology and trust it without having to understand every detail behind it. But what does interface design and user experience mean when the system behind the screen starts making decisions that even its creators cannot fully explain This is the paradox I see in our profession.
At the front end, we continue to build smooth, elegant and easy-to-use interfaces.
At the back end, the foundation may be becoming more difficult to understand.
We polish the facade of a building while the structural engineers are telling us that we may not fully understand the structure underneath it.
That is not a comfortable thought for someone who has spent his professional life designing the connection between humans and technology.
The Need for Speed
One argument I keep hearing is that we cannot slow down because the competition is too strong. If one company or one country slows down, somebody else will simply continue. There is a real strategic problem here. Anthropic CEO Dario Amodei himself has acknowledged it.
But on September 12, 2026, Amodei also wrote something that I think deserves much more attention:
“We must slow the pace at which we improve the capabilities of AI models.”
Dario Amodei, September 12, 2026
His argument is not to stop AI development completely. His point is that safety work needs time to catch up with rapidly increasing capabilities. He specifically warned that recursive self-improvement could eventually “outrun our ability to understand and control these systems.”
That sounds to me less like a call to stop building and more like a call to check the plans before adding another floor. And perhaps that is exactly the conversation we need.
Where Does That Leave Us as Designers?
If governments struggle to slow things down and researchers inside the industry are increasingly raising alarms, another question becomes important for all of us working at the intersection of humans and technology: What is our responsibility as designers? I cannot, We cannot simply look away.
The warnings from Jacob Coxon and Bilal Chughtai, together with the much older scientific work of people like Stuart Russell, should not be treated as interesting footnotes.
For me, they are more like the canary in the coal mine. And perhaps that metaphor means a little more to me than to some people. I come from a region shaped by coal mining. So when I think of the canary in the coal mine, I don't just think of a nice metaphor. I think of a warning system. The canary was not there to explain exactly what was happening. It was there to tell the miners: Something is wrong. Pay attention.
It is time for designers, developers, product leaders and users to stop polishing only the beautiful surfaces and start asking much harder questions about the structure underneath.
- Are these systems understandable?
- Can they be monitored?
- Can they be corrected?
- Can we reliably stop them when necessary?
And perhaps most importantly: Do the people building them really understand the structure they are creating?
The People Building the Building Are Also Worried
The current discussion becomes even more interesting when other leading figures in AI start saying similar things. On September 14, 2026, Geoffrey Hinton told ABC that there is a “significant risk” that AI could escape human control. He also said:
“Until we've solved that problem, it would be foolish to develop them.”
Geoffrey Hinton, September 14, 2026
Hinton has been warning about this problem for years. In a Nobel Prize interview, he explained that he believes AI systems will probably become smarter than humans, while we still do not know whether we will be able to keep them under control. And Demis Hassabis, chair of Google DeepMind, also responded to Amodei's September proposal. On September 12, 2026, Hassabis wrote:
“Dario’s essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment.”
Demis Hassabis, September 12, 2026
That is not the same statement I had originally attributed to him. His actual words are more cautious – and I think that makes them more interesting. He is not saying that we already know exactly what to do. He is saying that the direction deserves serious consideration. And perhaps that is the point. We do not need to know everything before we stop and check the structure.
Trust Needs a Foundation — Trust does not build itself.
And trust cannot be built on a foundation that even its builders are beginning to question.
I am for sure not arguing that AI development should stop. I am arguing that we should take the warnings seriously. As an architect, I would never accept the argument that we have to keep building because another building is going up next door. As a UX designer, I would never accept a user interface simply because it looks beautiful if the system underneath it cannot be trusted. And as someone who has spent almost three decades designing the connection between people and technology, I believe we need to ask the same questions about AI.
Who is responsible if something goes wrong?
And who has the courage to stop the construction when the structural engineers tell us that we may not fully understand what we are building? I firmly believe we all are f...cking responsible.
Comments
Post a Comment