The AI Can Leave the Room

What AI-built tells us about the maker, and what it does not tell us about the product.

I built an application for my wife using AI. When she uses it, no AI is involved.

Her work involves sensitive information connected to children, families, meetings and follow-up actions. The administrative problem was not a lack of intelligence. It was that dates, notes, contacts and actions had to remain connected to the right person across a large number of cases.

The real information could not leave the protected laptop on which she worked. So I described the process, reproduced only non-identifying field structures and created synthetic records that behaved like the real ones. AI helped me design, write and test the application. The real records never entered that development process.

When the finished tool runs, defined code organises the information locally. It does not send a child’s name or a meeting note to a model. It does not generate a different answer depending on how it feels that morning. It does the useful, comparatively unglamorous work of keeping related things related.

That application taught me something which will be obvious to anyone who has been writing software for years: AI can be essential to the act of creation and irrelevant to the operation of what it creates. It can open the door for a new kind of builder, then leave the room.

The boundary AI disturbed

For a long time, many of us in IT lived on different sides of an educational boundary. Some people specialised in governance, risk or security. Developers and programmers created the products the rest of us used. They were, with some justification, treated as the DNA of IT. They built the systems through which everyone else’s expertise became useful.

I understood systems. I understood risk, controls, privacy and what happened when technology met an organisation. But I was not the person who turned those things into software. That required another education, another vocabulary and, particularly as you got older, a willingness to begin again as a novice.

Generative AI disturbed that boundary.

For many of us, it began with emails, reports and summaries. Then the question changed. Could this create something? Not more text describing an idea, but a digital object that did something useful in the physical world. Something for your home, your partner, your child or a persistent problem in your working day.

That is where AI becomes more than an efficient writing assistant. It gives people access to a category of creation that previously belonged to someone else.

The strange part is that, because AI opened the door, it can feel as though AI must remain inside everything we build. The intelligence that helped create the application becomes confused with the application itself.

It does not have to be there.

What the label actually tells us

“AI-built” tells us something interesting about the maker and almost nothing about the data flow. “AI-powered” tells us something about the operation and still does not tell us whether the product is good.

An AI-built product may have been created by a professional developer working more quickly. It may have been created by a risk specialist who could never previously turn an understanding of a problem into working software. Once finished, it may run entirely through defined code without any connection to a model.

An AI-powered product uses a model while it operates. Real inputs may be interpreted or transformed during use. That may be useful and properly controlled. It may also introduce questions about disclosure, accuracy, cost, dependency and unexpected output.

The labels describe different things: provenance and operation.

We already understand this distinction with conventional software. We do not imagine the programmer still living inside an application, typing a fresh answer every time we press a button. The programmer’s choices remain, including their mistakes, but the programmer does not receive every piece of data processed by the code.

AI-assisted development can work the same way. The model can help produce the application using safe descriptions and synthetic examples, then have no role in its normal operation. The data does not have to follow the maker out of the room.

The imp in the iconograph

I remember a joke in Terry Pratchett’s Discworld about an “iconograph”, a camera containing a tiny imp who rapidly paints whatever it sees through the opening when the button is pressed. It captures the idea of a seemingly magical box whose workings depend on a hidden worker inside.

There is a similar creature in our imagination when someone says a product was built with AI. We picture something coiled inside the software, waiting for an instruction. It might read our data, interpret us and produce something unexpected. The phrase AI-enabled reinforces that feeling. It suggests a latent capacity: something inside the box that might wake up.

When a model is genuinely operating inside a product, that instinct is not entirely wrong. The same instruction may produce a different response. The product can interpret rather than merely follow a fixed path. There is a kind of dynamism inside it.

But we project the same imp into software where no model is running at all.

In the application I built, the imp helped to make the box. It does not live in it.

A larger population of builders

Technology has widened skilled work before. Power tools did not make every person a master furniture maker, but they increased the number of people who could make a decent table. Low-code platforms let people assemble useful internal tools without becoming conventional programmers.

The important effect is not that everyone becomes an expert. They do not. It is that expertise can now emerge from a much larger population.

Imagine, purely as a thought experiment, that 100 people in every 100,000 once had the combination of training, opportunity and persistence needed to build software. If new tools increase that group to 2,000, most of the new builders will not become exceptional. They do not need to. Even if excellence remains rare, a rare outcome occurring across a much larger population produces more excellent builders.

Those figures are not a forecast. They are a way of thinking about what happens when access changes.

Someone may have spent their life noticing how systems fail, how people actually work or where a small administrative problem consumes hours every week. They may have had good software ideas for years and no way to express them. AI does not give that person judgement, curiosity or knowledge of the problem. It gives those qualities a route into code.

If no software existed, you had an idea, you used the tools available to you, and useful software existed afterwards, I think you built software.

That does not make you the best software builder. It makes you a software builder at some level. Whether you are a good one depends on what you made and what happens when other people have to rely on it.

Professional developers do not disappear from this picture. The best builders still have deeper technical knowledge, and serious products still need people who understand scale, security, integration, performance and maintenance. Power tools did not abolish master carpenters. They changed the size of the workshop.

Effort is not the same as care

Once more people can build software, another uncomfortable question appears. What is the work worth?

We often use effort as evidence of value. Tell us that one tool required six months of professional development and another took a non-developer three weeks with AI, and the first feels more valuable before we have used either of them.

That instinct is understandable. Care matters. So do experience, craft and the willingness to take responsibility for a result. But ease is not the same as absence of care. A person may spend hundreds of hours directing, testing and refining an AI-assisted application. Another person may produce something in seconds and put no thought into whether it helps anyone.

The difference is not simply whether AI touched it. The difference is whether anybody cared about the result.

We struggle with this in other creative work. If we enjoy a book and then learn that it was generated quickly, our opinion may change even though the words have not. A photograph feels different when we know someone travelled, waited and worked to capture it. We want some blood in the object. We want value to record the difficulty of creation.

Sometimes it does. Effort can produce knowledge, refinement and depth. Sometimes effort is simply effort. A good sandwich made in five minutes and a good cake made over many hours can both do what they were made to do.

Software at least gives the result a chance to answer back. Does the tool solve the problem? Can somebody use it? Does it behave reliably? Is it secure enough for the consequences of failure? Those questions do not erase craft, but they let us judge the product separately from the biography of its maker.

They also make it harder to dismiss useful work merely because its maker used a tool that made creation easier.

Knowing when the intelligence should stay

None of this is an argument for removing AI from every finished product.

I have a fitness application where AI creates and assesses routines. Its ability to interpret context and produce a tailored response contributes something I do not think a fixed set of choices would reproduce as well. In that application, the intelligence earns its place.

I also built a landlord dashboard in which AI decided whether I should use a tradesperson again. That sounded clever until I used it. I had to enter explanatory text so the model could infer an answer I already knew. A simple use again or do not use again choice solved the real problem more cleanly.

AI was capable of doing the job. That did not mean the job needed AI.

The useful test is not whether a model can perform a function. It is whether interpretation, generation or variability adds enough to justify having the model there. If defined code can do the work perfectly adequately, it may be cheaper, more predictable and less intrusive. If the feature depends on interpretation that fixed rules cannot provide sensibly, runtime AI may be the right choice.

Neither answer deserves to become an ideology.

Useful does not mean safe

There is a danger in correcting one simplistic story by creating another.

Deterministic code is not a moral category. Defined rules can be badly written. A local tool can store information insecurely. Code can repeat the same incorrect decision perfectly every time. A non-developer can create a useful application and still create a new weakness in the system around it.

The widening population will produce more good software because it produces more software. It will also produce more poor software for exactly the same reason.

Testing, validation and security do not disappear when AI leaves the finished product. The safeguards remain familiar: know what result you expect, test whether you get it, and understand the consequences when you do not. AI may make it easier to generate and run tests, but it can also reproduce the same misunderstanding in both the code and the test that supposedly validates it.

Professional expertise may therefore become more valuable in some places, particularly when it helps new builders recognise what they do not know. The old boundary is becoming more permeable, not meaningless.

That is why I am uncomfortable with both extremes. It is wrong to dismiss a product because its maker used AI. It is equally wrong to treat AI-assisted creation as proof that expertise no longer matters. One position protects an old gate. The other pretends there was never anything valuable behind it.

The AI can leave the room

AI can open software creation to people who spent their lives on the other side of an educational boundary. It can translate their understanding of work, risk, administration or ordinary life into a functioning digital object.

Then it can leave.

When it leaves, nothing has necessarily been lost. There is no empty chair and no ghost missing from the machine. The application continues to execute the rules that were produced. The absence of runtime AI is not a deficiency if the work never required it.

When AI stays, it may add precisely the interpretation or flexibility the product needs. It also changes how the product operates and the questions we should ask about it.

Neither provenance nor effort settles whether the result is good. Neither AI assistance nor deterministic code makes it safe. Those judgements still belong to the people who build, test, use and live with what has been created.

The door is wider now. What matters is not that everyone walking through it will become an expert. It is that expertise can begin in far more places than it could before.