What Is a Software Factory?

Lately, there’s been a lot of talk about software factories. The idea is that AI can take on more of the work of building software, increasing how much we can produce and reducing the time between an idea and a finished product.

I recently watched the Syntax episode “WTF is a Software Factory?”, which got me thinking about what a software factory actually is and what it might become.

To me, a software factory is an arrangement of people, models and tools designed to make software creation more effective. It might include models that investigate a problem, propose a design and write code, along with tools that test the result and people who guide the work and decide what to do with it.

The factory is not just the model. It is the whole process that turns a request into a useful result. And one of the hardest parts of building that factory is making sure its output meets the quality standards needed for the work.

So how do we take advantage of faster production while making sure the output is useful? I think some answers can be found in the factories that came before software.

From handwork to machinery

Before textile factories became widespread, much spinning and weaving took place in homes and small workshops. People worked with tools they understood to produce yarn and cloth, sometimes for their own use and sometimes for sale.

The Industrial Revolution changed that. New machinery automated parts of the work, and production increasingly moved into factories. A person operating a machine could produce more than they could by doing the same work by hand.

The work did not disappear. People still needed to operate machines, supply materials, maintain equipment, resolve problems and decide what to produce. The way they worked changed.

As machinery improved, producing the same goods by hand became harder to justify economically. Handwork remained useful where its qualities were worth the additional effort, and it still has a place today. But for much of the market, machine production became the competitive choice.

Software development may be going through a similar transition. Instead of producing every part by hand, an engineer increasingly works with machines that can investigate, write, test and revise software.

I expect using capable AI tools to become necessary for much software work to remain competitive. That does not make the people involved useless. It changes where their time and judgment are most valuable.

We will still need people who understand the systems being built, including people capable of working close to the underlying code. But more of their work may involve directing, examining and improving machinery that produces software.

When someone makes a product by hand, they work directly with its materials and can notice problems as they go. When a machine takes over more of that work, the person responsible no longer has the same opportunity to see every step.

One answer would be to inspect every result as closely as if it had been made by hand. But if that takes as much work as producing it ourselves, we lose much of the speed the machine gives us.

We need a process that helps us reach our quality goals while still benefiting from the machinery. That means deciding what to check, when to check it and when a person needs to intervene.

Supervision and deviation

The earlier we catch a problem, the less work there may be to undo.

Suppose an agent is asked to add a feature. Early in the process, it makes an architectural decision based on a misunderstanding of the requirements. It then implements the feature around that decision. Another agent writes tests, and a reviewer checks whether the code follows the design.

Each step can look successful. The code compiles, the tests pass and the review finds no obvious implementation error. But the process has built an entire solution around the wrong assumption.

The stations agree, but they agree around a mistake.

If someone had examined the architectural decision before implementation began, they might have caught the misunderstanding with a short conversation. Once the decision has spread through the code and tests, fixing it can require taking much of the work apart.

That does not mean a person needs to approve every decision. It means the location of an intervention matters.

A review before an expensive architectural change may prevent substantial downstream work. A review of every routine formatting change would probably cost more time than it saves. The right process depends on the work, the decisions involved and what the machinery has demonstrated it can handle.

The questions are: where should supervision happen, who is responsible for it, and how often does someone need to intervene?

Models are useful precisely because they can interpret requests and make decisions we have not specified in detail. But that freedom also means their decisions need to be evaluated. A model can make a convincing choice that does not fit the system or solve the problem we intended.

We can reduce the amount of supervision through clearer requirements, better tools and checks, and experience with the work being automated. I do not think we can eliminate the need for someone to take responsibility for what a model produces.

That person might work directly with the model throughout a task. Or they might monitor a process, investigate exceptions and review selected results. In both cases, someone needs the understanding and authority to decide whether the machinery is doing useful work and when it needs to stop.

The question is not simply whether the machinery is producing something. It is whether someone can tell when it is producing the wrong thing, and whether the process gives them a chance to do something about it.

That is a problem factories have faced before.

A machine needs a way to stop

For an automobile factory, stopping the production line is expensive. While the line is stopped, it is not producing cars.

In traditional mass production, keeping the line moving was a priority. When workers encountered a problem, production could continue and the defect could be addressed through inspection and rework later.

Toyota took a different approach. Workers could call attention to a problem and stop the line when it could not be resolved in time. Machines could also stop when they detected abnormalities. Finding and addressing problems during production became part of the process itself.

At first, deliberately stopping production to investigate a problem might seem like a strange way to make a factory more productive. But the number of cars coming off the line is only part of the result. Those cars also need to meet the factory’s quality standards.

A part outside its required tolerances may need to be repaired or discarded. If the problem goes unnoticed, the factory can keep producing defective parts. Those parts can then create more work further down the line.

Stopping costs time now, but it gives the people responsible an opportunity to fix the immediate problem and prevent it from recurring. Over time, those improvements can reduce both defects and the work needed to correct them.

That does not mean the goal is to make every part as precise as possible. Tolerances can be too demanding for the job. A factory that produces a small number of exceptionally precise parts may be doing more work than necessary if the product does not require that precision.

The goal is to produce enough useful output at the quality the job requires. Quantity matters, but output that cannot be used does not provide the value the production numbers suggest.

Software factories face a similar trade-off.

Right now, much of the excitement is about speed: how much code agents can write, how many tasks they can complete and how quickly changes can move through a pipeline.

But more output is not necessarily more useful output. A factory can produce large amounts of code that looks plausible while solving the wrong problem, adding unnecessary complexity or failing to fit the system it is meant to improve. Someone eventually has to understand, correct or remove that work.

At the other extreme, requiring a person to examine every small operation could make the process too slow to justify automating it.

The question is where to set the tolerances and when to intervene. Which decisions can the machinery make reliably? Which results can we check automatically? When should the process stop and bring a problem back to a person?

I think human supervision will remain an important part of producing software we can trust with models. That does not mean a person has to inspect everything forever.

Every intervention takes time. If too much of the factory depends on a person reviewing its output, supervision becomes a limit on how much work it can do.

We can try to reduce that burden by improving the process: making the task more specific, adding checks and identifying the points where a person’s judgment matters most.

We can also take another approach. We can use a supervised model to build a machine that carries out repeatable work without asking the model to make the same decisions on every run.

Both approaches have a place. I’ve been exploring the second one while building my game.

Building machines with AI

A year ago, I looked at AI-generated 3D models and thought they were almost there. The results had obvious problems, but it seemed reasonable to expect that a few more months of improvement would make them good enough for my game.

My quality bar was not especially high. I was not looking for perfect models. I needed assets that fit together, followed a consistent style and did not contain distracting artifacts or deformed parts.

From what I’ve seen, getting that reliably is still difficult. A generated model can look promising at first, then reveal a problem that needs to be fixed before I can use it. If the model cannot make that change reliably, I have to open the asset in Blender and modify it myself.

I could generate models repeatedly, inspect them and keep the best ones. But selection alone may not be enough. The chosen models can still need manual corrections, and they still need to fit together as part of the same game.

At that point, I have to ask how much time the process actually saves. It may give me a useful starting point, but I still spend time generating, selecting, correcting and preparing the assets. I may also have to compromise on creative direction to work with what the model happens to produce.

I’ve encountered versions of the same difficulty with images, video and music. A result can look close to what I want, but that is different from producing a consistent collection that meets the quality bar for the project.

I could build a factory that generates assets and assembles them into a game. It might produce something quickly, but based on what I’ve seen, that does not mean the result would meet the quality bar I have for Arcanex Online.

That is the difference between producing output and producing output within the tolerances the project needs.

Working with AI to build procedural generators gave me a different route. Instead of asking a model to produce every asset, I could use it to help write the code that produces them.

I can describe what I want, provide examples or reference images, and work with the model to develop a generator. Then I can inspect the code, change specific rules and reproduce its output. If I want to adjust one part, I can work on the part of the generator responsible for it.

For my game, I’ve worked with AI to write code that generates maps deterministically. Given the same inputs, the generator can recreate the same result. That matters because I need to be able to remove parts of the world from memory and bring them back when they are needed.

I’ve also used code to create materials, clothing, tools and other items from existing 3D models. The work still involves experimentation and judgment. But once a generator behaves as intended, I can reuse it to produce variations without asking a model to make every decision again.

The same idea can apply to music. A model can help write code that generates a composition. I can then adjust the rules, instrumentation or structure and reproduce the result as I develop it.

In each case, the model helps me build the machine. The code becomes the machine that runs.

Deterministic code is not automatically correct. It can reproduce a mistake just as reliably as a useful result, and a generator can produce assets that do not meet the creative goal. But code gives me a way to inspect the logic, test its behavior, reproduce an output and make a targeted change.

What will a software factory be made of?

I think the most valuable software factories will be made mostly of code.

People will work with models to build and improve generators and other tools. Those tools will then produce the code, assets, music or other outputs the factory is designed to create.

The factory itself becomes software. Some of its machines may produce more software, while others produce assets or perform different kinds of work. Above those machines, models can help people investigate problems, develop new tools and improve the factory.

That moves the developer a layer up. People are not necessarily writing every part of the final product by hand. They are working with models to build, operate and improve the machinery that produces it.

Code is useful for repeatable work because we can inspect it, test it and understand what to expect from it. Once a program has earned our trust within its intended scope, it can run repeatedly without someone intervening on every execution.

People remain responsible for the factory. They need to maintain and manage it, make sure it has what it needs, and keep its output within the quality and efficiency goals it was built to meet. When something goes wrong, someone needs to recognize the problem, intervene and understand enough to fix the machinery.

Models can help with that work too, but they do not replace the people responsible for the factory. Only people can be held accountable for what it produces. Together, supervised AI and dependable code give those people capabilities that would have taken far more time and effort before.

I expect well-built software factories to lower the cost of creating and maintaining software so much that organizations without them will struggle to compete. They will also make it possible for smaller teams to build systems that once required much larger organizations.

That matters for more than cost. Growing a team brings additional coordination, handovers and communication. Giving a smaller group better machinery may let them take on problems that would otherwise have required an organization too large or expensive to assemble.

Some systems may become possible for the first time. Others were always technically possible, but beyond the resources available to the people who wanted to build them.

I’m trying to build Arcanex Online, an MMORPG. Games of that kind can require large teams and budgets reaching tens or hundreds of millions. I don’t have those resources, so I’m personally invested in finding out what a much smaller team can accomplish with better machinery.

For me, that is the opportunity: not simply generating software faster, but giving capable people the tools to build things they could not have built before.