An AI coding assistant is still human-orchestrated

A developer who uses an AI tool to explain a stack trace or generate a test is still working in a human-orchestrated model. The human decides what to ask for and when the answer is good enough, then runs the tests, reads the failures, and shepherds the pull request through review.

That can be valuable, because it makes a good engineer faster. It also makes a sloppy engineer faster, which is not always a gift.

A dark factory workflow asks whether the delivery system itself could take work from intent to plan to implementation to validation to release, with humans involved only at defined control points. Whether AI can write code is the less interesting question. What matters is whether your engineering system is explicit enough for work to move through it without someone babysitting every step, and for most teams the answer is no. I do not mean that as an insult, since finding it out is the reason to run the test.

What the initial workflow would have to look like

If you diagram the idealized AI delivery workflow, it looks simple enough.

Every arrow hides a hard problem.

Who decides whether the request is clear enough? Who knows which repo owns the behavior, or whether the change is safe to deploy during business hours? Does the rollback path work, are the dashboards telling the truth, and who owns the customer impact when the system gets it wrong?

The thought experiment earns its keep here, because it will not let you hide behind the happy path. Most software organizations can write code just fine. Where they struggle is everything around the code, which tends to be vague, from intake and ownership to the rollback plan. Experienced people watching and correcting all day make that vagueness survivable. An automated workflow has nobody doing that job, so every gap those people used to cover turns into a stalled change or a bad deploy.

Most teams are not ready, and that is useful to know

AI is imperfect, but that is the obvious reason teams are not ready. The bigger one is that most delivery systems are not explicit enough, and you can see it in the gap between how the system is described and how it behaves. The build works, as long as you know the local setup steps nobody wrote down. The tests pass, except for the ones everyone knows to ignore. The pipeline works too, apart from the service that needs one manual step afterward. Then there is the rollback plan nobody has practiced in months, the ownership model that stays clear right up until a defect crosses three repos and two teams, and the alert that only one person knows how to read.

That mess is the system your customers use every day, and neither the architecture diagram nor the onboarding document describes it.

Walk through the workflow as a thought experiment

Take a simple, hypothetical change. A customer reports a defect, and the workflow has to get from that report to a fix running safely in production, with a decision at the end about whether to keep it or roll it back.

Now ask the dark factory question at each step. Could the system understand the customer's report, or would someone need to translate customer language into engineering language? Could it judge customer impact, or is that still inferred from who is yelling the loudest? Could it find the owning service, or does ownership depend on someone remembering that this behavior moved last year? Could it reproduce the problem, or do the lower environments lack the data and permissions to do it?

Could it write a failing test, or is the suite too brittle and too slow to trust? Could it make the fix safely in a codebase full of clever abstractions that only one person understands? And once the fix exists, could it open a pull request reviewers can trust, deploy through something sturdier than a ritual held together by Slack messages and memory, and roll back without everyone discovering that rollback only ever existed on paper?

Each of those questions is practical even if the factory never will be. Wherever the workflow breaks, you have found an engineering problem worth fixing.

The hidden human labor is the system

A lot of software organizations run on invisible human patching. Someone knows which tests are flaky and which deploys are risky. Someone else knows which customer complaint is urgent and which team to pull into the incident. And usually one person remembers that the rollback plan works for the code but not for the database migration, and that the feature flag exists but not every path honors it.

That knowledge is valuable, and it is also a risk. If the system only works because experienced people keep compensating for missing structure, then it has been manually stabilized, whatever the pipeline dashboard says about automation, and AI tends to expose that. An AI workflow can only move confidently through a system with enough explicit structure to support it, meaning clear ownership, reliable tests, and product intent written down well enough to reason about. Without that structure, the AI is wandering through a dark warehouse with a flashlight.

Review cannot be based on vibes

One of the more uncomfortable parts of this thought experiment is code review. Teams talk about review as a quality gate, but in practice it depends on who is available, how tired they are, and whether the pull request looks familiar enough to feel safe, which makes it more of a social habit than a process.

So if an AI agent opens a pull request, what should happen? Having an engineer look at it is the weak answer. A useful pull request arrives with evidence: what changed and why, what tests were added or updated, what risks remain, how it rolls out and rolls back, and which production signals will be watched. That evidence helps just as much when the author is a person.

The thought experiment forces the team to define good review, turning it from a feeling into a contract. Review has to answer whether the change should exist, whether it is safe to release, and whether the team can operate it after it ships, and code style is the least of those. Most architecture problems are ownership problems, and a lot of review problems are ownership problems wearing a different shirt. When nobody owns the consequences, review drifts toward the surface, where naming and formatting get attention because they are easy to see and operational risk gets missed because spotting it means thinking beyond the diff.

Validation is where the fantasy usually breaks

Generating code is the easy part of the AI story. Knowing whether the generated change is safe is the hard part, so validation sits at the center of the workflow. If the tests are slow, flaky, or ignored, the workflow stops being a factory and turns into a risk amplifier. Without reliable integration environments it cannot build confidence, and without contract tests it cannot reason across service boundaries. Weak observability leaves it unable to tell whether production is healthy, and a painful rollback leaves it unable to recover from its own mistakes.

Dark factory thinking leads straight back to boring fundamentals like trunk-based development, fast builds, and rollback discipline. None of it is trendy. It is the price of a delivery system that does not run on heroics, and it buys you boring production, where things still go wrong now and then but the customer is not left waiting while engineering discovers how the system works.

The design lives in the exception queue

Run the thought experiment once more, from the other direction. If you were designing an AI delivery workflow you meant to trust, the most interesting design question would be what should force it to stop. It should stop when the requirement is ambiguous, when the blast radius is too large, or when the tests do not prove the fix. It should also stop when a change crosses an ownership boundary or the rollback path is weak. Customer impact is its own trigger, because past a certain point a person needs to make the call.

A mature AI workflow is not one that never stops. It is one that stops for the right reasons, and that is the difference between automation and recklessness.

The exception queue tells you where human judgment is required and where your engineering model is still too vague to trust, which makes it one of the most important parts of the design. It also doubles as an improvement backlog, and a good one, because it shows you where to fix something once instead of many times. If too much work stops because ownership is unclear, fix ownership. If it keeps stopping because the tests are weak, fix the tests.

Humans move to the control points

The useful version of a dark factory workflow moves humans to the control points where their judgment counts the most. Humans still own intent and product judgment. They own the architecture decisions and the risk tolerance, and final accountability for what happens to customers stays with them too. The point is to decide where that judgment is required and make every other step explicit, testable, and repeatable, instead of putting a human on every button.

That is a much healthier model than pretending every manual step is valuable just because a person does it. Some manual steps are judgment, and some are bureaucracy or scar tissue. And some are unpaid automation work being performed by tired engineers, which the thought experiment is good at flushing out.

The dark factory is a mirror

I like the dark factory idea even though most teams are nowhere near it. It makes you look at the path from a customer problem to a production fix and see where that path leans on memory, heroics, and rollback plans nobody has practiced. Nobody needs a fully autonomous software factory tomorrow, or probably ever. What you need is to ask the uncomfortable question and be honest about the answer, because wherever the system would fail first is probably where your engineering work should go next.

Take the smallest fix your team shipped last week, walk it through that workflow with the lights off, and write down the first place a person had to reach in and help.