Teaching an AI the process, not just the task

Pre-alpha. Nebuko is usable today, but expect bugs and breaking changes between releases, including the workflow format.

AI is remarkably capable. Give it a bug and it can read the code, find the cause and propose a fix, often faster than we can. We have watched it do that many times, and it’s the reason we built Nebuko. But after a while we noticed that intelligence is only one of the pieces a team needs.

Think about how a software team actually works. We don’t just fix bugs. We build, test, review, deploy and monitor, and we do it together. Knowledge has to be shared: the reviewer needs to see what the developer did, the next person needs to know why a decision was made. A bug fix is only finished when the process around it has been followed.

For a human team, that process is the company’s way of working. Some of it is generic, like “get a second pair of eyes before merging.” Some of it is specific to the team, like “the payment module needs sign-off from the finance owner.” Either way, someone explained it to you, or you wrote it down, or you learned it by breaking it once.

An AI needs the same thing, and it needs it written down in a form a machine can follow. That’s a workflow.

The problem with asking nicely

Here is the uncomfortable part: an LLM is a lot like a person. Left to its own choices, it takes the shortest path. It skips the test run when the fix looks obvious. It writes “all tests pass” in its summary without showing the output. It moves on to the next task before the review is done. Nobody has to be careless for this to happen. It’s what happens when a step is optional.

Most agent setups deal with this by asking nicely. The instructions say “always run the tests,” and the agent usually does. Usually isn’t enough for anything you’d ship.

So we went the other way. Our workflow is a path, and the agent is free to think and act however it wants along that path. The path itself is not optional.

steps:
  - name: context
    agent: context
    next: plan
  - name: plan
    agent: architect
    human_gate: true          # the run stops here until a person approves
    conditions:
      - on_result: approved
        next: implement
      - on_result: rejected
        next: context         # gather fresh context, then plan again
  - name: implement
    agent: dev
    next: review
  - name: review
    agent: review

The agent decides how to do each step. The workflow decides which step comes next. An agent can’t invent a step that isn’t in the file, and it can’t skip one that is.

Nebuko ships a default workflow with this shape: analyze → context → plan → implement → review. The plan waits for a person to approve it. Once it’s approved, the dev agent does the work, and the review agent checks it. If the review rejects the work, the run goes back to implement until the fix holds.

Tests are not the agent’s word

In a normal team, you don’t merge until CI is green. The developer can say the tests pass, but the pipeline decides.

A workflow can do the same. A step can run a command, and its result decides where the run goes next:

  - name: implement
    agent: dev
    next: run_tests

  - name: run_tests
    script:
      command: uv
      args: ["run", "pytest", "-q"]
    conditions:
      - on_result: approved     # exit code 0: the tests passed
        next: review
      - on_result: rejected     # any failure
        next: implement

The engine runs the tests, not the dev agent. If they fail, the output goes back to the dev agent as input, and the dev agent has to fix it. It keeps looping until the tests pass or the engine’s circuit breaker stops the run: by default, after 25 agent calls on the same issue, it halts and asks a human to take a look. That limit is what keeps a run from looping forever.

Give the reviewer the evidence, and a different mind

Green tests aren’t enough. A passing suite can still hide a bad change, so the reviewer needs to see what changed:

  - name: run_tests
    script:
      command: uv
      args: ["run", "pytest", "-q"]
    conditions:
      - on_result: approved
        next: git_diff
      - on_result: rejected
        next: implement

  - name: git_diff
    script:
      command: sh
      args: ["-c", "git fetch --no-tags origin main:refs/remotes/origin/main && git diff origin/main...HEAD"]
    next: review

  - name: review
    agent: review
    inputs: [plan, git_diff, run_tests]
    conditions:
      - on_result: approved
        next: null
      - on_result: rejected
        next: implement

The review agent gets the test output and the diff, both produced by the engine. It can also get the original plan, or anything else the run needs. If the review is satisfied, the run ends and a human can merge. If not, the run goes back to the dev agent until the review is satisfied.

This is the loop that catches the failure from the opening. The sentence “all tests pass” in an agent’s summary no longer decides anything. The engine ran the tests, the reviewer read the result, and a red run sends the work back to the dev agent.

We also think the reviewer should be a different model from the developer when you can manage it. Two models trained on different data make different mistakes. A second model is less likely to share the first one’s blind spots, and it’s less likely to be persuaded by the first one’s confident summary.

The human stays in the loop where it matters

Some decisions aren’t the agent’s to make. In a workflow, a step marked human_gate finishes its work, posts the result on the issue, and then the run stops. The next step doesn’t start until someone approves.

The agent can’t release that gate by saying it has been approved, and it can’t approve its own plan. Approval is a /nebuko approve gate comment from a person with the right access to the repository, and the engine checks who sent it. A rejection with feedback sends the step back with that feedback, and the gate closes again until a new decision is made.

AI is a great tool; a harness makes it shine

A Nebuko workflow builds a mandatory path for the AI, which gives your process more determinism: the same steps, in the same order, every time.

A workflow is also an asset you can replay on many different issues. And if it works for you, you can share it with other teams, because it’s a simple YAML file.

The path guarantees the steps, the gates and the evidence. It doesn’t guarantee that the work inside each step is good. A reviewer can still approve weak code. Judgment still lives with the agents and the humans. The workflow just makes sure they’re asked.

Try it yourself

Nebuko is open source and takes a few minutes to set up on any GitHub repository. The quickstart is in the README.

Nebuko is pre-alpha, so tell us what breaks. That’s how the workflow format will get better.