Agentics: what is developer process automation?

The only sane way to actually automate parts of the developer role.

Here’s something that you might want to do these days.

You have some product deployed in prod, doesn’t really matter what exactly but let’s say your product is not literally perfect and therefore has a bug hidden somewhere inside. You don’t know what that bug is, you don’t know when it’s going to fire. But you do know that when it does pop up, it will inevitably trigger your pager only after you managed to get your kid to finally fall asleep. Wouldn’t it be nice to have an agent automatically waiting for you, ready with a full triage report and a pr that will fix the thing in your review queue?

Many teams have implemented something like this. It’s not that tricky, assuming you use the right tools. You have a webhook from sentry kick off a cloud agent on nori, and the agent runtime is set up with all the credentials it needs. We actually wrote about how you could set up your own here.

Here’s another thing that you might want to do these days. We already established that you have a product in prod. Presumably that product has a codebase, and presumably that codebase has docs. Now it’s not necessarily obvious who the docs are for — could be a human, could be a robot — but they’re definitely important because when they drift the code starts to get worse and new features get harder to deploy. Wouldn’t it be nice to have some regularly running process that goes through the codebase and makes sure everything is up to date?

And in fact, many teams have implemented something like this too! Instead of a webhook you have a cron job, and a slightly different prompt, but the rest is basically the same.

What would you call these processes? They are clearly the same category of thing, right?

If you were on the nontechnical side, this would all sound pretty familiar — it’s business process automation. Folks who are familiar with zapier or retool probably have a sense of what this entails. There’s some bespoke software that stitches together data from a few different sources, and maybe runs some transformation on the data too.

Devs have a few things that are BPA shaped — ci/cd, for example. But up until very recently, it was basically impossible to do any developer work that required ingesting or working with free form inputs. Code is basically all free form input! So no one seriously thought about how to automate things that required any amount of code generation, at least not in practice.

Things have obviously changed. LLMs and coding agents are the default way many engineers write code. As the agents have become increasingly autonomous, it has become easier and easier to shift boilerplate engineering to processes that spin up and do work without any human intervention.

I’ve taken to calling this category of tasks ‘developer process automation.’ This is such a simple name that it is almost bad; certainly it wants for creativity. And it is also so obvious — frankly I was convinced that this term must exist elsewhere. But no, as far as I can tell it’s free real estate.

I like the phrase ‘developer process automation’ because I find the simple terminology to be far less aggressive and hypey than “software factory” or, worse, “self driving codebase.” A developer process automation is, well, like a business process automation. It’s something that is intentionally set up by a dev (it does not just poof into existence), though there may be helpful / reusable templates. The best developer process automations simplify and standardize repeatable tasks that mostly don’t change — small, targeted, easy to review, easy to dismiss or fix if necessary. And most importantly, developer process automations do not imply that a developer is no longer useful or that the developer can simply turn off their brain and blindly prompt a model. Like, we wouldn’t expect an ops person to just stop working because someone set up a few zapier workflows. If anything, the ops person’s entire job is to set up and manage those workflows.

What are some good DPAs? Here are a few that we currently have running with our cloud agents:

1. Automatic bug triage. Any production error spins up an agent to root cause the issue and draft a PR.

2. Docs gardening. Every few days, an agent makes sure internal / external docs match what is actually in the product.

3. Dead code removal. Once a week, an agent scans a codebase for code paths that are never hit and creates a removal PR.

4. Feature flag cleanup. Remove code paths that have been deployed to production for a while.

As agents write more and more code (and, bluntly, as humans review less and less of it) we found it useful to set up some automations that ensure agents continue coloring inside the lines. That includes:

• Util cleanup. Many codebases have a well tested suite of utilities that the agents will happily ignore in favor of rewriting the exact same code. Have another agent fix that once every few days.

• Folder structure. Without careful observation, agents will start dumping code in random parts of the codebase. This happens slowly — some code will sneak, then slightly more — and then suddenly you realize that the agent has no idea where any code goes. You can avoid the problem with a little evaluator that will dynamically flag when code is not following the requirements of a given folder.

• Test cleanup. The AI writes a lot of really shitty tests. It’s worth doing a pass to remove redundancies, if only to save your poor CPUs when they need to run the test suite.

• We never automate things like end-to-end refactors. But we do have an agent that runs once a week, that proposes a list of possible refactors ordered by complexity. Most of these are garbage. One in ten is so good that we drop what we were previously doing and work on that instead.

For the most part, we are very wary of automations that aim to be ‘self improving’. When agents try and write memory, they aim for ‘summarization’, when what they should aim for is ‘will this be useful to me in the future?’ We have yet to write a skill that successfully captures that intent. As a result, we don’t trust agents picking their own long term memory; and we really don’t trust agents that set up their own automations. But for these sorts of things, it is very easy to simply have a human in the loop that can quickly evaluate if something is slop or not.

Right now, it is very ‘in vogue’ to hate on software factories. “It’s all slop,” they say. “It’s so over,” they say. My team has been running a ‘software factory’ for like 8 months, and we haven’t looked at the code in maybe 90% of that time. That’s because we never thought of it primarily as a way to stop doing the hard work of understanding the code model. Instead, we got there one DPA at a time.

Developer process automation is a pretty new area, and I think people don’t really know what it is or how to buy tools for it. But as time goes on, I think there will be a standard suite of DPA that every team will have, because it is simply silly not to. And that ends up being the foundation for automating more and more of the software development life cycle, until you actually do have something that can mostly run with less human intervention.

---

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. Check out our agent learning hub.

If you are trying to set up developer automations, we can help! We deploy everything you need to automate your agents in one easy package. Check out noriagentic.com for more.

---