no obvious trends. The most important thing we’ve seen is giving the agent the ability to fully drive the box, whatever the box may be. For terminal apps that’s driving through tmux. For webapps, that’s playwright. For slack apps, we straight up created a dummy slack app in a dummy slack env that it can play with. It’s challenging but works well when done right
This is what we're doing as well with our agentic coding workflows. We have worked to automate more and more of the verification.
Also, a lot of my time when creating tickets is figuring out the verification conditions to ensure that the agent implemented the features correctly.
One thing Ive noticed is that coding skills stay similar between projects but verification skills and tooling change between projects.
What trends have you observed with verification?
no obvious trends. The most important thing we’ve seen is giving the agent the ability to fully drive the box, whatever the box may be. For terminal apps that’s driving through tmux. For webapps, that’s playwright. For slack apps, we straight up created a dummy slack app in a dummy slack env that it can play with. It’s challenging but works well when done right