Why do we struggle to get our agents to do the right thing?

  • Client requirements not quite met
  • Trying to beat our agents into choosing the right path
  • Strange inconsistencies and unforseen outcomes

From my experience, it tends to boil down not just to upfront planning and planning refinement (grill-me-with-docs for example) but also including all those little things that your concious mind forgot about or didn’t think were important during planning.

These are the things that you would have remembered were you writing the code in the old fashioned manual way. Your coding would have prompted you, the human to remember that thing you forgot to write down.

The answer, I’ve found is to do the following:

  • Give your agent all of your notes, even if they’re crappy - this can be done via just taking a photo of them if they’re on paper and feeding them in (yup, agents are excellent at OCR / understanding diagrams etc.)
  • Record all your meetings/calls. Make sure you have permission first though. Feed the full transcripts into your agent with metadata (eg when the meeting was).
  • Get all your notes into one place. Date everything (really important!)
  • Use something like grill-me-with-docs to vet all the notes into sensible ADRs (architectural design records) that agents read before getting stuck into things.

I’m still learning, but definitely getting there.