The sublayer gem is actually set up to make it possible to build any type of trigger you like - for Asana I’m pretty sure their gem uses polling, but I’ve also built triggers listening to webhooks, filesystem changes, and cron in the past…
Is there an example you have in mind? Happy to throw something together if you’d like…
What I'm thinking about is an integration with Github Issues for comments as an Agent context and a human-in-the-loop to approve agent plan in front of the bugfix, commit and pr automated processes. I've used the top-tier llms planning with each other until they come to an agreement then assign the plan to a single llm for implementing the plan.
I like your idea of having multi-llms generating their own PR then the human-in-the-loop to approve or reject the pr.
Compelling approach! Do you suspect simplicity in the approach is important to this working well? What are noticing now in terms of “weak spots” in the performance? Will your approach scale for more complex and varied codebases?
Yeah, simplicity in approach is definitely important for it working well, though that's the main reason I'm pretty skeptical of the claims that software engineering is dead as a profession.
Loose coupling and high cohesion for an application's code as well as at a higher level with code bases will make it possible to scale this pattern out, though I don't see it being a guarantee for all code bases. We'll likely need to rewrite a lot of apps that currently have architectures that don't lend themselves well to things like this, and likely need to do a lot of work to ensure codebases don't drift outside coding agents being able to work easily with them.
> This is obvious even in this essay, which is written unassisted by me. Turns out a natural pattern for being an AI-assisted writer (for the next 3 months at least) is to write building-block essays with AI help, and then glue them together with artisanal unassisted ones, as I’m doing here. I’m doing this both because I do still enjoy the creative challenge of handwritten words where there is something new to be tried (just as I still enjoy walking though I could drive most places), but also because LLMs aren’t currently that good at this kind of glued-together contraption-of-sloptraptions writing.
Hey Scott. It's hard to find folks "who get it" right now and are playing around like this. Catchup soon? https://ghuntley.com/subagents/ && https://ghuntley.com/redlining
I had not considered the automated trigger as a way to kick off a workflow/pipeline of agents. Do you have a poll or push trigger?
The sublayer gem is actually set up to make it possible to build any type of trigger you like - for Asana I’m pretty sure their gem uses polling, but I’ve also built triggers listening to webhooks, filesystem changes, and cron in the past…
Is there an example you have in mind? Happy to throw something together if you’d like…
What I'm thinking about is an integration with Github Issues for comments as an Agent context and a human-in-the-loop to approve agent plan in front of the bugfix, commit and pr automated processes. I've used the top-tier llms planning with each other until they come to an agreement then assign the plan to a single llm for implementing the plan.
I like your idea of having multi-llms generating their own PR then the human-in-the-loop to approve or reject the pr.
Ah cool, yeah could probably put something like that together pretty quick or would be fun to chat through that with you!
If you haven't seen it - I shared this one a while back: https://worksonmymachine.substack.com/p/automating-test-generation-with-gemini
The "human-in-the-loop to approve the agent plan" was done through PRs that the agent made against a feature branch.
1) Agent makes PR of integration tests against the feature branch
2) Human reviews PR, optionally makes edits, and merging the PR is seen as "approving"
3) Once "approved" (merged), the next agent starts and generates an implementation plan
Compelling approach! Do you suspect simplicity in the approach is important to this working well? What are noticing now in terms of “weak spots” in the performance? Will your approach scale for more complex and varied codebases?
Thanks!
Yeah, simplicity in approach is definitely important for it working well, though that's the main reason I'm pretty skeptical of the claims that software engineering is dead as a profession.
Loose coupling and high cohesion for an application's code as well as at a higher level with code bases will make it possible to scale this pattern out, though I don't see it being a guarantee for all code bases. We'll likely need to rewrite a lot of apps that currently have architectures that don't lend themselves well to things like this, and likely need to do a lot of work to ensure codebases don't drift outside coding agents being able to work easily with them.
Venkatesh Rao over on Contraptions actually had a post recently where he talks about what I see as a similar pattern but with his writing with AI: https://contraptions.venkateshrao.com/p/terms-of-centaur-service
In particular this part:
> This is obvious even in this essay, which is written unassisted by me. Turns out a natural pattern for being an AI-assisted writer (for the next 3 months at least) is to write building-block essays with AI help, and then glue them together with artisanal unassisted ones, as I’m doing here. I’m doing this both because I do still enjoy the creative challenge of handwritten words where there is something new to be tried (just as I still enjoy walking though I could drive most places), but also because LLMs aren’t currently that good at this kind of glued-together contraption-of-sloptraptions writing.