I am really trying to get AI to do software engineering and be genuinely helpful at the higher end of coding. Still working on it 🙂 but the Observable toolchain I co-produced with AI is a step up from where I was a year ago. The technique that helped was feeding the test suite back into the LLM context so we would co-evolve the suite and implementation at the same time. Furthermore I had extra context tools, so if I wanted it to concentrate on a specific test case I had a UI to select that case. I don't think it could make the whole thing alone yet, I definitely wrote a good chunk of the Observable decompiler/compiler pair, but it was very helpful at 1. Writing semanticallt broken but executing and close first draft of code 2. Choosing which bug to fix next and coming up with a suggestion 3. Using new libraries idiomatically (e.g. acorn-walk) https://observablehq.com/@tomlarkworthy/observablejs-toolchain I've been wanting a decompiler for observable for ages, and I don't think I could have done it alone either.