Scenery
I took the Christmas break end of 2024 to sit down with an AI coding assistant and do some real coding. For this, I used a project that I had prototyped some 10 years ago and that I actually tried to get a patent for in 2003 or so. It's an Android Java project.
Mind you, I'm not a Java coder. I've worked in Java teams for many years and you can ask any Java coder who paired with me: They'll probably tell you that I can be a pain in the rear with all my questions like
- "What does this do?"
- "Why is this an interface?"
- "Does this really belong here?"
- "Couldn't you simplify it like this ...?"
But put me solo in front of a Java project and I'm mostly lost. After years of functional programming and more year of not programming at all, my OOO skills are rusty at best.
Also, I don't bring a lot of Android expertise. Yes, I did read about the required part years ago and some of that memory was – surprisingly – still accessible, but expertise clearly no.
So, for this project, I consider myself a very junior programmer.
After some research, I settled on Codeium with their Windsurf editor. I also installed the latest Android Studio. When I first launched the emulator I was pleasantly surprised to find my old prototype still there. I did create a private project on GitHub and also connected Android Studio to my Google account to be able to ask Gemini questions.
- Codeium and Windsurf with Claude Sonnet 3.5 (Pro tier)
- GitHub Copilot with ChatGPT 4o (free tier)
- Google Gemini (free tier)
High Level Summary
- Both GitHub Copilot and Google Gemini failed. They disappointed me by repeatedly suggesting Kotlin solutions while clearly inside a Java project. I did not expect such problems. Even when given the information, they would suggest Kotlin again later. (Personal note: There's nothing exciting enough for me to learn Kotlin.)
- It's impressive how well the machine can write code.
- Generated code often prefers outdated solutions.
- Generated code very often breaks other existing code in surprising ways.
- While the mechanics of refactoring work reasonably well, the approach to refactoring is poor. The model contradicted itself.
- The machine is essentially a very junior developer who can read documentation and type _really_ fast.
- Git, git, git. Use git to capture the rare moments when your software is in a working state.
- You will not get better. Learning is hard and by having someone or something else write your code, you're taking the easy path. There's not improvement down that road.
To Be Continued
I plan to update this document with more details and more insights.
Expect:
- Musing on open source, licenses and the software industry
- Thoughts on risk for commercial development
- Detailed reports from the project work and insights gathered
- Maybe even a working software in the end ... Who knows
- Links