It feels like over the last few weeks ChatGPT has been on the top of everyone’s lips. However I don’t want to talk about that today, instead, late to the party as always I want to talk about GitHub Copilot.
Years ago, I tried a tool that used AI to generate unit tests. I thought it might be a good idea for adding unit tests to code I was writing for a side project. I remember it being quite impressive but getting in my way too much and I quickly abandoned it.
With some extra spare time around Christmas, I decided to give GitHub Copilot a try. With a 30 day trial in hand, I worked on some of my projects.
For those who aren’t familiar with GitHub Copilot, it is a product offered by GitHub from 2021. GitHub Copilot is an AI that can provide suggestions for code as you are developing. It has been trained on billions of lines of code. This can range from guessing some arguments to a function, implementing the whole thing and everything in between. The aim of it is to help you as you develop, rather than write your code for you.
So what is my experience? Well it varies… Over the last week I wrote some TypeScript/React, Flask and a little Django code. With React, it was very useful, I had forgotten some of the syntax with hooks but it was helpful in guessing what I wanted. It helped me with setting up types as I wrote the props to the components. It made some weird decisions when I started to use GraphQL. I think it definitely saved me a lot of time.
With Flask and Django is where things get a little bit complicated. Like with TypeScript it was good at helping me add typing to my code. However it confused syntax between MongoEngine and Marshmallow MongoEngine. Kind of painful, I should have been more mindful, in this instance I was actually better off copying and pasting the code snippets from the library. I assume it is because the Marshmallow MongoEngine is not a popular library (based off pip and GitHub stats). It also did the below:
Confused? Yeah, same here… My experience with Python was not as fruitful as React. My project isn’t exactly standard, so I kind of feel like I can let it have some wiggle room. It was very useful for generating test data and general Python stuff. Overall my React experience was a lot more positive than my Python experience.
In general on a coding level, I found with Copilot’s help I was writing a lot more code before verifying it. In Copilot’s defence, this was more of a reflection of myself getting lazy and letting it take the controls a bit. Also in Copilot’s defence, the code I was writing normally worked when it wasn’t going off the rails. The important question though is, did it get in my way? No, for the most part it was more of a help than a hinderance. The next most important question, is it worth the money? As of writing it is $10 USD a month or $100 USD for the year, if it can save me at least 1 hour a month I think $10 is worth that.
This experience is very personal. The few times it tried to swerve the plane off course I was able to stop it because of my developer experience, particularly in Python and React. If I was more junior of a developer or was unfamiliar with React and Python, I think Copilot would have been more of a hindrance than a help. I would have missed out on a lot of opportunities to learn and opportunities to read documentation. I feel like letting something else do that heavy lifting without explanation for a developer early on would stunt growth.
After all is said and done, would I gladly pay the $10 a month? I think so, at least based on what I’ve seen. It definitely is exciting to how things are progressing.
So I lied, I will talk about ChatGPT. I tried to use it to help me write this article, it made up some features as below:
I assume this is due to GitHub Copilot was released in 2021 and might not have had a lot of information in the ChatGPT 2021 dataset.
AI is really in an interesting place at the moment. You probably guessed it but it is also worth noting the two images were also AI generated, they both came from DALI-2.