Christopher Gandrud

July 17, 2021

Quick thoughts on GitHub Co-pilot

I've been playing around with GitHub Copilot over the past few days. So far, my main impression is that it is a Google level autocomplete for code. You start an R file with the comment:
 
# Import packages

and it will fill in

library(ggplot2)

Just as Google autocomplete reveals the hive-mind of the internet with its autocompletions, GitHub copilot reveals the hive-mind of a language community.

In a non-trivial way, copilot feels like a substitute for googling answers on StackExchange. Which is both useful (the answers are more tailored to your use case) and lacking (I often find the explanations given alongside StackExchange answers to be just as useful as the code examples). 

Better coding
I see a lot of potential for Copilot to improve a human's coding in at least a couple of ways.

Specification driven code with great human readable comments
One of the best ways to use Copilot is to write a comment for the desired action and have Copilot begin building it out. Now you've got your code and detailed human readable comments.

Automate automated testing development
I love test driven development. Start with a test of what we want to achieve and don't accept the code as finished until you meet the test. This results in robust programs. However, there is a time inconsistency problem. Writing tests is costly and slows down your coding, while improving performance over the long-term. By making it easier to write tests, Copilot helps overcome the test driven development time inconsistency problem.  

Speed up learning new languages and operations in languages
In my current management role, I don't code very often. So when I do, I'm rusty. It usually takes me  a non-trivial amount of time (< 1 hour) to get back up to speed. Copilot definitely has the potential to reduce this ramp up time. Likewise, it could help speed up learning a new language in much the same way that having a human with more experience in the new language give you hints can increase your learning.

Areas for improvement

  • Sometimes (not often) copilot will create functions that call objects that have not been created.
  • The quality of the suggestions definitely seem like they are dependent on the quality of the codebase. Automated testing in R for example is notably much worse than in Python. The former language community is much less likely to be developing production code and less experienced with test driven development.