• This article assumes basic programming proficiency - I'll explain some Go specifics along the way, but mostly focus on Git
• I might forget to include the specific packages I've utilized for that snippet. I'll try my best to catch all of them, but if you get any errors, check Go's package documentation to see which one you need.
• This is meant to be a very quick exploratory project - i.e. it has very little error handling and manual testing - not a good practice for a project you want to build to be stable / long-term
• Ironically, since Part 1 takes an atomized per-function approach and it's something I've just been toying with, the code isn't currently on my GitHub (it will be by Part 2 though, when we dive a bit deeper)

By the end of this series, the goal will be to use Got as a replacement for basic Git commands - initializing a repository, tracking files, making commits, and pushing/pulling, along with interfacing with GitHub.

Here's the Go logo for thumbnail purposes! So far that's my one complaint with HEY World - @ HEY team please let me set a thumbnail picture without having to awkwardly insert it somewhere.

What is Git?

My friend Carl and I are working on a project together, and decide to upload what we've already whipped up to a Git provider, in this case GitHub.

We've now got ourselves a remote repository - this solves some potential headaches by providing a home for our code that isn't dependent on a single developer's system.

Anyone who needs to work on this project can clone the remote repository to their computer, thus creating a local repository. Let's say I add a file to my local repository. I tell Git to track that file, then push my changes back to the remote repository.

Now that this file exists on the remote repository, Carl can pull any new or changed data to his local repository. He then makes some changes to the file I added, and pushes the updates to the remote repository.

And then this process repeats ad nauseum.

Initializing a Repository

.git
│   HEAD    
│
└───refs
│   └───heads
│   
└───objects

ref: refs/heads/main

import (
    "path/filepath"
    "os"
)

func gotInit(repo string) {
    gotDir := filepath.Join(repo, ".git")
    gotObj := filepath.Join(gotDir, "objects")
    gotRef := filepath.Join(gotDir, "refs", "heads")
 
    os.MkdirAll(gotObj, os.ModePerm)
    os.MkdirAll(gotRef, os.ModePerm)
 
    head := []byte("ref: refs/head/main")
    os.WriteFile(filepath.Join(gotDir, "HEAD"), head, os.ModePerm)
}

Trees, Commits, and Blobs - Oh My!

Before we move forward with our implementation, we have to take a detour and look into how Git keeps track of a repository.

We'll start with blobs, as they're the most straight forward of the three object types. Blobs represent an ordinary file that exists within a repository. And that's it! Of course, blobs and the other object types go through a compression process for Git to be able to efficiently track our repository, but we'll touch on that in just a bit.

Tree objects represent the working tree at time of commit. That is, when we are getting ready to send files to the remote repository, a tree object will be made and act as the snapshot of the files we're currently tracking. It does this by listing out the blob objects that have been created and tracked.

Commit objects are where everything comes together. The commit object contains references to the tree object for the current commit and a reference to the parent commit, along with the author, timestamp and commit message. We already know that the tree object contains references to our tracked files, but what is the parent commit?

The parent commit is a reference to the object that the HEAD file aliases. While we won't be implementing branching within Got just yet, the head objects are utilized for that purpose. This way a repository can tell which branch the changes should be pushed to. I'll elaborate on this in greater detail when (at this point considering my Capstone schedule, if) we come to implementing branching.

Finally, let's talk about the compression process we need to send our data through for Git to understand it. Each object consists of a header, which includes the object type and the objects size. The header is followed by a NUL byte before continuing with the raw data. All of this information is then zlib compressed before being written to a file.

To identify objects, we also need to send the full data through an SHA-1 hashing algorithm, creating a unique 40-character long string. Once all of this is done, we store the data within .git/objects/ab/cd..., where ab are the first two characters of the SHA-1 digest, and cd... being the remaining 38 characters.

While this might seem like an arduous process, Go's core library includes some very useful packages that we'll be utilizing to accomplish this.

Creating and Storing Objects

import (
    "compress/zlib"
    "strings"
    "bytes"
    "io"
)
 
func zlibCompress(data string) []byte {
    var b bytes.Buffer
 
    w := zlib.NewWriter(&b)
    w.Write([]byte(data))
    w.Close()
 
    return b.Bytes()
}
  
func zlibDecompress(data []byte) string {
    b := bytes.NewBuffer(data)
 
    r, err := zlib.NewReader(b)
    if err != nil {
         panic(err)
    }
 
    buf := new(strings.Builder)
    io.Copy(buf, r)
 
    r.Close()
    return buf.String()
}

...
func hashObj(data string, objType string, write bool) string {
    rawData := fmt.Sprintf("%s %d \x00%s", objType, len(data), data)
 
    h := sha1.New()
    io.WriteString(h, rawData)
    hashedData := fmt.Sprintf("%x", h.Sum(nil))
 
    if write {
        path := filepath.Join(".git", "objects", hashedData[:2])
        os.MkdirAll(path, os.ModePerm)
        file := filepath.Join(path, hashedData[2:])
        os.WriteFile(file, zlibCompress(rawData), os.ModePerm)
    }
 
    return hashedData
}

Next Steps