Code Organization - Part One

So you’ve dabbled a bit with Go, and you’re ready to create your first package. A few questions may arise like where should my code live? How should it be structured? What about incorporating third party packages? I spent a long time playing with different structures and workflows, so let’s walk through some options and how I learned to better structure and organize my code.

Workspace

So the first thing we need to understand is the concept of a workspace. By design, Go code is kept in a structured workspace. A workspace is simply a directory containing three directories named bin, pkg, and src.

Creating a new workspace

Let’s create this structure in our home directory with the root of the workspace called gocode, but this can be any name and in any place you like.

$ cd ~                                    # Go to our home directory.
$ mkdir -p gocode/{'bin','pkg','src'}     # Create our workspace structure.

Awesome, we now have a workspace structure setup. Next we need to tell Go where our workspace is. This is done through the $GOPATH environment variable. If you followed my previous post, you will have your $GOPATH setup already. If not, you need to set the environment variable like so:

$ export GOPATH=$HOME/gocode

Thats it. Your Go workspace is complete and ready to be used. If you added the export line above to your .bashrc or .profile, be sure to reload your profile so it takes effect.

For more information about workspaces, see the official documentation here.

Dependency management

Here’s where things get interesting. When I first started learning Go, the vast majority of my knowledge was in Python. Anyone that has done development in Python knows about virtualenv. Virtualenv was a package you installed globally on your system. When you wanted to start a new project you followed a pattern like this:

$ cd ~/path/to/projects/        # Go to your projects directory.
$ mkdir new_super_project       # Create new project directory.
$ cd new_super_project          # Jump into new directory.
$ virtualenv ./                 # Make this directory a virtual Python environment.
$ source bin/activate           # This runs a bash script that modifies a few environment variables.

After you created and activated your virtual environment, you have a self-contained Python environment. You can pip install packages or setup your project however you like with out worrying about your other projects. This is very important when dealing with third party packages. You could have two projects running Django but one is running an older 1.5.3 version and your new project requires 1.6.5. If both projects shared the same Python environment, one would overwrite the Django package breaking the the other project. Separate virtual environments prevent this from happening.

Alternative workspaces

I always thought virtualenv was a cool way to separate your projects and it’s dependencies. So my natural thought was, can I do this in Go? Turns out the answer is yes! So I borrowed the bin/activate bash script and made some tweaks for Go. Checkout the gist here.

My Go workflow then looked like this:

$ cd ~/path/to/projects/            # Go to your projects directory.
$ mkdir new_super_go_project        # Create new project directory.
$ cd new_super_go_project           # Jump into new directory.
$ source ~/scripts/activate_go.sh   # This updated my $GOPATH environment variable to this directory.
$ mkdir -p {'bin','pkg','src'}      # Manually create my Go workspace.
$ cd src                            # Jump into the `src` directory where my code would live.
$ git init                          # This would be my git root.

After that was all setup, this is what my src directory would look like:

src
├── app
│   ├── subpackage
│   │   └── other.go
│   └── main.go
├── github.com
│   └── lib
│        └── pq
│              ├── ....go
└──  README.md

Yes, you are seeing another package vendored right into my code. I would usually set this up as a git submodule or subtree. This worked just fine, and even better, git solved my dependency management issues.

While this worked for me for a few projects, it turned into a pain. I would forget to switch workspaces or forget to deactivate and reactivate when switching projects. I often found myself creating new terminal sessions because my $GOPATH would become messed up.

On top of that, there is a huge problem with creating these types of workspaces. If you open source your project, say on GitHub, it’s not go gettable. So for that reason, I would say don’t do this.

Other solutions

There are a few other solutions to dependency management in Go. I had looked into goven a while back and thought it was not a bad idea. Basically it copies the source of one project into the root of your project while updating the import paths. Not bad, but not perfect either.

Then godep came onto the scene. This looks the most promising by far. Setup your project and run godep save to lock in your dependencies. Then whenever you want to run a go command, you would run godep go instead. Pretty clean solution.

Today’s workflow

Currently I run a single workspace just like I setup at the beginning of this post. One workspace for all my projects. When working on a project, I create or clone it into it’s canonical path: $GOPATH/src/github.com/unrolled/myproject. When importing this package into another project I would use it’s canonical path: import github.com/unrolled/myproject. This also makes the project gettable: go get github.com/unrolled/myproject.

For third party packages, I simply run go get url-to-package to bring down that package into my workspace. This is definitely not a perfect solution as there is no dependency management here. But it’s working so far. I’m sure I’ll need to manage some dependencies in the future, in which case I’ll be giving godep a spin.

I would also like to point out a great blog post and presentation from Peter Bourgon on how they manage their workspaces and dependencies at SoundCloud.



Cross-Compiling Go Apps

Have you ever wanted to compile your Go app on your Mac, but deploy that binary on Ubuntu or CentOS? I always knew there was a way, but figured it would be a pain to get setup. Well, I’m here to tell you it’s super easy and works great for production deployments. For simplicities sake, I’m going to focus on compiling on a Mac and deploying to a Linux system. But the theory applies across all platforms.

One liner

Yes, it’s literally a single command to make it happen (minus the directory change):

$ go version
go version go1.2.2 darwin/amd64
$ cd $GOROOT/src
$ GOOS=linux GOARCH=amd64 ./make.bash

Note: Dave Cheney pointed out that as of Go 1.2, the go tool knows to disable CGO for cross compiles. So you no longer need to provide CGO_ENABLED=0 with your commands.

You’re all set to compile to Linux now. Your normal usage of Go will continue to work exactly as before. But to build for Linux you just prepend your target platform details to the go build command like so:

$ GOOS=linux GOARCH=amd64 go build -o app.linux app.go

Note that I like to append .linux to my build output file so I can easily differentiate between local builds and Linux builds. After I copy the file to my Linux server, I just rename it and drop the extension.

The cgo caveat

Unfortunately this does not work if you use cgo. Since the Go compiler invokes the C compiler on the local platform, which has no idea that we are attempting to compile for another platform… ya long story short, it’ll crap out.

For more information on this topic, I would suggest Dave Cheney’s awesome post here.



Getting Started With Go

In this tutorial we are going to install Go. There’s several ways to do this, but this is my preferred method, and is dead simple. I should note that this will cover the installation process on Mac OS X and Linux, but not Windows. (I have installed Go on Windows 7 before, very strait forward if your use the MSI installer.) So lets get started.

Downloading Go

First things first. We need to download the appropriate achieve from the Go download page. Visiting the download page should yield something like this:

alt Go Download

You will notice that the go1.2.2 section is expanded by default. This is the most resent stable version (as of the writing of this tutorial), and as you probably guessed, the version we want to install. There are several links for different OS’s and architectures, so lets figure out what we need.

Choosing an archive

  1. First determine which system your installing to. You will most likely want OS X 10.8+ or Linux. If your running an older Mac OS like Snow Leopard or Lion, you’ll want to look at OS X 10.6+.
  2. Next we can look at the architecture type. Choose 64-bit unless you have a very good reason not to.
  3. Finally you only want to focus on the Archive in the kind column. Ignore the Installer or Source.

The processes of elimination should leave you with one of the following options:

  • go1.2.2.darwin-amd64-osx10.6.tar.gz for Mac OS X 10.6 and 10.7 (Snow Leopard or Lion)
  • go1.2.2.darwin-amd64-osx10.8.tar.gz for Mac OS X 10.8 and 10.9 (Mountain Lion or Mavericks)
  • go1.2.2.linux-amd64.tar.gz for Linux

Download it! Since I’m on my Mac, I’ll be downloading the go1.2.2.darwin-amd64-osx10.8.tar.gz file.

Checksum

Did you notice that last column on the download page labels SHA1 Checksum? For my OS X 10.8 download the checksum is 19be1eca8fc01b32bb6588a70773b84cdce6bed1. The checksum lets us verify that the file we have downloaded is in fact the one we intended to download. If the download lost a few bytes along the way, or someone tampered with the file along it’s path to your computer, our checksum verification would let us know.

Verify your download

To run the verification we will be running an OpenSSL command:

# I downloaded the file to my downloads folder.
# `cd` to wherever you downloaded your file to.
$ cd ~/downloads
$ ls
go1.2.2.darwin-amd64-osx10.8.tar.gz
$ openssl sha1 go1.2.2.darwin-amd64-osx10.8.tar.gz
SHA1(go1.2.2.darwin-amd64-osx10.8.tar.gz)= 19be1eca8fc01b32bb6588a70773b84cdce6bed1

Hey look at that, it matches what the download page displayed! That means we have the proper file and are ready to proceed.

Installing

Now on to the easy part. We need to complete two steps to finish the installation. First we will move the file into our home directory, and second we will unpack the file.

$ cd ~
$ mv ~/downloads/go1.2.2.darwin-amd64-osx10.8.tar.gz .
$ tar -xzf go1.2.2.darwin-amd64-osx10.8.tar.gz

You will now have a go folder in your home directory. To verify everything is working correctly, run the following:

$ ~/go/bin/go version
go version go1.2.2 darwin/amd64

Awesome, it’s working! Feel free to delete the Go archive file now.

$PATH

Now that Go is installed and working, we need to add a couple of environment variables. First GOROOT will tell Go where it’s been installed to, and second GOPATH specifies the location of our workspace.

# Create a new folder for our workspace.
$ mkdir -p gocode/src

# Add our environment variables to our .profile file.
# Note: use .bashrc on Linux.
$ cat <<EOM >~/.profile
export GOROOT=\$HOME/go
export GOPATH=\$HOME/gocode
export PATH=\$GOROOT/bin:\$GOPATH/bin:\$PATH
EOM

# Reload our environment.
# Note: use .bashrc on Linux.
$ source .profile

Now you can test out the installation again:

$ go version
go version go1.2.2 darwin/amd64

Since we’ve added Go to your $PATH, the go command will work from any directory.

Upgrading Go

Some time in the near future Go 1.3 will become the latest stable release. So you will need to update your installation of Go. Well I’m happy to say it’s super easy. All you will need to do is replace the current ~/go folder with the new archive. I’ll use Go 1.3RC2 as an example:

$ cd ~
$ rm -rf go/
$ wget http://golang.org/dl/go1.3rc2.darwin-amd64-osx10.8.tar.gz
$ openssl sha1 go1.3rc2.darwin-amd64-osx10.8.tar.gz

# Verify checksums match.
$ tar xzf go1.3rc2.darwin-amd64-osx10.8.tar.gz
$ go version
go version go1.3rc2 darwin/amd64

Hello World

So now we are going to write the canonical hello world application. First lets choose a package path, I’m going to use github.com/unrolled. We will also need a package name, I’ll use hello. Lets create this structure in our workspace:

$ mkdir -p $GOPATH/src/github.com/unrolled/hello
$ cd $GOPATH/src/github.com/unrolled/hello

Now in that folder that we just created, create a new file called hello.go and copy the following contents into it:

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello world.")
}

That’s it. We just need to run our application now:

$ cd $GOPATH/src/github.com/unrolled/hello
$ go run hello.go
Hello world.

We can also install our hello application and run it as a standalone program:

$ cd $GOPATH/src/github.com/unrolled/hello
$ go install
$ hello
Hello world.

Since we added our GOPATH/bin to our path above, we can just run the application like any other program!



An Introduction

While this may not be my first attempt at blogging, it’s definitely my most ambitious attempt. In the past I usually just added noise to internet, making obvious/sarcastic comments to currently trending tech news. But this time it’s going to be different. I have a line up of Go posts in the works that will get a new Go programmer well versed in writing clean idiomatic Go code, focusing on web apps to begin.

So a brief summary of me. I’ve been developing for over 10 years now, that is to say I’ve been a paid developer for 10 years. My first taste of computers began way before that on the Apple IIe. I remember hole punching 5.25-inch disks in order to make them double sided, felt like magic at the time. Since then I’ve jump around many versions of Windows, OS X, and several distros of Linux. At the same time I’ve mastered a few programming languages along the way… PHP, Python, Object-C, and most recently Go.

I’ve found a sweet spot with Go as of late. Python used to be my language of choice, but with Go you can easily write clean expressive code. It just feels so nature. You can write a huge application, not look at it for months, come back and it feels like you wrote it yesterday. I have to say that I’ve never had that feeling with a large Django app.

So that brings me to this blog. One of the best ways to master something is to teach it to others. I’ll be introducing a bunch of Go tutorials in hopes of helping others new to the language. That’s not to say that it will be the only topic on this blog, but it will be the primary focus to start.

Hope you enjoy.