Building a versioned Athens binary from source

You can do that easily with just few commands:

Bash

git clone https://github.com/gomods/athens
cd athens
make build-ver VERSION="0.2.0"

PowerShell

git clone https://github.com/gomods/athens
cd athens
$env:GO111MODULE="on"
$env:GOPROXY="https://proxy.golang.org"
$version = "0.2.0"
$date = (Get-Date).ToUniversalTime()
go build -ldflags "-X github.com/gomods/athens/pkg/build.version=$version -X github.com/gomods/athens/pkg/build.buildDate=$date" -o athens ./cmd/proxy

This will give you a binary named athens. You can print the version and time information by running:

 ./athens -version

which should return something like:

Build Details:
        Version:        0.2.0
        Date:           2018-12-13-20:51:06-UTC

Install Athens on Kubernetes

When you follow the instructions in the Walkthrough, you end up with an Athens Proxy that uses in-memory storage. This is only suitable for trying out the Athens proxy for a short period of time, as you will quickly run out of memory and Athens won’t persist modules between restarts. In order to run a more production-like proxy, you may want to run Athens on a Kubernetes cluster. To aid in deployment of the Athens proxy on Kubernetes, a Helm chart has been provided. This guide will walk you through installing Athens on a Kubernetes cluster using Helm.

• Prerequisites
• Configure Helm
• Deploy Athens

Prerequisites

In order to install Athens on your Kubernetes cluster, there are a few prerequisites that you must satisfy. If you already have completed the following steps, please continue to configuring helm. This guide assumes you have already created a Kubernetes cluster.

• Install the Kubernetes CLI.
• Install the Helm CLI.

Install the Kubernetes CLI

In order to interact with your Kubernetes Cluster, you will need to install kubectl.

Install The Helm CLI

Helm is a tool for installing pre-configured applications on Kubernetes. Install helm by running the following command:

MacOS

brew install kubernetes-helm

Windows

1. Download the latest Helm release.
2. Decompress the tar file.
3. Copy helm.exe to a directory on your PATH.

Linux

curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash

Configure Helm

If your cluster has already been configured to use Helm, please continue to deploy Athens.

If not, please read on.

RBAC Cluster

If your cluster has RBAC enabled, you will need to create a ServiceAccount, ClusterRole and ClusterRoleBinding for Helm to use. The following command will create these and initialize Helm.

kubectl create -f https://raw.githubusercontent.com/Azure/helm-charts/master/docs/prerequisities/helm-rbac-config.yaml
helm init --service-account tiller

Non RBAC Cluster

If your cluster has does not have RBAC enabled, you can simply initialize Helm.

helm init

Before deploying Athens, you will need to wait for the Tiller pod to become Ready. You can check the status by watching the pods in kube-system:

$ kubectl get pods -n kube-system -w
NAME                                    READY     STATUS    RESTARTS   AGE
tiller-deploy-5456568744-76c6s          1/1       Running   0          5s

Deploy Athens

The fastest way to install Athens using Helm is to deploy it from our public Helm chart repository. First, add the repository with this command:

$ helm repo add gomods https://gomods.github.io/athens-charts
$ helm repo update

Next, install the chart with default values to athens namespace:

$ helm install gomods/athens-proxy -n athens --namespace athens

This will deploy a single Athens instance in the athens namespace with disk storage enabled. Additionally, a ClusterIP service will be created.

Advanced Configuration

Replicas

By default, the chart will install Athens with a replica count of 1. To change this, change the replicaCount value:

helm install gomods/athens-proxy -n athens --namespace athens --set replicaCount=3

Resources

By default, the chart will install Athens without specific resource requests or limits. To change this, change the resources value:

helm install gomods/athens-proxy -n athens --namespace athens \
  --set resources.requests.cpu=100m \
  --set resources.requests.memory=64Mi \
  --set resources.limits.cpu=100m \
  --set resources.limits.memory=64Mi

For more information, see Managing Compute Resources for Containers in the Kubernetes documentation.

Give Athens access to private repositories via Github Token (Optional)

1. Create a token at https://github.com/settings/tokens
2. Provide the token to the Athens proxy either through the config.toml file (the GithubToken field) or by setting the ATHENS_GITHUB_TOKEN environment variable.

Storage Providers

The Helm chart currently supports running Athens with two different storage providers: disk and mongo. The default behavior is to use the disk storage provider.

Disk Storage Configuration

When using the disk storage provider, you can configure a number of options regarding data persistence. By default, Athens will deploy using an emptyDir volume. This probably isn’t sufficient for production use cases, so the chart also allows you to configure persistence via a PersistentVolumeClaim. The chart currently allows you to set the following values:

persistence:
  enabled: false
  accessMode: ReadWriteOnce
  size: 4Gi
  storageClass:

Add it to override-values.yaml file and run:

helm install gomods/athens-proxy -n athens --namespace athens -f override-values.yaml

enabled is used to turn on the PVC feature of the chart, while the other values relate directly to the values defined in the PersistentVolumeClaim documentation.

Mongo DB Configuration

To use the Mongo DB storage provider, you will first need a MongoDB instance. Once you have deployed MongoDB, you can configure Athens using the connection string via storage.mongo.url. You will also need to set storage.type to “mongo”.

helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=mongo --set storage.mongo.url=<some-mongodb-connection-string>

S3 Configuration

To use S3 storage with Athens, set storage.type to s3 and set storage.s3.region and storage.s3.bucket to the desired AWS region and S3 bucket name, respectively. By default, Athens will attempt to load AWS credentials using the AWS SDK from the chain of environment variables, shared credentials files, and EC2 instance credentials. To manually specify AWS credentials, set storage.s3.access_key_id, storage.s3.secret_access_key, and change storage.s3.useDefaultConfiguration to false.

helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=s3 --set storage.s3.region=<your-aws-region> --set storage.s3.bucket=<your-bucket>

Minio Configuration

To use S3 storage with Athens, set storage.type to minio. You need to set storage.minio.endpoint as the URL of your minio-installation. This URL can also be an kubernetes-internal one (e.g. something like minio-service.default.svc). You need to create a bucket inside your minio-installation or use an existing one. The bucket needs to be referenced in storage.minio.bucket. Last athens need authentication credentials for your minio in storage.minio.accessKey and storage.minio.secretKey.

helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=minio --set storage.minio.endpoint=<your-minio-endpoint> --set storage.minio.bucket=<your-bucket> --set storage.minio.accessKey=<your-minio-access-key> --set storage.minio.secretKey=<your-minio-secret-key>

Google Cloud Storage

To use Google Cloud Storage storage with Athens, set storage.type to gcp. You need to set storage.gcp.projectID and storage.gcp.bucket to the desired GCP project and bucket name, respectively.

Depending on your deployment environment you will also need to set storage.gcp.serviceAccount to a key which has read/write access to the GCS bucket. If you are running Athens inside GCP, you will most likely not need this as GCP figures out internal authentication between products for you.

helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=gcp --set storage.gcp.projectID=<your-gcp-project> --set storage.gcp.bucket=<your-bucket>

Kubernetes Service

By default, a Kubernetes ClusterIP service is created for the Athens proxy. “ClusterIP” is sufficient in the case when the Athens proxy will be used from within the cluster. To expose Athens outside of the cluster, consider using a “NodePort” or “LoadBalancer” service. This can be changed by setting the service.type value when installing the chart. For example, to deploy Athens using a NodePort service, the following command could be used:

helm install gomods/athens-proxy -n athens --namespace athens --set service.type=NodePort

Ingress Resource

The chart can optionally create a Kubernetes Ingress Resource for you as well. To enable this feature, set the ingress.enabled resource to true.

helm install gomods/athens-proxy -n athens --namespace athens --set ingress.enabled=true

Further configuration values are available in the values.yaml file:

ingress:
  enabled: true
  annotations:
    certmanager.k8s.io/cluster-issuer: "letsencrypt-prod"
    kubernetes.io/tls-acme: "true"
    ingress.kubernetes.io/force-ssl-redirect: "true"
    kubernetes.io/ingress.class: nginx
  hosts:
    - athens.mydomain.com
  tls:
    - secretName: athens.mydomain.com
      hosts:
        - "athens.mydomain.com

Example above sets automatic creation/retrieval of TLS certificates from Let’s Encrypt with cert-manager and uses nginx-ingress controller to expose Athens externally to the Internet.

Add it to override-values.yaml file and run:

helm install gomods/athens-proxy -n athens --namespace athens -f override-values.yaml

Upstream module repository

You can set the URL for the upstream module repository then Athens will try to download modules from the upstream when it doesn’t find them in its own storage.

You have a few good options for what you can set as an upstream:

• https://gocenter.io to use JFrog’s GoCenter
• https://proxy.golang.org to use the Go Module Mirror
• The URL to any other Athens server

The example below shows you how to set GoCenter up as upstream module repository:

upstreamProxy:
  enabled: true
  url: "https://gocenter.io"

Add it to override-values.yaml file and run:

helm install gomods/athens-proxy -n athens --namespace athens -f override-values.yaml

.netrc file support

A .netrc file can be shared as a secret to allow the access to private modules. The secret must be created from a netrc file using the following command (the name of the file must be netrc):

kubectl create secret generic netrcsecret --from-file=./netrc

In order to instruct athens to fetch and use the secret, netrc.enabled flag must be set to true:

helm install gomods/athens-proxy -n athens --namespace athens --set netrc.enabled=true

gitconfig support

A gitconfig file can be shared as a secret to allow the access to modules in private git repositories. For example, you can configure access to private repositories via HTTPS using personal access tokens on GitHub, GitLab and other git services.

First of all, prepare your gitconfig file:

cat << EOF > /tmp/gitconfig
[url "https://user:token@git.example.com/"]
    insteadOf = ssh://git@git.example.com/
    insteadOf = https://git.example.com/
EOF

Next, create the secret using the file created above:

kubectl create secret generic athens-proxy-gitconfig --from-file=gitconfig=/tmp/gitconfig

In order to instruct athens to use the secret, set appropriate flags (or parameters in values.yaml):

helm install gomods/athens-proxy --name athens --namespace athens \
    --set gitconfig.enabled=true \
    --set gitconfig.secretName=athens-proxy-gitconfig \
    --set gitconfig.secretKey=gitconfig

Using the Athens Docker images

Whether setting Athens up using Kubernetes or using the Walkthrough, you’ll most likely be using one of the images that the Athens project produces. This document details what images are available, and has a recap from the Walkthrough of how to use them on their own.

Available Docker images

The Athens project produces two docker images, available via Docker Hub

1. A release version as gomods/athens, each tag corresponds with an Athens release, e.g. v0.7.1. Additionally, a canary tag is available and tracks each commit to main
2. A tip version, as gomods/athens-dev, tagged with every commit to main, e.g. 1573339

For a detailed tags list, check each image’s Docker Hub

Running Athens as a Docker image

This is a quick recap of the Walkthrough

Using the docker cli

In order to run the Athens Proxy using docker, we need first to create a directory that will store the persitant modules. In the example below, the new directory is named athens-storage and is located in our userspace (i.e. $HOME). Then we need to set the ATHENS_STORAGE_TYPE and ATHENS_DISK_STORAGE_ROOT environment variables when we run the Docker container.

Bash

export ATHENS_STORAGE=$HOME/athens-storage
mkdir -p $ATHENS_STORAGE
docker run -d -v $ATHENS_STORAGE:/var/lib/athens \
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens \
   -e ATHENS_STORAGE_TYPE=disk \
   --name athens-proxy \
   --restart always \
   -p 3000:3000 \
   gomods/athens:latest

PowerShell

$env:ATHENS_STORAGE = "$(Join-Path $HOME athens-storage)"
md -Path $env:ATHENS_STORAGE
docker run -d -v "$($env:ATHENS_STORAGE):/var/lib/athens" `
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens `
   -e ATHENS_STORAGE_TYPE=disk `
   --name athens-proxy `
   --restart always `
   -p 3000:3000 `
   gomods/athens:latest

Non-Root User

The Athens docker images comes with a non-root user athens with uid: 1000, gid: 1000 and home directory /home/athens. In situations where running as root is not permitted, this user can be used instead. In all other instuctions replace /root/ with /home/athens/ and set the user and group ids in the run environment to 1000.

docker run -d -v $ATHENS_STORAGE:/var/lib/athens \
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens \
   -e ATHENS_STORAGE_TYPE=disk \
   -v "$PWD/gitconfig/.gitconfig:/home/athens/.gitconfig" \
   --name athens-proxy \
   --restart always \
   -p 3000:3000 \
   -u 1000:1000 \
   gomods/athens:latest

Troubleshooting Athens in Docker

init issues

The Athens docker image uses tini so that defunct processes get reaped. Docker 1.13 and greater includes tini and lets you enable it by passing the --init flag to docker run or by configuring the docker deamon with "init": true. When running in this mode. you may see a warning like this:

[WARN  tini (6)] Tini is not running as PID 1 and isn't registered as a child subreaper.
 Zombie processes will not be re-parented to Tini, so zombie reaping won't work.
 To fix the problem, use the -s option or set the environment variable TINI_SUBREAPER to register Tini as a child subreaper, or run Tini as PID 1.

This is the “Athens-tini” complaining that it’s not running as PID 1. There is no harm in that, since the zombie processes will be reaped by the tini included in Docker.

Shared Team Instance

When you follow the instructions in the Walkthrough, you end up with an Athens Proxy that uses in-memory storage. This is only suitable for trying out the Athens proxy for a short period of time, as you will quickly run out of memory and Athens won’t persist modules between restarts. This guide will help you get Athens running in a more suitable manner for scenarios like providing an instance for your development team to share.

We will use Docker to run the Athens proxy, so first make sure you have Docker installed.

Selecting a Storage Provider

Athens currently supports a number of storage drivers. For local, use we recommend starting with the local disk provider. For other providers, please see the Storage Provider documentation.

Running Athens with Local Disk Storage

In order to run Athens with disk storage, you will next need to identify where you would like to persist modules. In the example below, we will create a new directory named athens-storage in our current directory. Now you are ready to run Athens with disk storage enabled. To enable disk storage, you need to set the ATHENS_STORAGE_TYPE and ATHENS_DISK_STORAGE_ROOT environment variables when you run the Docker container.

The examples below use the :latest Docker tags for simplicity, however we strongly recommend that after your environment is up and running that you switch to using an explicit version (for example :v0.3.0).

Bash

export ATHENS_STORAGE=~/athens-storage
mkdir -p $ATHENS_STORAGE
docker run -d -v $ATHENS_STORAGE:/var/lib/athens \
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens \
   -e ATHENS_STORAGE_TYPE=disk \
   --name athens-proxy \
   --restart always \
   -p 3000:3000 \
   gomods/athens:latest

PowerShell

$env:ATHENS_STORAGE = "$(Join-Path $pwd athens-storage)"
md -Path $env:ATHENS_STORAGE
docker run -d -v "$($env:ATHENS_STORAGE):/var/lib/athens" `
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens `
   -e ATHENS_STORAGE_TYPE=disk `
   --name athens-proxy `
   --restart always `
   -p 3000:3000 `
   gomods/athens:latest

Note: if you have not previously mounted this drive with Docker for Windows, you may be prompted to allow access

Athens should now be running as a Docker container with the local directory, athens-storage mounted as a volume. When Athens retrieves the modules, they will be will be stored in the directory previously created. First, let’s verify that Athens is running:

$ docker ps
CONTAINER ID        IMAGE                               COMMAND           PORTS                    NAMES
f0429b81a4f9        gomods/athens:latest   "/bin/app"        0.0.0.0:3000->3000/tcp   athens-proxy

Now, we can use Athens from any development machine that has Go v1.12+ installed. To verify this, try the following example:

Bash

$ export GO111MODULE=on
$ export GOPROXY=http://127.0.0.1:3000
$ git clone https://github.com/athens-artifacts/walkthrough.git
$ cd walkthrough
$ go run .
go: downloading github.com/athens-artifacts/samplelib v1.0.0
The 🦁 says rawr!

PowerShell

$env:GO111MODULE = "on"
$env:GOPROXY = "http://127.0.0.1:3000"
git clone https://github.com/athens-artifacts/walkthrough.git
cd walkthrough
$ go run .
go: downloading github.com/athens-artifacts/samplelib v1.0.0
The 🦁 says rawr!

We can verify that Athens handled this request by examining the Docker logs:

$ docker logs -f athens-proxy
time="2018-08-21T17:28:53Z" level=warning msg="Unless you set SESSION_SECRET env variable, your session storage is not protected!"
time="2018-08-21T17:28:53Z" level=info msg="Starting application at 0.0.0.0:3000"
handler: GET /github.com/athens-artifacts/samplelib/@v/v1.0.0.info [200]
handler: GET /github.com/athens-artifacts/samplelib/@v/v1.0.0.mod [200]
handler: GET /github.com/athens-artifacts/samplelib/@v/v1.0.0.zip [200]

Now, if you view the contents of the athens_storage directory, you will see that you now have additional files representing the samplelib module.

Bash

$ ls -lr $ATHENS_STORAGE/github.com/athens-artifacts/samplelib/v1.0.0/
total 24
-rwxr-xr-x  1 jeremyrickard  wheel    50 Aug 21 10:52 v1.0.0.info
-rwxr-xr-x  1 jeremyrickard  wheel  2391 Aug 21 10:52 source.zip
-rwxr-xr-x  1 jeremyrickard  wheel    45 Aug 21 10:52 go.mod

PowerShell

$ dir $env:ATHENS_STORAGE\github.com\athens-artifacts\samplelib\v1.0.0\


    Directory: C:\athens-storage\github.com\athens-artifacts\samplelib\v1.0.0


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        8/21/2018   3:31 PM             45 go.mod
-a----        8/21/2018   3:31 PM           2391 source.zip
-a----        8/21/2018   3:31 PM             50 v1.0.0.info

When Athens is restarted, it will serve the module from this location without re-downloading it. To verify that, we need to first remove the Athens container.

docker rm -f athens-proxy

Now, we need to clear the local Go modules storage. This is needed so that your local Go command line tool will re-download the module from Athens. The following commands will clear the local module storage:

Bash

sudo rm -fr "$(go env GOPATH)/pkg/mod"

PowerShell

rm -recurse -force $(go env GOPATH)\pkg\mod

Now, we can re-run the Athens container:

Bash

docker run -d -v $ATHENS_STORAGE:/var/lib/athens \
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens \
   -e ATHENS_STORAGE_TYPE=disk \
   --name athens-proxy \
   --restart always \
   -p 3000:3000 \
   gomods/athens:latest

PowerShell

docker run -d -v "$($env:ATHENS_STORAGE):/var/lib/athens" `
   -e ATHENS_DISK_STORAGE_ROOT=/var/lib/athens `
   -e ATHENS_STORAGE_TYPE=disk `
   --name athens-proxy `
   --restart always `
   -p 3000:3000 `
   gomods/athens:latest

When we re-run our Go example, the Go cli will again download module from Athens. Athens, however, will not need to retrieve the module. It will be served from the Athens on-disk storage.

Bash

$ ls -lr $ATHENS_STORAGE/github.com/athens-artifacts/samplelib/v1.0.0/
total 24
-rwxr-xr-x  1 jeremyrickard  wheel    50 Aug 21 10:52 v1.0.0.info
-rwxr-xr-x  1 jeremyrickard  wheel  2391 Aug 21 10:52 source.zip
-rwxr-xr-x  1 jeremyrickard  wheel    45 Aug 21 10:52 go.mod

PowerShell

$ dir $env:ATHENS_STORAGE\github.com\athens-artifacts\samplelib\v1.0.0\


    Directory: C:\athens-storage\github.com\athens-artifacts\samplelib\v1.0.0


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        8/21/2018   3:31 PM             45 go.mod
-a----        8/21/2018   3:31 PM           2391 source.zip
-a----        8/21/2018   3:31 PM             50 v1.0.0.info

Notice that the timestamps given have not changed.

Next Steps:

• Run the Athens Proxy on Kubernetes with Helm
• Explore best practices for running Athens in Production. Coming Soon

Install on AWS Fargate (ECS)

In this document, we’ll show how to use AWS Fargate (ECS) to run the Athens proxy.

Selecting a Storage Provider

There is documentation about how to use environment variables to configure the various storage providers. However, for this particular example we will use Amazon S3 Storage (s3).

Before You Begin

This guide assumes you already have an AWS account as well as the necessary authentication and permissions to create resources in the account.

Whether you choose to create your resources using the awscli or use something like Terraform, the resources required are the same.

S3 Bucket

In order to persist modules, we will create a s3 bucket for storage.

Below are two examples of creating the s3 bucket using the awscli and Terraform.

awscli:

$ aws s3api create-bucket --bucket athens-proxy-us-east-1-123456789012 --region us-east-1

terraform:

resource "aws_s3_bucket" "cache" {
  bucket = "athens-proxy-us-east-1-123456789012"
}

note: it is a good idea to use environment, region, and/or account ID as components to the bucket name due to their globally unique naming rules.

ECS Task IAM Role

In order for the ECS container instances to use the s3 bucket, we will need to configure the task IAM role to include the proper allow rules.

Below is a least-privileged policy document in both JSON and Terraform to enable ECS containers s3 bucket access to store and retrieve cache assets.

json:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::athens-proxy-us-east-1-123456789012"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::athens-proxy-us-east-1-123456789012/*"
        }
    ]
}

terraform:

resource "aws_iam_policy" "task_role" {
  name   = "athens-proxy-task-role"
  path   = "/"
  policy = data.aws_iam_policy_document.task_role_policy.json
}

data "aws_iam_policy_document" "task_role_policy" {
  statement {
    effect = "Allow"
    actions = [
      "s3:ListBucket",
      "s3:GetBucketLocation"
    ]
    resources = [aws_s3_bucket.cache.arn]
  }
  statement {
    effect = "Allow"
    actions = [
      "s3:PutObject",
      "s3:GetObject",
      "s3:DeleteObject"
    ]
    resources = ["${aws_s3_bucket.cache.arn}/*"]
  }
  statement {
    effect = "Allow"
    actions = [
      "sts:AssumeRole",
      "sts:TagSession"
    ]
    resources = ["*"]
  }
}

ECS Task Definition

In order for Athens to be able to authenticate to the s3 bucket, we will need to configure the storage variables associated with s3.

Below is an excerpt from a task definition that shows the minimum environment variables needed.

"environment": [
  {"name": "AWS_REGION", "value": "us-east-1"},
  {"name": "AWS_USE_DEFAULT_CONFIGURATION", "value": "true"},
  {"name": "ATHENS_STORAGE_TYPE", "value": "s3"},
  {"name": "ATHENS_S3_BUCKET_NAME", "value": "athens-proxy-us-east-1-123456789012"},
]

Install on Azure Container Instances

When you follow the instructions in the Walkthrough, you end up with an Athens Proxy that uses in-memory storage. This is only suitable for trying out the Athens proxy for a short period of time, as you will quickly run out of memory and Athens won’t persist modules between restarts. This guide will help you get Athens running in a more suitable manner for scenarios like providing an instance for your development team to share.

In this document, we’ll show how to use Azure Container Instances (ACI) to run the Athens proxy.

Selecting a Storage Provider

Athens currently supports a number of storage drivers. For quick and easy use on ACI, we recommend using the local disk provider. For more permanent use, we recommend using MongoDB or other more persistent infrastructure. For other providers, please see the storage provider documentation.

Required Environment Variables

Before executing any of the commands below, make sure you have the following environment variables set up on your system:

• AZURE_ATHENS_RESOURCE_GROUP - The Azure Resource Group to install the container in. You need to already have one of these before installing Athens
  • See here for details on how to create a resource group
• AZURE_ATHENS_CONTAINER_NAME - The name of the container. This should be alphanumeric and you can have - and _ characters
• LOCATION - The Azure region to install the container in. See the previous link for an exhaustive list, but here’s a useful cheat sheet that you can use immediately, without reading any docs:
  • North America: eastus2
  • Europe: westeurope
  • Asia: southeastasia
• AZURE_ATHENS_DNS_NAME - The DNS name to assign to the container. It has to be globally unique inside of the region you set (LOCATION)

Installing with the Disk Storage Driver

az container create \
-g "${AZURE_ATHENS_RESOURCE_GROUP}" \
-n "${AZURE_ATHENS_CONTAINER_NAME}-${LOCATION}" \
--image gomods/athens:v0.3.0 \
-e "ATHENS_STORAGE_TYPE=disk" "ATHENS_DISK_STORAGE_ROOT=/var/lib/athens" \
--ip-address=Public \
--dns-name="${AZURE_ATHENS_DNS_NAME}" \
--ports="3000" \
--location=${LOCATION}

Once you’ve created the ACI container, you’ll see a JSON blob that includes the public IP address of the container. You’ll also see the fully qualified domain name (FQDN) of the running container (it will be prefixed by AZURE_ATHENS_DNS_NAME).

Installing with the MongoDB Storage Driver

First, make sure you have the following environment variable set up:

• AZURE_ATHENS_MONGO_URL - The MongoDB connection string. For example: mongodb://username:password@mongo.server.com/?ssl=true

Then run the create command:

az container create \
-g "${AZURE_ATHENS_RESOURCE_GROUP}" \
-n "${AZURE_ATHENS_CONTAINER_NAME}-${LOCATION}" \
--image gomods/athens:v0.3.0 \
-e "ATHENS_STORAGE_TYPE=mongo" "ATHENS_MONGO_STORAGE_URL=${AZURE_ATHENS_MONGO_URL}" \
--ip-address=Public \
--dns-name="${AZURE_ATHENS_DNS_NAME}" \
--ports="3000" \
--location=${LOCATION}

Once you’ve created the ACI container, you’ll see a JSON blob that includes the public IP address of the container. You’ll also see the fully qualified domain name (FQDN) of the running container (it will be prefixed by AZURE_ATHENS_DNS_NAME).

Install on Google Cloud Run

Google Cloud Run is a service that aims to bridge the gap between the maintainance benefits of serverless architecture and the flexibility of Kubernetes. It is built on top of the opensource Knative project. Deploying using Cloud Run is similar to deploying using Google App Engine with the benefits of a free tier and a simpler build process.

Selecting a Storage Provider

There is documentaion about how to use environment variables to configure a large number of storage providers; however, for this prarticular example we will use Google Cloud Storage(GCS) because it fits nicely with Cloud Run.

Before You Begin

This guide assumes you have completed the following tasks:

• Signed up for Google Cloud
• Installed the gcloud command line tool
• Installed the beta plugin for the gcloud command line tool (this is how to set it up)
• Created a (GCS) bucket for your go modules

Setup a GCS Bucket

If you do not already have GCS bucket you can set one up using the gsutil tool.

First select a region you would like to have your storage in. You can then create a bucket in that region using the following command substituting your in your region and bucket name.

$ gsutil mb -l europe-west-4 gs://some-bucket

Setup

Change the values of these environment variables to be appropriate for your application. For GOOGLE_CLOUD_PROJECT, this needs to be the name of the project that has your cloud run deployment in it. ATHENS_REGION should be the region that your cloud run instance will be in, and GCS_BUCKET should be the Google Cloud Storage bucket that Athens will store module code and metadata in..

$ export GOOGLE_CLOUD_PROJECT=your-project
$ export ATHENS_REGION=asia-northeast1
$ export GCS_BUCKET=your-bucket-name
$ gcloud auth login
$ gcloud auth configure-docker

You will then need to push a copy of the Athens docker image to your google cloud container registry.

Below is an example using v0.11.0, for the latest version, check out the latest Athens release

$ docker pull gomods/athens:v0.11.0

$ docker tag gomods/athens:v0.11.0 gcr.io/$GOOGLE_CLOUD_PROJECT/athens:v0.11.0

$ docker push gcr.io/$GOOGLE_CLOUD_PROJECT/athens:v0.11.0

Once you have the container image in your registry you can use gcloud to provision your Athens instance.

$ gcloud beta run deploy \
    --image gcr.io/$GOOGLE_CLOUD_PROJECT/athens:v0.11.0 \
    --platform managed \
    --region $ATHENS_REGION \
    --allow-unauthenticated \
    --set-env-vars=ATHENS_STORAGE_TYPE=gcp \
    --set-env-vars=GOOGLE_CLOUD_PROJECT=$GOOGLE_CLOUD_PROJECT \
    --set-env-vars=ATHENS_STORAGE_GCP_BUCKET=$GCS_BUCKET \
    athens

Once this command finishes is will provide a url to your instance, but you can always find this through the cli:

$ gcloud beta run services describe athens --platform managed --region $ATHENS_REGION | grep hostname

Install on Google App Engine

Google App Engine (GAE) is a Google service allows applications to be deployed without provisioning the underlying hardware. It is similar to Azure Container Engine which is covered in a previous section. This guide will demonstrate how you can get Athens running on GAE.

Selecting a Storage Provider

There is documentaion about how to use environment variables to configure a large number of storage providers; however, for this prarticular example we will use Google Cloud Storage(GCS) because it fits nicely with Cloud Run.

Before You Begin

This guide assumes you have completed the following tasks:

• Signed up for Google Cloud
• Installed the gcloud command line tool

Setup a GCS Bucket

If you do not already have GCS bucket you can set one up using the gsutil tool.

First select a global region you would like to have your storage in. You can then create a bucket in that region using the following command substituting your in your region and bucket name.

$ gsutil mb -l europe-west-4 gs://some-bucket

Setup

First clone the Athens repository

$ git clone https://github.com/gomods/athens.git

There is already a Google Application Engine scaffold set up for you. Copy it into a new file and make changes to the environment variables.

$ cd athens
$ cp scripts/gae/app.sample.yaml scripts/gae/app.yaml
$ code scripts/gae/app.yaml

Once you have configured the environment variables you can deploy Athens as a GAE service.

$ make deploy-gae

Install Athens with BOSH

Athens can be deployed in many ways. The following guide explains how to use BOSH, a deployment and lifecycle tool, in order to deploy an Athens server on virtual machines (VMs) on any infrastructure as a service (IaaS) that is supported by BOSH.

Prerequisites

• Install BOSH
• Setup the infrastructure

Install BOSH

Make sure to have the BOSH CLI installed and set up a BOSH Director on an infrastructure of your choice.

Setup the Infrastructure

If you choose to deploy on a IaaS provider, there are a few prerequisites that need to be set up before starting with the deployment. Depending on which IaaS you will be deploying, you may need to create:

• Public IP: a public IP address for association with the Athens VM.

• Firewall Rules: the following ingress ports must be allowed

  • 3000/tcp - Athens proxy port (if you specify a different port than the default port 3000 in the job properties, adapt this rule accordingly).

  Egress traffic should be restricted depending on your requirements.

Amazon Web Services (AWS)

AWS requires additional settings that should be added to a credentials.yml file using the following template:

# security group IDs to apply to the VM
athens_security_groups: [sg-0123456abcdefgh]

# VPC subnet to deploy Athens to
athens_subnet_id: subnet-0123456789abcdefgh

# a specific, elastic IP address for the VM
external_ip: 3.123.200.100

The credentials need to be added to the deploy command, i.e.

-o manifests/operations/aws-ops.yml

VirtualBox

The fastest way to install Athens using BOSH is probably a Director VM running on VirtualBox which is sufficient for development or testing purposes. If you follow the bosh-lite installation guide, no further preparation is required to deploy Athens.

Deployment

A deployment manifest contains all the information for managing and updating a BOSH deployment. To aid in the deployment of Athens on BOSH, the athens-bosh-release repository provides manifests for basic deployment configurations inside the manifests directory. For quickly creating a standalone Athens server, clone the release repository and cd into it:

git clone --recursive https://github.com/s4heid/athens-bosh-release.git
cd athens-bosh-release

Once the infrastructure has been prepared and the BOSH Director is running, make sure that a stemcell has been uploaded. If this has not been done yet, choose a stemcell from the stemcells section of bosh.io, and upload it via the command line. Additionally, a cloud config is required for IaaS specific configuration used by the Director and the Athens deployment. The manifests directory also contains an example cloud config, which can be uploaded to the Director via

bosh update-config --type=cloud --name=athens \
    --vars-file=credentials.yml manifests/cloud-config.yml

Execute the deploy command which can be extended with ops/vars files depending on which IaaS you will be deploying to.

bosh -d athens deploy manifests/athens.yml  # add extra arguments

For example, when using AWS the deploy command for an Athens Proxy with disk storage would look like

bosh -d athens deploy \
    -o manifests/operations/aws-ops.yml \
    -o manifests/operations/with-persistent-disk.yml \
    -v disk_size=1024 \
    --vars-file=credentials.yml manifests/athens.yml

This will deploy a single Athens instance in the athens deployment with a persistent disk of 1024MB. The IP address of that instance can be obtained with

bosh -d athens instances

which is useful for targeting Athens, e.g. with the GOPROXY variable. You can follow this quickstart guide for more information.

Managing private repos with .netrc files

Authenticate private repositories via .netrc

1. Create a .netrc file that looks like the following:

   machine <ip or fqdn>

   login <username>

   password <user password>

2. Tell Athens through an environment variable the location of that file

   ATHENS_NETRC_PATH=<location/to/.netrc>

3. Athens will copy the file into the home directory and override whatever .netrc file is in home directory. Alternatively, if the host of the Athens server already has a .netrc file in the home directory, then authentication should work out of the box.

Authenticate Mercurial private repositories via .hgrc

1. Create a .hgrc file with authentication data

2. Tell Athens through an environment variable the location of that file

   ATHENS_HGRC_PATH=<location/to/.hgrc>

3. Athens will copy the file into the home directory and override whatever .hgrc file is in home directory. Alternatively, if the host of the Athens server already has a .hgrc file in the home directory, then authentication should work out of the box.