x/playground: Deploy Playground Automatically

[toothrot]

We want to reduce toil and ensure that we’re running the latest Playground version for play.golang.org. The current process involves manual steps, which can be error-prone, forgotten, or interruptive. After this work, changes to Playground or a new Go release should be available on play.golang.org shortly after they’re made.

In either scenario below, we need to ensure that we're always deploying playground built on the latest stable release of Go. It's possible to get this information from maintner.

We have a number of other services that could benefit from CI/CD, but we should focus on solving this issue for playground for the moment, and see what we learn.

Deploying the latest Playground:

It’s possible to configure this flow entirely within Cloud Console and a cloudbuild.yaml file.

• Grant App Engine access to Cloud Build
• Create a cloudbuild.yaml file in the playground repo
  • Add step for gcloud app deploy
  • Configure build timeout to avoid currently documented issue
• Create a Cloud Build trigger, and link to github.com/golang/playground
  • The user authorizing this trigger needs to be in the golang github org, and an admin of the playground repo on github.
• Convert GO_VERSION in the playground Dockerfile to an ARG with a default value

Our docker build step must populate the latest GO_VERSION when it builds.

There are other options available for pushing directly from gerrit, but would involve additional manual configuration, as well as figuring out where to store it and document it.

Deploying Playground with the latest Go release

This update flow requires determining the latest stable Go release after it happens, then building deploying Playground with the new release. This workflow more involved, as Cloud Build doesn't provide a way to add the business logic that we want out of the box: mainly that we want to trigger based on the latest stable tagged release of a different repository, and not accidentally downgrade (say, if 1.11.x is tagged after 1.12.x)

If either of these workflows feels too difficult to configure, or feels fragile, we should try a gerrit based flow instead.

cloudbuild.yaml reference: https://cloud.google.com/cloud-build/docs/build-config
App Engine automation quickstart: https://cloud.google.com/source-repositories/docs/quickstart-triggering-builds-with-source-repositories
Cloud Build API for manual (or programmatic) triggers: https://cloud.google.com/cloud-build/docs/running-builds/start-build-manually

[gopherbot]

Change https://golang.org/cl/182180 mentions this issue: playground: parameterize GO_VERSION in Dockerfile

[toothrot]

After some experimentation, a GitHub Cloud Build Trigger seems to support our requirements.

When a new version of Go is released, we push tags, then build and push releases (see x/build: cmd/releasebot/main.go). We can configure a trigger on Cloud Build for new tags being pushed to GitHub for the golang/go repository. The trigger can whitelist to tags matching a regex (for example /^go[0-9](\.[0-9]+)+$/). Cloud Build can then run arbitrary steps to build and release Playground.

The Cloud Build steps would look something like:

• Clone the latest master branch of Playground
• Determine the latest, stable release of Go
• Build Playground docker image
• Deploy Playground to AppEngine

It is possible that this trigger would fire before the Go binaries have been released. The Playground Dockerfile may need to be updated to build Go from source instead of fetching the release. As Playground already modifies and rebuilds Go, this should be a minor increase in complexity.

Determining the latest, stable release of Go could be done a number of ways. One option is to use git ls-remote --tags, filtering and sorting by version, such as by sort -Vr. Another option is to create a new command in x/build/cmd that either lists releases from Maintner, or acts as a Maintner API CLI.

Gerrit Cloud Build triggers do not support the same level of filtering as the GitHub triggers. Filtering is important, as deploying Playground on every single push to Go would be excessive.

Another option entirely would be to run a daemon to poll for new releases of Go, triggering releases manually. I would prefer to use Could Build if we can to reduce maintenance overhead.

[toothrot]

Another thought: we could build and release Playground daily as well, using whatever the latest release of Go is. This would avoid the issue of triggers entirely. Building and deploying daily (on working weekdays) is a good continuous delivery practice, so it is a solid fallback option.

[toothrot]

@dmitshur Initial testing for tag-based triggers works great.

If we're using a tag based trigger to start the Cloud Build, and rely on maintner to tell us what the latest release is, what is the delay between the tag being pushed to github and being able to query it from maintner?

[dmitshur]

A tag pushed to Gerrit should be available in maintner within a few seconds or so.

[toothrot]

Great, so we'll be able to move forward with a command to fetch the latest go tag from maintner, and use that to build the playground in a GCB triggered build.

[gopherbot]

Change https://golang.org/cl/182981 mentions this issue: cmd/latestgo: print the latest go version tag