Version management

This article explains how Spacelift can help you manage Terraform versions, as well as how you can help Spacelift achieve it

Intro to Terraform versioning

Terraform is an actively developed open source product with a mighty sponsor. This means frequent releases - in fact, over the last few months (since minor version 0.12) we've been seeing nearly weekly releases. While that's all great news for us as Terraform users, we need to be aware how version management works in order not to be caught off-guard.

In plain Terraform, version management philosophy is pretty straightforward - within a minor version (0.11, 0.12 etc.), backwards compatibility is always maintained. Terraform configs written for 0.12.1 should still work in 0.12.16. However, that does not apply to the state file.

Once the state is written to (applied) with a higher version of Terraform, there is no way back.

The backwards compatibility also does not work between minor versions. For example, the most recent migration from 0.11 to 0.12 affected a few core elements and required a rewrite not only of .tf files, but also providers. While officially supported providers were more or less quick to update, some are still incompatible and we know of a good few organizations that haven't yet made the transition, or aborted the transition that was under way.

Terraform versions in Spacelift

Terraform binaries neither distributed with Spacelift, nor with its runner Docker image. Instead, Spacelift dynamically detects the right version for each workflow (run or task), downloads the right binary on demand, verifies it and mounts it as read-only on the runner Docker container as /bin/terraform to be directly available in the runner's $PATH:

There are two ways to tell Spacelift which Terraform version to use. The one we advise is to set it through runtime configuration. The second one - only available to programmatically created stacks - allows setting the initial version directly on the stack. In this case, we advise to ignore any subsequent changes to make migrating to newer versions painless.

In order to determine the version of the Terraform binary to mount, Spacelift will first look at the runtime configuration. If it does not contain the version setting for the current stack, the stack setting is then considered. If there is no version set on the current stack, the newest supported Terraform version is used. We always advise starting stacks with the newest available version of Terraform, though we realise it may not be the best option if the project is imported from elsewhere or incompatible providers are used.

Newest Terraform version supported by Spacelift may lag a bit behind the latest available Terraform release. We err on the side of caution and thus separately verify each version to ensure that it works as expected and that our code is compatible with its output and general behaviour. We're trying to catch up roughly within a week but may temporarily blacklist a faulty version. If you need a compatibility check and a bump sooner than that, please file a ticket.

Once we apply a run with a particular version of Terraform, we set it on the stack to make sure that we don't implicitly attempt to update it using a lower version.

Migrating to newer versions

In order to migrate a stack to a newer version of Terraform, we suggest opening a feature branch bumping the version through runtime configuration. Open a Pull Request in GitHub from the feature branch to the tracked branch to easily get a link to your proposed run and see if everything looks good. If it does, merge your Pull Request and enjoy working with the latest and greatest version of Terraform. Otherwise, try making necessary changes until your code is working or postpone the migration until you have the bandwidth to do so.

In general, we suggest to try and keep up with the latest Terraform releases. The longer you wait, the more serious is the migration work going to be. Terraform evolves, providers evolve, external APIs evolve and so should your code.