In Terraform, modules help you abstract away common functionality in your infrastructure.
The name of a module managed by Spacelift is of the following form:
In this name we have:
- The source module registry -
spacelift.iois used here;
- The organization which owns and maintains the module;
- The module name, this will usually be the best shorthand descriptor of what the module actually does, i.e. it could be starting a machine with an HTTP server running.
- The main Terraform provider this module is meant to work with, i.e. the provider for the cloud service the resources should be created on.
You can use a module in your Terraform configuration this way:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
As you can see, we've explicitly used a module which can make cakes using an oven. We can specify variables the module depends on, and finally use the outputs the cake module exports.
Spacelift obviously lets you host modules, but it also does much more, providing you with robust CI/CD for your modules, leading us to the question...
Why host your Modules on Spacelift?»
Spacelift provides everything you need to make your module easily maintainable and usable. There is CI/CD for multiple specified versions of Terraform, which "runs" your module on each commit. You get an autogenerated page describing your Module and its intricacies, so your users can explore them and gather required information at a glimpse. It's also deeply integrated with all the features Stacks use which you know and love, like Environments, Policies, Contexts and Worker Pools.
Setting up a Module»
Git repository structure»
You will have to set up a repository for your module, the structure of the repository should be as follows:
1 2 3 4 5 6 7
Each module must have a
config.yml file in its
.spacelift directory, containing information about the module along with any test cases. Details of the format of this file can be found in the module configuration section of this page.
You can check out an example module here: https://github.com/spacelift-io/terraform-spacelift-example
The source code for a module can be stored in a subdirectory of your repository because you can specify the project root when configuring your module in Spacelift. An example of a repository containing multiple modules can be found here: https://github.com/spacelift-io/multimodule
In order to add a module to Spacelift, navigate to the Modules section of the account view, and click the Add module button:
The setup steps are pretty similar to the ones for stacks. First you you point Spacelift at the right repo and choose the "tracked" branch - note that repositories whose name don't follow the convention are filtered out:
In the behavior section there are just three settings: administrative, worker pool and project root. You will only need to set administrative to
true if your module manages Spacelift resources (and most likely it does not). Setting worker pool to the one you manage yourself makes sense if the module tests will be touching resources or accounts you don't want Spacelift to access directly. Plus, your private workers may have more bandwidth than the shared ones, so you may get feedback faster. The project root let's you specify the module source code root inside of your repository:
Last but not least, you will be able to add a name, provider, labels and description.
The name and provider will be inferred from your repository name if it follows the
terraform-<provider>-<name> convention. However, if it can't be inferred or you want a custom name, then you can specify them directly. The final module slug will then be based on the name.
Environment, contexts and policies»
Environment and context management in modules is identical to that for stacks. The only thing worth noting here is the fact that environment variables and mounted files set either through the module environment directly, or via one of its attached contexts will be passed to each of the test cases for the module.
Attaching policies works in a similar way. One thing worth pointing out is that the behavior of Trigger policies are slightly different for modules. Instead of being provided with the list of all other accessible stacks, module trigger policies receive a list of the current consumers of the module. This allows you to automatically trigger dependent stacks when new module versions are published.
While by convention a single Git repository hosts a single module, that root module can have multiple submodules. Thus, we've created a way to create a number of test cases:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
This configuration is nearly identical to the one described in the Runtime configuration section, with both
test_defaults and each test case accepting the same configuration block. Note that settings explicitly specified in each test case will override those in the
test_defaults section. Also, notice that each test case has a name, which is a required field.
While we don't check for name uniqueness, it's always good idea to give your test cases descriptive names, as these are then used to report job status on your commits and pull requests.
In order to verify that your module is working correctly, Spacelift can run a number of test cases for your module. Note how the configuration above allows you to set up different runtime environment (Docker image, Terraform version) etc. If you want to test the module with different inputs, these can be passed as Terraform variables (starting with
TF_VAR_) through the test-level
environment configuration option - see above for an example.
While coverage is not yet calculated or enforced, we suggest that tests set up all resources defined by the module and submodules. It's generally a good idea to provide examples in the
examples/ directory of your repository showing users how they can use the module in practice. These examples can then become your test cases, and you can test them against multiple supported Terraform version to maximize compatibility.
While running each test case, Spacelift will - as usual, initialize, plan and apply the resource, but also destroy everything in the end, checking for errors. In the meantime, it will also validate that some resources have actually been created by the tests - though as for now it does not care what these are.
A test case can be marked as
negative, which means that it is expected to fail. In an example above one of the test cases is expected to fail if one of the required Terraform variables is not set. Negative test cases are as useful as positive ones because they can prove that the module will not work under certain - unexpected or erroneous - circumstances.
Test cases will be executed in parallel (as much as worker count permits) for each of the test cases version you have specified in the module configuration.
Tests run both on proposed and tracked changes. When a tracked change occurs, we create a Version. Versions are described in more detail in the next section.
Each test case will have its own commit status in GitHub / GitLab.
Test case ordering»
You can specify the order in which test cases should be executed by setting the
depends_on property on a test case. This property accepts a list of test case
ids that must be executed before the current test case. For example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Note that in order to refer to a test case, you need to set a unique
id to it.
Whenever tests succeed on a tracked change, a new Version is created based on the
module_version in the configuration. Important thing to note is that Spacelift will not let you reuse the number of a successful version, and will require you to strictly follow semantic versioning - ie. you can't go to from
Two proposed git flow are as follows:
The first one would be to have a main branch and create feature branches for changes. Whenever you merge to the main branch you bump the version and release it.
If you want more control over release schedules, you could go with the following:
- A release branch
- A main branch
- Feature branches
Whenever you add a new functionality, you may want to create a feature branch and open Pull Request from it to the main branch. Whenever you want to release a new version, you merge the main branch into the release branch.
You can also use Git push policies to further customize this.
If no test cases are present, the version is immediately marked green.
Marking versions as bad»
If you don't want people to use a specific version of your module, you can mark it as bad. Currently, this feature doesn't have any technical implications - it is still downloadable and usable, but it's a good way to communicate to your users that a specific version is not recommended.
You need to click on the version number and then click on the Mark as bad button on the right. Make sure to leave a note explaining why the version is bad.
Modules in practice»
In order to use modules, you have to source them from the Spacelift module registry. You can generate the necessary snippet, by opening the page of the specific module version, and clicking show instructions.
Unlike Stacks, modules can be shared between Spacelift accounts in a sense that while they're always managed by a single account, they can be made accessible to an arbitrary number of other accounts.
In order to share the module with other accounts, please add their names in subdomain form (all lowercase) in the module settings Sharing section:
This can also be accomplished programmatically using our Terraform provider.
Using modules outside of Spacelift»
Modules hosted in the private registry can be used outside of Spacelift.
The easiest way is to have Terraform retrieve and store the credentials by running the following command in a terminal:
After you confirm that you want to proceed, Terraform will open your default web browser and ask you to log in to your Spacelift account. Once this is done, Terraform will store the credentials in the
~/.terraform.d/credentials.tfrc.json file for use by subsequent commands.
The method above requires a web browser which is not always practical, for example on remote server with no GUI. In that case, you can use credentials generated from API keys. The credentials file generated upon the creation of each API key contains a section explaining how a key can be used to set up credentials in the Terraform configuration file (
.terraformrc). To learn more about this please refer directly to Terraform documentation.
If you want to use Dependabot to automatically update your module versions, you can use the following
1 2 3 4 5 6 7 8 9 10 11 12 13
It is important for the
url to be
https://app.spacelift.io and for the
token to be a Spacelift API key. Admin access is not required.