Caching

Private workers download tool binaries and provider plugins when they process runs. Caching these files reduces download time and limits traffic to external registries.

The configuration depends on the type of worker:

• Docker-based workers cache tool binaries automatically. You can mount a shared provider cache into run containers.
• Kubernetes workers can use shared volumes for both tool binaries and providers.

Tool binaries

Tool binaries include executables such as terraform, tofu and kubectl.

• Docker-based workers
  • Path: /opt/spacelift/binaries on the host.
  • Lifetime: The launcher reuses binaries for later runs on the same worker instance.
  • Configuration: None.
• Kubernetes workers
  • Path: /opt/spacelift/binaries_cache in the run pod.
  • Lifetime: The default volume is an emptyDir, so its contents are discarded with the run pod.
  • Configuration: To reuse binaries across runs, configure a shared volume for the cache.

Provider plugins

Terraform and OpenTofu store downloaded providers in the directory set by the TF_PLUGIN_CACHE_DIR environment variable. Later runs can reuse the providers in this directory instead of downloading them again.

To share providers across a worker pool:

1. Mount the same directory into each run container or pod.
2. Set TF_PLUGIN_CACHE_DIR to the mounted path.

You can enable the cache for every stack on a worker pool or set the environment variable only on selected stacks.

Shared storage requirements

Use a shared filesystem when workers or run pods on different nodes need access to the same cache. The filesystem must:

• Allow concurrent reads and writes from every worker.
• Make writes from one worker immediately visible to the others.
• Be writable by UID 1983, which run containers use.

Managed network filesystems such as Amazon EFS, Amazon FSx for OpenZFS, Azure Files and Google Cloud Filestore meet these requirements. However, Amazon EFS is not recommended for these caches because its per-operation latency makes access to the many small cache files too slow.

Block storage is not a shared filesystem

Multi-attach features only share a raw block device. Filesystems such as ext4 and XFS are not cluster-aware, so nodes do not see each other's writes and concurrent access can corrupt the filesystem.

To use a block volume for a shared cache, attach it to an NFS server and let workers access it through NFS.

Cache security

Provider plugins are executable files. Any stack that can write to a shared provider cache can affect other stacks that use it. Only share a cache between stacks in the same trust boundary, and use separate caches or worker pools when stacks require isolation.

Restrict network access to the filesystem and grant write access only to UID 1983. NFS with the default AUTH_SYS security uses numeric user and group IDs; it does not provide strong client authentication or encrypt traffic. If your security requirements call for authenticated and encrypted NFS, configure Kerberos with sec=krb5p or use a managed filesystem that provides equivalent controls.

Commit .terraform.lock.hcl files so that Terraform and OpenTofu can verify provider checksums against the lock file.

Dependency lock files

Terraform 1.4 and later, and OpenTofu, only use a cached provider when the stack's .terraform.lock.hcl contains a matching hash. Otherwise, the provider is downloaded again to calculate the hash. Stacks that commit their lock file can use the cache without additional configuration.

For stacks that do not commit a lock file, set TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE=true alongside TF_PLUGIN_CACHE_DIR. This accepts the tradeoff described in the Terraform documentation.

Concurrent downloads

OpenTofu uses a file lock for each provider version when writing to the cache. Concurrent downloads are safe when the shared filesystem supports flock, as NFSv4 does.

Terraform does not use the same locking. If many Terraform runs start with an empty cache, an init can fail because of a corrupted download. The next run replaces the file and succeeds. To avoid this, warm the cache with a single Terraform run before using it with a busy pool.