Update: Terraform and Terragrunt have both evolved considerably since this blog post was written, so some of this content is out of date! We instead recommend that you read our updated blog post Terragrunt: how to keep your Terraform code DRY and maintainable to see the new role of Terragrunt in 2019 and beyond.
Terraform is an awesome tool. It allows you to define all of your infrastructure as code, works with any cloud provider, and has a massive open source community.
But no tool is perfect. Every time Terraform creates a bunch of infrastructure for you, it records the resources it created in a Terraform state file (.tfstate) so it can manage and update those resources later. This leads to two problems:
1. Configuring Terraform Remote State. Terraform supports the concept of “remote state,” where you can store state in a shared location, such as an S3 bucket, consul, or etcd. This is a great way to collaborate with Terraform, but developers often forget to enable remote state and end up creating a bunch of duplicate resources, or they enable remote state, but mess up the configuration and end up changing the wrong resources and then overwriting the wrong remote state.
2. Remote State Locking. Although remote state allows multiple team members to share the same state files, it does not provide locking, so if two team members run Terraform at the same time, they may overwrite each other’s changes. Ideally, Terraform would include a locking mechanism, but instead the Terraform docs explicitly say locking isn’t supported and that if you want it, you should use Atlas (now known as Terraform Enterprise), a paid solution.
To solve these problems, we created a tool called Terragrunt, and we are pleased to announce that it’s now open source:
https://github.com/gruntwork-io/terragrunt
Terragrunt is a thin wrapper for Terraform that manages remote state for you automatically and provides locking by using Amazon DynamoDB, which should be free for most teams.
Let’s walk through how to use Terragrunt step-by-step.
# Configure Terragrunt to use DynamoDB for locking
dynamoDbLock = {
stateFileId = "my-app"
}
# Configure Terragrunt to automatically store tfstate files in S3
remoteState = {
backend = "s3"
backendConfigs = {
encrypt = "true"
bucket = "my-bucket"
key = "terraform.tfstate"
region = "us-east-1"
}
}Be sure to update the “bucket”, “key” and “region” properties to correspond to an S3 Bucket for which you have permissions. Also, be sure you have full permissions on DynamoDB. If you need to use restricted permissions on DynamoDB, see the Terragrunt docs on how to specify a specific DynamoDB region and table name.
terragrunt get
terragrunt plan
terragrunt apply
terragrunt output
terragrunt destroyTerragrunt forwards almost all commands, arguments, and options directly to Terraform, using whatever version of Terraform you already have installed. However, before running Terraform, Terragrunt will ensure your remote state is configured according to the settings in the .terragrunt file. Moreover, for the apply and destroy commands, Terragrunt will first try to acquire a lock using DynamoDB.
$ terragrunt apply
[terragrunt] Configuring remote state for the s3 backend
[terragrunt] Running command: terraform remote config -backend s3 -backend-config=key=terraform.tfstate -backend-config=region=us-east-1 -backend-config=encrypt=true -backend-config=bucket=my-bucket
Initialized blank state with remote state enabled!
[terragrunt] Attempting to acquire lock for state file my-app in DynamoDB
[terragrunt] Attempting to create lock item for state file my-app in DynamoDB table terragrunt_locks
[terragrunt] Lock acquired!
[terragrunt] Running command: terraform apply
terraform applyaws_instance.example: Creating...
ami: "" => "ami-0d729a60"
instance_type: "" => "t2.micro"[...]Apply complete! Resources: 1 added, 0 changed, 0 destroyed.[terragrunt] Attempting to release lock for state file my-app in DynamoDB
[terragrunt] Lock released!Terragrunt automatically configured remote state as declared in the .terragrunt file, acquired a lock from DynamoDB, ran “terraform apply”, and then released the lock. Future developers need only “git clone” the repo containing this folder and run “terragrunt apply” to achieve an identical result!
To learn more about how Terragrunt obtains a lock from DynamoDB and other details, see the Terragrunt Docs.
An obvious question is why not just submit these changes directly to Terraform as Pull Requests?
The first reason is that the locking solution is AWS-specific. At Gruntwork, we specialize in AWS, so that makes sense for us, but it’s not clear if such a feature would make sense as part of Terraform core.
The second reason is that, as a business that uses Terraform, we needed solutions for these problems immediately. A standalone tool allowed us to experiment with this functionality quickly (the initial version took only a day of work) and to vet it with customers. A Pull Request would have taken significantly longer, and we’d have to wait for a new Terraform release.
That said, we hope that the release of Terragrunt starts a conversation about these features in the Terraform community, and that someday they make it directly into Terraform core. We’d be happy to submit a Pull Request to that effect. A tool plus a wrapper is always less desirable than a single tool that does what you want.
Part of our goal in open sourcing Terragrunt is to see what other best practices Terraform users want to incorporate. For example, we recently received a request to implement remote state locking using Consul.
We’d love to hear your feedback on Terragrunt, and we hope it makes your experience with infrastructure-as-code even more pleasant. Feel free to leave a comment on this post, file an issue in the Terragrunt github repo, or email us at info@gruntwork.io with your thoughts.
Your entire infrastructure. Defined as code. In about a day. Gruntwork.io.