Once a month, we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news in the DevOps industry, and important security updates. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.
Hello Grunts,
In the last month, we launched Gruntwork Compliance for the CIS AWS Foundations Benchmark, added cluster autoscaling support to our EKS modules, open sourced two new modules for setting up CI / CD with Google Cloud Build, used Terraform 0.12 features to consolidate and simplify the code for module-ecs, package-lambda, and module-load-balancer, and announced the dates for our winter holiday break. In other news, Helm 3 has been released, EKS now supports managed worker nodes, the ALB supports weighted routing, and Terraform 0.11 provider support is being deprecated.
As always, if you have any questions or need help, email us at support@gruntwork.io!
Motivation: The AWS Foundations Benchmark is an objective, consensus-driven guideline for establishing secure infrastructure on AWS from the Center for Internet Security. Several of our customers were interested in improving their security posture by achieving compliance with this benchmark, but meeting all the recommendations—from IAM to VPCs and CloudTrail to AWS Config—from scratch proved to be challenging.
Solution: We’re excited to announce Gruntwork Compliance for the CIS AWS Foundations Benchmark! We’ve made a number of changes that will make it possible for you to achieve compliance in days, not months:
What to do about it: Check out our announcement blog post and the compliance product page for all the details. If you’re interested in becoming a Gruntwork Compliance customer, contact us!
Motivation: While Kubernetes natively supports autoscaling of the Pods that are deployed on a cluster, additional work was needed to setup autoscaling of the worker nodes. This meant that when you ran out of capacity on your EKS clusters such that you couldn’t schedule any more Pods, the only solution was to manually scale out more nodes in the cluster.
Solution: We’re happy to announce that we have added support for setting up cluster-autoscaler for use with EKS in terraform-aws-eks! This monitors Pod scheduling activities on your EKS cluster and automatically scales up more worker nodes if the cluster needs more capacity to schedule Pods. Additionally, the cluster-autoscaler application will automatically scale down worker nodes that are underutilized!
Other EKS updates:
var.enabled_metrics variable on the eks-cluster-workers module.eks-alb-ingress-controller . You can now also set Pod priority class and configure liveness and readiness probes in the controller.What to do about it: Update your EKS worker module calls to v0.9.7 and deploy the new eks-k8s-cluster-autoscaler module to try it out!
Motivation: Due to limitations with Terraform that prevented usage of dynamic inline blocks, we had to break up the ECS service code into 4 modules: one for use with either a CLB or no load balancer, one for use with an ALB or NLB, one for use with Service Discovery, and one for use with Fargate. If you needed some combination of these features, it wasn’t possible. Moreover, maintenance was more difficult, and some of the modules started to drift apart in terms of what features they offered.
Solution: With Terraform 0.12, we now have the ability to dynamically set inline blocks on resources. As a result we were able to consolidate all the separate ecs-service flavors into a single ecs-service module that can be configured for any of the different scenarios. Starting with v0.16.0, module-ecs will now only ship one ecs-service module.
A nice side effect of this is that all 4 use cases now have feature parity:
ecs-service. Also, the module now outputs service_app_autoscaling_target_resource_id, which can be used for creating auto scaling policies.awsvpc network mode in all flavors. Specifically, the subnets and security groups of the ECS task are now configurable. Previously this was only available on ecs-fargate and ecs-service-with-discovery.ecs-service and ecs-service-with-alb.ecs-service-with-discovery.However, the consequence of this is that this is a massive backwards incompatible change. Due to the differences in naming of resources across the three modules (especially IAM resources), there is no way to avoid redeploying the ECS service when upgrading to the consolidated version.
Refer to the migration guide in the release notes for more information on how to upgrade.
What to do about it: Follow the migration guide in the release notes and update to module-ecs v0.16.1 to try out the consolidated ecs-service module!
Motivation: Just as with module-ecs, the lack of support for dynamic inline blocks in older Terraform versions led to a lot of code duplication in our package-lambda and module-load-balancer modules.
Solution: We’ve updated the modules to take advantage of Terraform 0.12’s support of dynamic inline blocks!
idle_timeout was only defined on alb with logs; additional_security_group_ids was only being used on alb with logsmodules/lambda and modules/lambda-edge, taking advantage of the TF12 features that allow it. This allows for better maintainability of the modules.What to do about it: Update to the new module versions above and let us know how they work for you!
Motivation: In May, in partnership with Google, we open-sourced a collection of reusable modules for Google Cloud Platform (GCP), including modules for Kubernetes (GKE), VPCs, Cloud SQL, Cloud Load Balancer, and more. These modules make it much easier to deploy and manage infrastructure on GCP, but lacked a way to setup an automated CI/CD pipeline.
Solution: We’ve open-sourced two new repos that show how you can set up an automated CI/CD pipeline using Google Cloud Build, a Google Kubernetes Engine (GKE) cluster and either a Google Cloud Source Repository or a GitHub repo.
What to do about it: Try out the two repos above and let us know how they work for you!
Motivation: At Gruntwork, we are a human-friendly company, and we believe employees should be able to take time off to spend time with their friends and families, away from work.
Solution: The entire Gruntwork team will be on vacation December 23rd — January 3rd. During this time, there may not be anyone around to respond to support inquiries, so please plan accordingly.
What to do about it: We hope you’re able to relax and enjoy some time off as well. Happy holidays!
k8s-namespace and k8s-namespace-roles with the behavior of create_resources. Starting with terraform v0.12.11, referencing resource index 0 of those that were not created with count = 0 is an error where as before it was null. This means that we need to add a conditional for referencing those resources to only index into it when count > 0. This release also adds the create_resources parameter to k8s-service-account.local-exec calls for k8s-tiller have been updated to be compatible with Windows. Note that this requires Powershell. Note that this is a backwards incompatible release. Refer to the migration guide in the release notes for more info.v0.6.0, where kubergrunt expects base64 encoded data, but we decode the data before passing it through.helm.RenderTemplate now supports specifying the chart namespace. The function will now use the namespace of the provided KubectlOptions struct to render the template with a chosen namespace.docker.Build and docker.Run functions to run docker build and docker run, respectively, at test time.helm.RenderTemplate now requires a releaseName to be passed. This allows you to check values that depend on the release name.--terragrunt-include-dir and --terragrunt-exclude-dir glob pattern matching to exact matching.terragrunt output call happening at once. This mitigates the issues #904 and #906.PAY_PER_REQUEST billing mode.terragrunt.hcl config using the new download_dir attribute.--dry-run flag to list resources to nuke.v0.7.4 with attach-eni, where some error messages were being swallowed in the script.ecs-fargate now supports ECS Service Discovery.elasticsearch-alarms module to include all AWS recommended alarms, as documented at https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/cloudwatch-alarms.html.asg-disk-alarms module now allows you to configure how to handle the INSUFFICIENT_DATA state via the treat_missing_data input variable.memcached version now sets the default version to 1.5.16. We were using 1.4.24 before, but that's no longer supported. If you wish to use a different version, use the memcached_version input variable. The redis module now sets the default version to 5.0.5. We were using 2.8.24 before, but that's now quite out of date. If you wish to use a different version, use the redis_version input variable.redis module now allows custom names for subnet and security group.memcached and redis modules, we removed the bastion_host_security_group_id variable and added an allow_connections_from_security_groups variable, so you can now pass in a list of security group IDs that can connect to your cache, rather than just one.aurora module now supports custom names for db subnets and security groups. We also made the tests more resilient to certain failures.What happened: A major new release of Helm, version 3.0.0, is now generally available.
Why it matters: The major new features in Helm 3 are:
What to do about it: We will be updating the Infrastructure as Code Library and Reference Architecture with support for Helm 3 as soon as we can. We will announce when this is ready.
What happened: AWS has added support for provisioning managed worker nodes for EKS.
Why it matters: Originally, EKS supported a fully managed control plane, but you had to run and manage the worker nodes yourself. Now, AWS can run the worker nodes for you automatically too, including handling node updates and terminations, gracefully draining nodes to ensure your applications stay available.
What to do about it: We will be updating terraform-aws-eks with support for managed worker nodes as soon as we can. We will announce when this is ready.
What happened: Amazon’s Application Load Balancers (ALBs) now support weighted target groups. For example, if you have two target groups, you could assign one a weight of 8 and the other a weight of 2, and the load balancer will route 80% of the traffic to the first target group and 20% to the other.
Why it matters: This gives more fine-grained control over ALB routing, which can be used to implement features such as blue/green deployments and canary deployments. For example, to do a blue/green deployment, you would start with your old code (e.g., running in an ASG) in a “blue” target group with weight 1. You would then deploy your new code (e.g., into a new ASG) and register it with a new “green” target group with weight 0. Once the green target group is passing all health checks, you can switch its weight to 1 and the blue target group’s weight to 0, and all traffic will switch to your new code. You can then undeploy the old code.
What to do about it: Weighted target groups are not yet supported in Terraform; follow this issue for progress. We will be looking into this feature to add support for blue/green deployment to our ASG, ECS, and EKS code. We will announce when this is ready.
What happened: AWS has released version 2 of the EC2 Instance Metadata Service.
Why it matters: The new version of the EC2 Instance Metadata Service protects every request via session authentication. This helps to mitigate open firewalls, reverse proxies, and SSRF vulnerabilities.
What to do about it: Check out the announcement blog post for all the details. We’ll be looking into this functionality to see if/when we should support it in our modules.
What happened: HashiCorp has announced that they will be deprecating Terraform 0.11 support for Terraform providers.
Why it matters: This change has two effects:
terraform init) with Terraform 0.12. So if you want any of the new features in your providers (e.g., support for a new AWS service announced at the upcoming re:Invent), you’ll have to upgrade to 0.12.What to do about it: If you haven’t already, it’s time to upgrade to Terraform 0.12. Check out our migration guide for instructions.
What happened: On September 3rd, AWS announced improvements to the way AWS Lambda works with VPCs. These improvements introduced what is called a Hyperplane ENI that acts as a single entrypoint for multiple Lambda execution environments into your VPC. However, this change was not compatible with the way Terraform queries for ENIs when destroying network resources for Lambda. This caused terraform operations to fail with DependencyViolation errors. You can read more about this in the relevant Github issue for the AWS Provider.
What to do about it: This has been fixed in the Terraform AWS Provider version 2.31.0 or later. We recommend upgrading your AWS provider versions. You can use the version property on provider blocks to force terraform to use a version later than 2.31.0 :
provider "aws" {
version = "~> 2.31"
}