Once a month, we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news in the DevOps industry, and important security updates. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.
Hello Grunts,
Last month, we created and open sourced modules for running Couchbase in AWS, updated our security code to make it possible to assume IAM Roles for a longer time, added migration and deployment docs for the Reference Architecture, added a keep-warm module to fix the “cold start” problem with Lambda functions, and fixed a bunch of bugs. In other news, Amazon EKS is now generally available and Microsoft is acquiring GitHub.
As always, if you have any questions or need help, email us at support@gruntwork.io!
Motivation: The Couchbase team wanted to make it 10x easier for users to get up and running with Couchbase on AWS.
Solution: We created the terraform-aws-couchbase module, which is open source on GitHub, and available in the Terraform Registry! This module makes it easy to spin up a Couchbase cluster in just a few lines of Terraform code:
provider "aws" {
region = "us-east-1"
}module "couchbase" {
source = "gruntwork-io/couchbase/aws"
version = "0.1.0"
}output "couchbase_web_console_url" {
value = "${module.couchbase.couchbase_web_console_url}"
}output "sync_gateway_url" {
value = "${module.couchbase.sync_gateway_url}"
}Put the code above into a file called main.tf, run terraform init and terraform apply, and in a few minutes, you’ll have a Couchbase cluster running in AWS!
For more info and examples, check out Get Couchbase running on AWS in 5 minutes.
What to do: Give the Couchbase modules a try and let us know how they work for you!
Motivation: Many of our customers assume IAM Roles to access their AWS accounts. The default max expiration time for IAM Role credentials used to be just 1 hour, so you’d have to login again and again throughout the day. AWS has finally added support for configuring the max expiration time for IAM role credentials and our customers wanted to be able to take advantage of that.
Solution: We made two changes in module-security:
cross-account-iam-roles module now sets a default max expiration of 12 hours for IAM Roles intended for human users (e.g., allow-read-only-access-from-other-accounts) and a default max expiration of 1 hour for IAM Roles intended for machine users (e.g., allow-auto-deploy-access-from-other-accounts). Both of these expiration values are configurable via the new input variables max_session_duration_human_users and max_session_duration_machine_users.aws-auth script now accepts optional --mfa-duration-seconds and --role-duration-seconds parameters that specify the session expiration for the creds you get back when authenticating with an MFA token or assuming an IAM role, respectively. The default for both of these has been set to 12 hours to be more human-friendly.What to do about it: If you want to take advantage of these more friendly IAM Role expiration periods, update your AWS accounts to use module-security, v0.10.0, install the latest aws-auth, and enjoy all the time you’ve saved by not having to re-authenticate 8 times per day!
Motivation: After deploying the Reference Architecture for our customers, many asked us for documentation on how to migrate their apps to it, and how to deploy it from scratch.
Solution: We’ve added the docs! All future Reference Architecture deployments will include detailed migration and deployment instructions. For existing customers, you can find the same docs as part of the Acme sample Reference Architecture repos. For example, here are the ones from the multi-account Acme Reference Architecture:
What to do about it: We hope you find the new docs useful! If anything is missing or unclear, please let us know by emailing support@gruntwork.io, and we’ll improve it!
Motivation: Building infrastructure on top of AWS Lambda is great, as you don’t have to manage servers or worry much about scalability or high availability. However, if you have a Lambda function where latency is important (e.g., you’re using Lambda + API Gateway for a web service), and that function is not invoked for a while, the next invocation may be hit with the overhead of a “cold start,” adding from a few hundred milliseconds all the way up to 15 extra seconds to that function’s execution time!
Solution: We created a new module called keep-warm which you can use to invoke your Lambda functions on a scheduled basis, effectively keeping those functions “warm” so that you can avoid cold starts. For example, here’s how you can invoke a Lambda function called example-func every 5 minutes, passing it the event object {"foo": "bar"} on each invocation:
module "keep_warm" {
source = "git::git@github.com:gruntwork-io/package-lambda.git//modules/keep-warm?ref=v0.2.2"name = "keep-warm"
schedule_expression = "rate(5 minutes)"
concurrency = 3function_to_event_map = {
example-func = {
foo = "bar"
}
}
}Note that the example above sets the concurrency level to 3. This tells the keep-warm module to invoke the example-func Lambda function 3 times concurrently each time it runs, thereby keeping 3 containers warm at the same time, rather than just one. See I’m afraid you’re thinking about AWS Lambda cold starts all wrong for more info on why concurrency is important.
What to do about it: If you have latency-sensitive Lambda functions (e.g., web services built with API Gateway and Lambda) with spiky, intermittent traffic, give the keep-warm module a shot for avoiding the cold start problem.
Motivation: One of our customers was trying to assume an IAM Role in the Reference Architecture that allows them to request an OpenVPN Cert in another AWS account, but due to a bug, it wasn’t working.
Solution: We found a typo in the IAM Group names the Reference Architecture creates for using OpenVPN in other accounts. The IAM Role name was set to openvpn-server-allow-certificate-requests-for-external-accounts instead of openvpn-allow-certificate-requests-for-external-accounts.
What to do about it: If you are using the multi-account Reference Architecture, fix the IAM Role name in security/_global/iam-groups/terraform.tfvars. We’ve updated the Acme Multi-Account example with the fix, so you can use this commit as a reference.
NewAuthenticatedSessionFromRole and CreateAwsSessionFromRole.retry.DoWithRetry methods were not returning stdout/stderr. Fix a bug where the retry mechanism in terraform.Apply was checking the error message rather than stdout/stderr.test_structure.CopyTerraformFolderToTemp would fail if called from inside of a subtest, such as t.Run("SubtestcaseName", func(t *testing.T) {...})Upgrade flag in terraform.Options to tell Terratest to set the -upgradeflag when running terraform init.vault-cluster module now requires a new parameter called allowed_inbound_security_group_count to be set. This is unfortunately required to work around a limitation in Terraform with the count parameter.cluster_addr == api_addr, so default api_addr to a different value, and make it configurable via a new --api-addr parameter.--s3-backend-enable but it should actually be --enable-s3-backend.--environment option with the run-consul script to set environment variables that should be exposed to Consul in its supervisord config.--environment option with the run-nomad script to set environment variables that should be exposed to Nomad in its supervisord config.server-group module now cleans up properly if a single server fails to pass health checks, exposes new vars to configure the rolling deploy timeout and add additional security groups, and updates to the latest version of Boto.asg-rolling-deploy that would cause the error argument --tag-value: expected one argument.kms_key_id parameter for the server-group module to enable EBS Volume encryption with your own Customer Master Key (CMK). This will also automatically create an IAM Policy that gives the server access to that CMK.modules/pre-commit are now in their own open source repo: https://github.com/gruntwork-io/pre-commit. Please update your .pre-commit-config.yml files to point to the new repo and its version numbers.git-add-commit-push has been moved from the gruntwork-module-circleci-helpers module to the git-helpers module. terraform-update-variable now depends on git-helpers being installed, as it uses git-add-commit-push under the hood to be able to more reliably commit and push changes.git-add-commit-push script no longer defaults the branch name to $CIRCLE_BRANCH. Instead, it uses git to look up the name of the currently checked-out branch in pwd.aurora module, you can now use the db_instance_parameter_group_name param to set the parameter group for instances separately from the parameter group for the entire cluster (which can be set via the db_cluster_parameter_group_name param).rds module now exposes a publicly_accessible parameter that you can set to true to make the DB accessible from the public Internet (NOT recommended for most use cases).nlb module now exposes a subnet_mapping_size parameter that you use to specify how many items there are in the subnet_mapping list. This fixes the issue that arises when using resources or module outputs as values in the maps within the subnet_mapping list.auto-update, ntp, fail2ban, and ip-lockdown modules now all use bash-commons under the hood. That means you must install bash-commons before installing any of those other modules. As a result, the auto-update and ntp modules now support Amazon Linux 2. We will add Amazon Linux 2 support for fail2ban and ip-lockdown modules in the future.ecs-service-with-alb using the alb_target_group_name parameter.What happened: Amazon Elastic Kubernetes Service (EKS) is now generally available in us-east-1 and us-west-2. Moreover, HashiCorp has already launched Terraform support for EKS.
Why it matters: Running Kubernetes yourself is a significant undertaking, so being able to take advantage of a managed service that does it for you can be a huge win. Up until now, EKS was only available in early preview mode, but now everyone should be able to use it. However, it’s still very early days for EKS, and there are a number of limitations to be aware of: it costs a minimum of $150/month just for the master nodes, it’s only available in us-east-1 and us-west-2, the provisioning process is strange (using CloudFormation under the hood), PrivateLink is not supported yet, HIPAA compliance is not supported yet, no integration with Fargate yet, and so on (see reddit and HN for discussion). AWS will be fixing all of these in the future, but for now, buyer beware!
What to do about it: We will be diving deep into EKS over the next month or two and will report back on what we find. In the meantime, feel free to play around with it and let us know what you think!
What happened: Microsoft has agreed to acquire GitHub for $7.5 billion.
Why it matters: A huge congrats to our friends at GitHub! We love the product, use it for all of our private and open source projects, and we are excited to see what the future holds.
What to do about it: Keep using GitHub and pushing Microsoft to make it even better.
What happened: You can now use ECS Service Discovery with ECS tasks that use the bridge and host networking modes.
Why it matters: Previously, ECS Service Discovery only supported tasks launched using the awsvpc networking mode, which assigns each running task a dedicated elastic networking interface.
What to do about it: Try it out on all your ECS Tasks! We’ll be adding support for it to module-ecs soon.
What happened: Amazon Aurora now has a new new “backtrack” feature that lets you roll back the database to any point in time, down to the granularity of seconds.
Why it matters: This feature is essentially an “undo” for when you screw something up in your database, such as deleting or corrupting critical data. Under the hood, Aurora stores every change to your database in a distributed, log-structured system, and is able to rollback to any previous log entry very quickly.
What to do about it: If you’re using Aurora, consider enabling this feature as an extra layer of protection (see the docs for more info). This functionality is now supported by Terraform and we will be updating module-data-storage with support for it soon.
What happened: The Application Load Balancer (ALB) now supports authentication of user requests when accessing web-based applications through social Identity Providers such as Facebook, Google, and Amazon or enterprise Identity Providers such as ActiveDirectory and PingFederate.
Why it matters: You can now offload authentication logic to the ALB and and Amazon Cognito instead of building it yourself. This allows your users to authenticate through any identity provider that supports OpenID Connect protocol or SAML2.0 or social identity providers such as Facebook, Google, and Amazon.
What to do about it: If you’d like to try out ALB user authentication, check out this blog post for a guide.
What happened: Amazon EC2 now allows you to control not only the instance type (e.g., m4.large), but also to customize how many vCPUs and how many threads per vCPU the instance will have access to.
Why it matters: You now have more flexibility and control over how many CPU cores and threads per CPU core your EC2 instances can access.
What to do about it: Check out the Optimize CPU docs for details.
What happened: Terraform now has a new filesystem provider that provides a cross-platform way to read and write files.
Why it matters: Before, you could only use the file interpolation function, which only worked for files that already existed, and was only useful for reading files. With the filesystem_xxx resources, you can read and write files on the fly, create directories, set file permissions, and more.
What to do about it: Check out the terraform-provider-filesystem docs for details.
Below is a list of critical security updates that may impact your services. We notify Gruntwork customers of these vulnerabilities as soon as we know of them via the Gruntwork Security Alerts mailing list. It is up to you to scan this list and decide which of these apply and what to do about them, but most of these are severe vulnerabilities, and we recommend patching them ASAP.