Once a month, we send out a newsletter to all Gruntwork customers that describes all the updates we’ve made in the last month, news in the DevOps industry, and important security updates. Note that many of the links below go to private repos in the Gruntwork Infrastructure as Code Library and Reference Architecture that are only accessible to customers.

Hello Grunts,

In the last month, we created a set of reusable modules to run your own ELK Stack (Elasticsearch, Logstash, Kibana) in AWS, released our comprehensive guide to authenticating to AWS on the CLI, and fixed a number of bugs. In other news, HashiCorp has released a blog post series outlining the powerful new features coming in Terraform 0.12, AWS has added support for redirects and fixed-content responses to the ALB, and Jenkins has another severe security vulnerability.

As always, if you have any questions or need help, email us at support@gruntwork.io!

Gruntwork Updates

ELK Package

Motivation: Several of our customers wanted to run the ELK stack—Elasticsearch, Logstash, and Kibana—but could not use Amazon’s hosted Elasticsearch Service due to a number of limitations, including:

  1. You can’t install custom Elasticsearch plugins.
  2. Authentication options are limited.
  3. Configuration options are limited.
  4. Monitoring options are limited.
  5. No support for in-place upgrades.
  6. You can only run backups once per day.
  7. Various other problems you can read about here and here.

Solution: We created a set of reusable modules that allow you to deploy and run your own ELK cluster in AWS! These modules can be combined configured in a variety of ways, such as the following architecture:

We’ve built modules to do the following:

What to do about it: All of this new code is in the package-elk repo. If you’re a Gruntwork subscriber, email us at support@gruntwork.io and we’ll grant you access (and if you’re not a subscriber, sign up now)! package-elk consists of a number of standalone modules that can be mixed and matched as you see fit. See the examples on how to deploy a full end to end ELK pipeline with all components included.

A Comprehensive Guide to Authenticating to AWS on the Command Line

Motivation:**** Logging into your AWS account on the web is fairly straightforward: you type in a username and password and you’re done. Logging into your AWS account on the command line — so you can use CLI tools such as aws, terraform, packer, and so on — is much harder. It’s so bad that “how do I access my AWS account?” is the #1 support ticket we get at Gruntwork!

Solution: We’ve put together a blog post series to walk you through the different ways to authenticate to AWS on the command-line:

  1. An Intro to AWS Authentication
  2. Authenticating to AWS with the Credentials File
  3. Authenticating to AWS with Environment Variables
  4. Authenticating to AWS with Instance Metadata
  5. Authenticating to AWS with Gruntwork Houston

What to do about it: Read through the blog post series and let us know if you find it helpful or still have questions! Also, if you’d like access to the private beta of Gruntwork Houston, email us at info@gruntwork.io.

Open source updates

Other updates

DevOps News

Terraform 0.12 preview

What happened: Terraform 0.12 is coming soon, and bringing with it a number of major new changes. HashiCorp has released a series of blog posts describing these changes:

Why it matters: These changes make Terraform more powerful, consistent, and predictable as a language. Here are a few of the highlights:

First-class expressions mean you don’t have to wrap all expressions with quotes and curly braces ("${}"), so code that used look like this:

resource "aws_instance" "example" {
ami           = "${var.ami}"
instance_type = "${var.instance_type}"
}

Now looks like this:

resource "aws_instance" "example" {
ami           = var.ami
instance_type = var.instance_type
}

The for and for-each syntax enable a lot of powerful new capabilities, including dynamic inline-blocks:

resource "aws_autoscaling_group" "example" {
# ...

dynamic "tag" {
for_each = local.standard_tags

content {
key                 = tag.key
value               = tag.value
propagate_at_launch = true
}
}
}

Conditional operator improvements mean that the ternary syntax is now short circuiting and supports lists and maps:

buckets = (var.env == "dev" ? [var.foo, var.bar] : [var.baz])

And you can finally mark arguments as “omitted” via null to get the behavior of their default values:

variable "override_private_ip" {
type    = string
default = null
}
resource "aws_instance" "example" {
# ... (other aws_instance arguments) ...

private_ip = var.override_private_ip
}

The rich value types will allow you to define explicit types for your module’s inputs:

variable "networks" {
type = map(object({
network_number    = number
availability_zone = string
tags              = map(string)
}))
}

And pass entire resources as inputs or outputs to other modules:

output "vpc" {
value = aws_vpc.example
}

What to do about it: Terraform 0.12 is still in preview mode. Once it approaches a full release, we will update all of our modules, and send upgrade instructions. In the meantime, sit tight!

ALB now supports redirects and fixed responses

What happened: The Application Load Balancer (ALB) now supports redirects and fixed responses.

Why it matters: You can now add listener rules to your ALB to tell it, for example, to redirect /foo to /bar, or to redirect all HTTP traffic to HTTPS. You can also have static responses (e.g., 200 OK) for specific URLs.

What to do about it: The aws_lb_listener_rule resource in Terraform does not yet support redirect or fixed-response actions. Follow this issue to see when this new functionality will be available.

Security Updates

Below is a list of critical security updates that may impact your services. We notify Gruntwork customers of these vulnerabilities as soon as we know of them via the Gruntwork Security Alerts mailing list. It is up to you to scan this list and decide which of these apply and what to do about them, but most of these are severe vulnerabilities, and we recommend patching them ASAP.

Jenkins