Introducing: The Gruntwork Module, Service, and Architecture Catalogs

The most secure, battle-tested way to go to production on AWS

Today, I’m excited to reveal the new design of the Gruntwork Infrastructure as Code Library (IaC Library):

Read on to learn how you can use these three types of catalogs to go to production on AWS reliably and securely, in days rather than months.

Motivation

Let’s say you just finished working on your new project — maybe it’s a Node.js app at a new startup or maybe it’s a set of Java microservices at a major enterprise — and after months of coding, it’s time to ship it to production. You sign up for an AWS account, log in, and you’re greeted with a page that looks something like this:

undefined

Phew. Well, I guess the good news is that you have access to a nearly endless list of tools and services. However, the bad news is that you have access to a nearly endless list of tools and services.

You figure that you probably need to deploy some servers, so you start learning about EC2 instances, Auto Scaling Groups, Amazon Machine Images, and Elastic Load Balancers. But as you’re trying to make progress with that, all you hear about is Kubernetes, Kubernetes, Kubernetes. So then you start reading about Docker Orchestration, EKS, control planes, worker nodes, auto scalers, RBAC, Istio, and Helm charts. And you’ve barely made any progress there before your teammates start yelling about the fact that Serverless is the future! So now you’re knee deep in docs about AWS Lambda, API Gateway, ECS Fargate, Serverless Framework, and Aurora.

But wait, first you have to get all the networking details in place. So you scrap everything you’ve done so far and start learning about VPCs, subnets, route tables, NAT Gateways, Network ACLs, and Transit Gateways. But before you can finish any of that, you discover something called Landing Zone. So then you’re digging through documentation about AWS Config, CloudTrail, IAM Roles, GuardDuty, Account Vending Machines… And after months of work, it dawns on you that you’ve still got to sort out a CI / CD pipeline, data storage, data backup, monitoring, log aggregation, alerting, secrets management, SSH access, VPN access, DNS, TLS, CDN, and so much more.

And here’s the crazy thing: there are a thousand other developers at a thousand other companies doing the exact same thing. Every single company that wants to go to production on the public cloud needs to solve the same set of problems: networking, orchestration, data storage, secrets management, CI / CD, monitoring, and so on. So why are you doing it all from scratch?

Perhaps you tried to use some of the open source solutions that are out there, but many of them don’t quite seem to do what you want, many don’t work at all, and virtually none of them have automated tests or docs. You start thinking to yourself, has anyone actually used this code before? In production? Is anyone maintaining these repos? Do I really want to bet my company on this code?

The reality is that, under the hood, most companies want more or less the same basic infrastructure to run their web services. You can think of it like this:

undefined

Your app is unique. The infrastructure under the hood can be almost completely generic, just so long as it is reliable, secure, tested, and maintained.

This is precisely what we do at Gruntwork. Over the last 5 years, we’ve built up a library of over 300,000 lines of reusable, battle-tested, commercially supported and maintained infrastructure code that is used in production by hundreds of companies.

The big change we’re announcing today is that the Gruntwork IaC Library now consists of three layers: modules, services, and architectures. Let’s go through these layers one at a time, starting by looking at modules.

The Gruntwork Module Catalog

The first step in going to prod is to create modules, which are the basic building blocks of your infrastructure. Each module is a single piece of reusable infrastructure defined as code: e.g., a Terraform module to deploy a server, or a load balancer, or a database. You can think of them like the furniture in a house:

undefined

For the last few years, the core of the Gruntwork Infrastructure as Code Library has been the Module Catalog, which contains hundreds of reusable, battle-tested modules that you can use to assemble your infrastructure. Here are just a few examples of the modules you’ll find in the Module Catalog:

And here’s an example of how you might use all these modules together to configure an EKS cluster. First, you create a Packer template that will build an Amazon Machine Image (AMI) for EKS worker nodes with cloudwatch-log-aggregation-scripts and ssh-grunt installed:

{
"builders": [{
"ami_name": "eks-worker-node-{{isotime | clean_resource_name}}",
"ami_description": "An EKS-optimized AMI for worker nodes.",
"instance_type": "t3.micro",
"region": "us-east-1",
"type": "amazon-ebs",
"source_ami": "ami-abcd1234",
"ssh_username": "ec2-user",
}],
"provisioners": [{
"type": "shell",
"inline": [
"gruntwork-install --module-name cloudwatch-log-aggregation-scripts --repo https://github.com/gruntwork-io/module-aws-monitoring --tag v0.22.1",
"gruntwork-install --module-name ssh-grunt --repo https://github.com/gruntwork-io/module-security --tag v0.34.5"
]
}]
}

Next, you write Terraform code to deploy the Kubernetes control plane using the eks-cluster-control-plane module and deploy Kubernetes worker nodes using the asg-rolling-deploy module, passing in the ID of the AMI you built with the Packer template via the ami input variable:

module "control_plane" {
source = "git::git@github.com:gruntwork-io/terraform-aws-eks.git//modules/eks-cluster-control-plane?ref=v0.22.0"
cluster_name               = "example-cluster"
kubernetes_version         = "1.17"
enabled_cluster_log_types  = ["api", "audit", "authenticator"]
vpc_id                     = var.vpc_id
vpc_master_subnet_ids      = var.control_plane_subnet_ids
# ... (other params omitted) ...
}
module "worker_nodes" {
source = "git::git@github.com:gruntwork-io/module-asg.git//modules/asg-rolling-deploy?ref=v0.10.0"

ami             = ""
vpc_subnet_ids  = var.worker_node_subnet_ids
min_size        = 2
max_size        = 10
# ... (other params omitted) ...
}

Run terraform apply and voila, you now have a fully working Kubernetes cluster and can start deploying Docker containers to it using kubectl apply or helm install!

The Gruntwork Service Catalog

As you just saw with the EKS cluster example, once you have modules, the next step is to combine multiple modules together to create services, which are designed to be deployed directly to production. A service includes everything you need in production, including the Terraform code for provisioning, Packer templates or Dockerfiles for configuration management, built-in monitoring (metrics, logging, alerting), built-in security features (SSH access, secrets management, server hardening, encryption in transit and at rest), and so on. You can think of services as the rooms in a house, which combine various sets of furniture (various sets of modules) for specific use cases:

undefined

Today, I’m excited to announce the Gruntwork Service Catalog, a major new addition to the IaC Library. It contains dozens of highly configurable, battle-tested, production-grade services that you can deploy off-the-shelf without writing any code. For example, instead of writing your own EKS cluster code as in the previous section, you can use the off-the-shelf eks-cluster service from the Gruntwork Service Catalog, which includes:

Here’s an example of how you can use this eks-cluster service to deploy a production-grade EKS cluster with a single Terragrunt configuration file:

# Use off-the-shelf code from the Gruntwork Service Catalog
terraform {
source = "git::git@github.com:gruntwork-io/aws-service-catalog.git//modules/services/eks-cluster?ref=v0.0.5"
}
# Pull in outputs from your VPC
dependency "vpc" {
config_path = "../../networking/vpc"
}
# Configuration for the eks-cluster service
inputs = {
cluster_name          = "example-eks-cluster"
cluster_instance_type = "m4.large"
cluster_instance_ami  = ""

# Deploy a a separate ASG per subnet / availability zone.
autoscaling_group_configurations = {
for subnet_id in dependency.vpc.outputs.private_app_subnet_ids :
subnet_id => {
min_size   = 3
max_size   = 5
subnet_ids = [subnet_id]
tags       = []
}
}
# ... (other params omitted) ...
}

Plug the code above into a file called terragrunt.hcl, run terragrunt apply, and you have a production-grade EKS cluster in minutes! Note: Terragrunt is not required; all the code above works with pure Terraform and Terraform Cloud / Terraform Enterprise too!

The Gruntwork Service Catalog includes dozens of other off-the-shelf services you can deploy in minutes, such as services for deploying and configuring other orchestration tools (e.g., ECS, ASGs), networking (e.g., VPCs, VPNs), data storage (e.g., RDS, ElastiCache, Elasticsearch, S3), AWS account structure (Landing Zone), CI / CD pipelines (both for apps and infrastructure), and much more. We also have a Service Catalog where all the services are configured to be compliant with the CIS AWS Foundations Benchmarks out-of-the-box. In the future, we’ll be adding out-of-the-box support for other major standards, such as PCI, HIPAA, and NIST 800–53.

The Gruntwork Architecture Catalog

The final step is to combine multiple services together to create your production architecture. The architecture is your entire end-to-end tech stack, that includes everything you use in production, including networking, orchestration, data storage, secrets management, CI / CD, monitoring, and so on, all wired together as a single, working system. You can think of the architecture as an entire house, which combines various sets of rooms (various sets of services) for specific use cases:

undefined

Today, I’m also excited to announce the Gruntwork Architecture Catalog, another major new addition to the IaC Library. For the last few years, we’ve offered the Gruntwork Reference Architecture as a product you can buy that made direct use of the Gruntwork Module Catalog under the hood. We are now turning the Reference Architecture into an Architecture Catalog*,* a collection of battle-tested, off-the-shelf architectures for AWS that make use of the Gruntwork Service Catalog under the hood.

For example, we have ECS-based, ASG-based, and EKS-based architectures in the Architecture Catalog. Here’s a glimpse of what the EKS-based architecture includes:

All of this is wired together for you, fully automated, and we can deploy it into your AWS accounts in about 1 day, giving you all the code to manage it.

Putting it all together

As a Gruntwork customer, you get access to all our Module Catalog and our Service Catalog. You could, for example, use the hundreds of modules from the Module Catalog directly to assemble your own services:

undefined

Or you could use the dozens of services from the Service Catalog and start deploying infrastructure right away:

undefined

Or you can have us deploy an architecture for you from our internal Architecture Catalog (in the future, we’ll grant customers direct access to this, too) and start building on top of a battle-tested, end-to-end tech stack, in about 1 day:

undefined

Whichever choice you pick, at the end of the day, you’ll be able to focus far more on your app, and get to production in days, rather than months:

undefined

And since it’s all commercially supported and maintained, you’ll be able to keep your infrastructure working smoothly and securely over the long-term.

Here’s how you get started:

Text Link

Platform

IaC Library
Account Factory
Pipelines
Patcher

Services

Accelerator
OpenTofu Services
Terragrunt Services
Pricing
Support

Open Source

Terragrunt
Terratest
Boilerplate
Cloudnuke
Git Xargs

Resources

Docs
Blog
Books
Community

Company

About
Careers
Merch
Legal
Security