Gruntwork Marketing Site

Open source, production-ready modules for Couchbase

Couchbase is an open source, distributed, NoSQL database that supports a huge range of features: key-value storage, JSON document storage, secondary indices, full-text search, live-cluster reconfiguration and rebalancing, replication (including cross data center replication), horizontal scaling, automatic failover, and mobile synchronization. And now, we can add one more item to this list: easy deployment in AWS!

We’ve worked with the Couchbase team to create a set of reusable infrastructure modules that make it easy to deploy and manage Couchbase in AWS. These production-ready modules are built on top of Terraform, Packer, and Bash, include thorough documentation, lots of example code, and automated tests. Today, we’re happy to announce that we’ve open sourced terraform-aws-couchbase under the Apache 2.0 license, and it’s available on GitHub and in the Terraform Registry!

A first example: spin up a Couchbase cluster in 5 minutes

Let’s take the modules for a spin. At the root of the terraform-aws-couchbaserepo is a simple example that makes it easy to try this code out. Create a file called main.tf with the following contents:

provider "aws" {
region = "us-east-1"
}

module "couchbase" {
source  = "gruntwork-io/couchbase/aws"
version = "0.1.0"
}

output "couchbase_web_console_url" {
value = "${module.couchbase.couchbase_web_console_url}"
}

output "sync_gateway_url" {
value = "${module.couchbase.sync_gateway_url}"
}

Next, run terraform init and terraform apply, and after a few minutes, you’ll see output that looks like this:

Apply complete! Resources: 42 added, 0 changed, 0 destroyed.

Outputs:

couchbase_web_console_url = couchbase-server-743533085.eu-west-1.elb.amazonaws.com:8091

sync_gateway_url = couchbase-server-743533085.eu-west-1.elb.amazonaws.com:4984

Open up couchbase_web_console_url in your web browser, login using the default credentials (admin/password), and you’ll see the Couchbase web console:

Congrats, you now have a Couchbase cluster running, including data nodes, index/query/search nodes, and Sync Gateway!

A second example: multi-dimensional scaling

The first example you just tried is a great way to experiment with these modules and learn how to use Couchbase, but it’s not necessarily the way you’d deploy Couchbase in production. For one thing, that root example runs everything on a single Auto Scaling Group:

In a production environment, you may want to run data nodes, index/query/search nodes, and Sync Gateway each in their own Auto Scaling Group so you can scale each one separately (this is known as* multi-dimensional scaling* or MDS):

To try out MDS, run terraform destroy to undeploy the first example, and then create a new main.tf with the following contents:

provider "aws" {
region = "us-east-1"
}

module "couchbase" {
source  =
"gruntwork-io/couchbase/aws//examples/couchbase-cluster-mds"
version = "0.1.0"
}

output "couchbase_data_nodes_web_console_url" {
value = "${module.couchbase.couchbase_data_nodes_web_console_url}"
}

output "couchbase_index_query_search_nodes_web_console_url" {
value = "${module.couchbase.couchbase_index_query_search_nodes_web_console_url}"
}

output "sync_gateway_url" {
value = "${module.couchbase.sync_gateway_url}"
}

Run terraform init and terraform apply once more, wait a few minutes, and you’ll get the following output:

Apply complete! Resources: 119 added, 0 changed, 0 destroyed.

Outputs:

couchbase_data_nodes_web_console_url = couchbase-server-data-430240563.eu-west-1.elb.amazonaws.com:8091

couchbase_index_query_search_nodes_web_console_url = couchbase-server-data-430240563.eu-west-1.elb.amazonaws.com:9091

sync_gateway_url = couchbase-server-data-430240563.eu-west-1.elb.amazonaws.com:4984

If you open couchbase_data_nodes_web_console_url in your browser, you’ll get the Web Console for the data nodes, which are in one Auto Scaling Group, and if you open couchbase_index_query_search_nodes_web_console_url, you’ll get the Web Console for the index, query, and search nodes, which are in their own Auto Scaling Group.

A third example: customizing the code for production

The two examples you’ve tried out are a great start, but you’ll likely want to customize the code before going to production. At a minimum, you’ll want to tweak the following:

To make testing/learning easier, the examples all deploy into the Default VPC. For production usage, you’ll most likely want to deploy into a custom VPC instead.
The examples all use a default username/password. You’ll definitely want to change this and use a secrets management tool to store it securely!

How do you make these changes? Well, under the hood, terraform-aws-couchbase consists of a number of standalone, orthogonal sub-modules, each of which is highly configurable and can be combined and composed any way you want. The examples you’ve been trying out above show a couple different ways of using those modules, but you’re free to come up with your own.

Here’s one way to do it. First, git clone the terraform-aws-couchbase repo to your computer:

git clone https://github.com/gruntwork-io/terraform-aws-couchbase.git

Next, go to the examples/couchbase-cluster-mds folder. This is the MDS example you deployed in the previous step! Open up main.tf and you’ll see all the code this example uses. At the bottom, of main.tf, you’ll see:

data "aws_vpc" "default" {
default = true
}

data "aws_subnet_ids" "default" {
vpc_id = "${data.aws_vpc.default.id}"
}

This is how that example finds your Default VPC. Feel free to tweak the settings to use a custom VPC instead.

In the user-data folder, you’ll find the scripts each of the Couchbase nodes runs during boot. Within those scripts, you’ll see some code that looks like this:

# To keep this example simple, we are hard-coding all credentials
# in this file in plain text. You should NOT do this in production
# usage!!! Instead, you should use tools such as Vault, Keywhiz,
# or KMS to fetch the credentials at runtime and only ever have
# the plaintext version in memory.

local readonly cluster_username="admin"
local readonly cluster_password="password"

As the comment indicates, you’ll want to change the username and password to your own values, and store/retrieve them using a secrets manager.

Once you’re done making customizations, run terraform init and terraform apply as before, and you’ll have a production-ready Couchbase cluster running in AWS!

Next steps

Check out the terraform-aws-couchbase repo for documentation and more examples, including how to do cross data center replication and how to build a custom Couchbase AMI. You should also browse through the full list of available submodules to get a sense of all the ways you can use this repo. Take the code for a spin and let us know how it works for you!

Your entire infrastructure. Defined as code. In about a day. Gruntwork.io.