googlecloudplatform-test: DevOps automation with Consul and Google Compute Engine

Google Cloud Platform Blog

Product updates, customer stories, and tips and tricks on Google Cloud Platform

DevOps automation with Consul and Google Compute Engine

September 17, 2015

Modern organizations need speed and agility to adapt to today's fast pace of changing business priorities. Rapid application updates are only part of the challenge; successful development and operations teams should be able to quickly:

Provision new system architectures (e.g., add Cassandra or Redis)

Deploy to new localities (e.g., expand geographic business)

Ensure cost-effective deployments (e.g., scale up and down based on usage)

Offer resiliency and stability (e.g., implement automation to reduce the number of failure points)

Google Compute Engine gives a DevOps team the tools to meet many of these challenges by enabling rapid provisioning of additional compute power. Features such as Managed Instance Groups, Autoscaling, and worldwide coverage help teams solve these challenges, while keeping costs under control with per-minute billing and sustained-use discounts.

However, some of these challenges affect multiple aspects of an application’s infrastructure. Even simple changes may require service configuration updates (IP addresses, ports, passwords) and restarts across an environment, which is potentially very dangerous to do manually.

Service discovery is one technique for abstracting away complexity by removing hard-coded information from service configurations. The service discovery system is the source of truth for an architecture, from which all other services are able to reference for cross-system awareness. The end result is that services can move through environments without any manual intervention.

In an architecture that utilizes service discovery, new nodes can be brought up and immediately join a cluster, while old nodes can be destroyed and immediately removed from the cluster. It makes an architecture resilient, and gives operations teams the ability to deploy more frequently without worrying that a configuration will be out of date. Furthermore, it will often contain some form of health-checking so when systems are dynamically locating each other, they only reach healthy systems.

Consul is a service discovery tool from HashiCorp and it is conceptually similar to other service discovery tools such as etcd or ZooKeeper. Consul can be a great choice for designing a service-oriented architecture. It can utilize DNS as a means by which services can locate each other and it includes other features such as health checking and multi-datacenter aware deployments. To learn more check out HashiCorp’s educational series Road to Modern Ops and try the Consul tutorial.

HashiCorp is hosting their inaugural conference at the end of September where I’ll give a talk about devops workflow with Google Cloud Platform and Atlas. After the talk we'll be publishing a follow-up post describing all the ways you can use the HashiCorp suite of tools with Google Cloud Platform. Stay tuned!

- Posted by Julia Ferraioli, Developer Advocate, Google Cloud Platform