Using Packer in EPAM Cloud

In software development, saving time and effort is very often as critical as producing clean code and ensuring consistent performance. For that reason, tools and methods allowing to reduce time and man-hours required for any configuration or setup work are in the focus of close attention of many development companies. Automation has grown into a separate layer of technology with lots of various tools, products and methods helping you spend less time on environment configuration and allowing developers to get started immediately.

Indeed, with a smart set of properly selected automation tools, you will benefit double - on one hand, you will need minimal manual operations and, therefore, will not need to engage DevOps engineers for a long time. On the other hand, your environments will be ready much faster for the actual development to begin.

The software market is full of tools for automating various configuration processes. EPAM Cloud vastly uses automation in deploying platform services - the Cloud Formation functionality to start infrastructures and Chef to install and configure software. And recently, our colleagues from the EPM-SNQ project have found the way to use other automation tools to perform native Cloud operations. They started with Packer, a machine image creation tool, which is rather popular in the EPAM development community.

Why Packer? Packer has a number of features making it a reasonable choice as the automation tool for image creation:

  • - Packer is versatile. You can use it to create identical machine images for different platforms - Amazon Web Services, Microsoft Azure, OpenStack- or VMware-based environments
  • - Packer is easy. To use Packer, you do not need any special knowledge or experience
  • - Packer is open-source. No comment
  • - Packer is consistent. You can create images or other artifacts from a single source and be sure that they will perform identically on different platforms
  • - Packer is fast. Machine images can be created and launched quickly to verify their consistency

How does Packer create machine images? For each supported platform, Packer has a so-called builder, a configuration sequence designed to generate images for such platform. For example, for Amazon, it has as many as four builders to be selected depending on the process of machine image creation. Each builder is configured with the parameters and options specific for the platform.

Moreover, since we mentioned Amazon, they have found the way to use Packer for creating custom platforms - Packer generates an AMI specially configured with all software required for the application and then the AMI is used by Elastic Beanstalk to create a custom platform. This may be a great solution for cases when the necessary platform is not supported by Elastic Beanstalk by default. For example, Expedia has already appreciated the possibilities offered by custom platforms and has launched an own application deployment platform using AWS services. With as many as 4000 custom applications under the belt, they can definitely claim success.

All that makes Packer very attractive and promising for application in EPAM Cloud. The guys from EPM-SNQ thought that having a private cloud, EPAM Systems should definitely have a way to support Packer-generated images in it.

What they did was create a plugin working as a dedicated Packer builder for EPAM Cloud, so that custom images can be automatically generated in the format supported by EPAM Cloud. The plugin flow is as follows:

  • 1. Start an instance in EPAM Cloud from a custom image
  • 2. Install all required software, if necessary
  • 3. Stop the instance
  • 4. Create an image from the stopped instance and verify that the image has the "ready" state
  • 5. Terminate the instance in Cloud

In the background, the plugin uses the API methods performing instance operations in EPAM Cloud as well as the Packer tools creating the machine image. This way, you are getting an automatically generated image without having to manually perform the instance manipulations required to create an image from an EPAM Cloud virtual machine.

In fact, this is another Packer "builder" specially designed for creating images for EPAM Cloud. The builder has the following structure:

We believe that EPM-SNQ did an amazing work finding alternative approaches to using native Cloud tools. On behalf of EPM-SNQ and Iurii Medvedev, Konstantin Valetov and Yury Rudenko who came up with the idea of adapting Packer to Cloud and then actually did it, we would like to share their know-how with the entire EPAM community for trying and testing. You can download the plugin from the EPAM repository to use for your project purposes. For more details on Packer support for EPAM Cloud see Iurii's presentation on IT Week 2017.

As the conclusion, we would like to say that this is not the only attempt of automating EPAM Cloud processes via external tools. Other projects also work in this area and are happy to share their technology. We will continue with more articles on this subject soon. Stay tuned!