Containerization - Let's Think Outside of the Box

By Ganna Shargorodska, Roman Rastiehaiev

Nowadays, software developers rarely start their projects from scratch - the project specifics usually suggest the tools and platforms which can be used to make the process more convenient and efficient. In the abundance of development facilities currently on the market, you are sure to find the one which suits your purposes perfectly, if you know where to look.

Naturally, EPAM Cloud always keeps an ear to the ground for the hottest trends in the development universe and checks whether a useful functionality can be made into a Cloud service for the benefit of the EPAM community.

One of the recent big projects in which EPAM Cloud enthusiastically engaged was containerization which is becoming a great hit among software developers. And rightly so, as its advantages are clear - faster and more efficient application installation, environment consistency, portability, security. Additionally, containerized deployment is more economic, as multiple containers share the resources of the same host OS. Certainly, EPAM Cloud could not stay out of it, and in 2014 Docker as a Service was rolled out, as the first shot at hosting a containerization platform in cloud.

In the two years which have passed since Docker was first implemented, it has evolved into a full-scale Cloud service for containerized infrastructure management with Docker Swarm as the clustering tool.

In March, 2016, we implemented a whole separate Cloud region to host containerization services, EPAM-DKR. This way, containerization services performance and reliability rose to a new level. The region is based on OpenStack and supports platform service usage on the CoreOS operating system.

However, containerization interests of EPAM professionals are not limited by Docker only. While Docker is used for actual container creation, their orchestration can be performed by different tools. The native Docker Swarm has an advantage of seamless compatibility, but, on the flip side, being a Docker product, it has the same limitations as Docker has. Using a third-party tool could help bypassing the native application limitations. Several container orchestrator tools were compared and evaluated (Kubernetes, Mesos and Marathon and Docker Swarm) and the choice was made in favor of Kubernetes as another container management platform to be deployed as a Cloud service.

Of course, this decision was not made without reason. We tapped heavily into the knowledge and experience of Kubernetes experts to weigh all pros and cons. It would be fair to say that the ideas, suggestions and propositions of the colleagues from the Kubernetes Group on Yammer played a significant part in the way Kubernetes as a Service turned out to be. We would like to specially thank Gyorgy Ruck for performing a solid comparative analysis of different containerization platforms and putting together recommendations on Kubernetes implementation over CoreOS. In May, 2016, Kubernetes became a rightful member of the family of EPAM Cloud Services.

Well, after two years we can boast of running a full-scale container service with a choice between two management platforms, all in a dedicated region. It looks like a reliable and comprehensive solution which may just need a tweak here and a patch there, but there is always somebody who wants to go further. With containerized infrastructure a big boom, everybody wants to run everything in containers. For their part, EPAM Cloud Team is closely watching the Kubernetes usage experience in various applications to be able to keep pace with the latest trends.

One of such trends is deployment of a data pipeline platform over Kubernetes. Gyorgy Ruck came across an implementation of Spring Cloud Data Flow, a native orchestration tool for data microservices, in a Kubernetes cluster and suggested that it had certain potential.

This became a "pet project" for Roman Rastiehaiev, a Software Engineer of EPAM Cloud who was determined on verifying whether such implementation was possible in Kubernetes environment offered by EPAM Cloud. It took him a week of research and experiment, sometimes after the office hours, to establish that, indeed, a Spring Cloud Data Flow server can be run on Kubernetes, however, with certain differences from the sequence suggested by the Spring Cloud Data Flow team. His recommendations on running this service can be found in the document attached to this page, and you are welcome to try them.

With containerization being the focus of developers' attention all around the globe, we are committed to continue our research in this area. And we would like to use this opportunity to thank you for valuable feedback and suggestions, as they can determine the direction in which our project will go. So, please try it, test it, experiment with it. And, of course, watch for our announcements, as you never know what we can come up with. Sometimes we don't, either.

Documentation

Below is a list of documents related to this section. You can find the full list of our documents in the Documentation Storage.

Please select a required document:

Spring Cloud Data Flow on Kubernetes

The document contains detailed instructions on deploying Spring Cloud Data Flow server in a Kubernetes cluster.