Auto Configuration Overview

Auto Configuration is an important part of Cloud service provisioning. Currently, EPAM Orchestrator supports convenient auto configuration scenarios for both Windows and Linux-based systems. Each user can select the one which is the most convenient for their purposes and expertise level.

The general scheme of available auto configuration options for Windows and Linux based VMs is given below:

There are three main auto configuration scenarios available for EPAM Cloud users: using Chef, using Ansible and SSH, and running user scripts. Each of them can be specifically convenient and effective, if used under proper conditions. All virtual machines run with scripts created by the EPAM Cloud team (Windows init and Linux init) which, if necessary, install the Chef Infra Client or run user scripts.

• Running a user script. This option is recommended if you need quick start and single interaction is enough. You can develop your own script, for either Linux or Windows VM, upload it to Orchestrator using or2uf Maestro CLI command, and then specify this script at instance run, so that it will be executed as soon as the instance is created.
• Using Ansible and SSH. Ansible can be used to configure VMs by push model from admin workstation. EPAM Orchestrator allows to set up Ansible environment and Dynamic Inventory using Maestro CLI tools. Please note that to use Ansible effectively, it is highly recommended to set up Multiple CLI workspaces. Such Platform Services provided by EPAM Cloud as OpenShift and Kubernetes are based on Ansible and SSH configuration model.
• Using Chef In case you plan to implement long-term auto configuration solutions, and need higher level of security, disaster-recovery, and availability, it is recommended to use Chef, which works by pull model. EPAM Orchestrator provides several pre-configured recipes that can be used to configure VMs with a single command call. In addition, you can develop your own recipes, depending on your project needs, and apply them when needed. You can also switch between different Chef servers by using the Chef service provided by Orchestrator. Such Platform Services provided by EPAM Cloud as Autoconfiguration, Cloud Monitoring, Log Aggregation, Load Balancer, Gerrit, Docker, Jenkins, Relational Database, Artifactory and Messenger are based on Chef configuration.

Related CLI Commands

The table below provides the list of the commands, related to Chef and Ansible manipulations, enabled in EPAM Cloud:

Engine	Command	Short Command	Description
Chef	or2-manage-service ... -s chef -a	or2ms ... -s chef -a	Starts the service in the specified project and region
	or2-validate-chef	or2vchef	Collects info on chef clients indicating any errors occured
	or2-chef-mode	or2cm	Sets one of the existing chef modes to the project
	or2-describe-chef	or2dchef	Describes the project Chef Server mode
	or2-set-instance-properties	or2setp	Used with -c/--chefattribute and -h/--chefrole flag to set the desired chef attribute to be used and the role to be set to the instance
Ansible	or2-ansible-init	or2ai	Initializes Ansible environment for the specified project and zone
	or2-ansible-hosts	or2ah	Is used to manage Ansible hosts
	or2-ansible-groups	or2ag	Is used to manage Ansible groups
	or2-ansible-group-properties	or2agp	Is used to manage Ansible group properties
	or2-ansible-dynamic-inventory	or2adi	Is used to retrieve Ansible dynamic inventory

Further on this page, you can find the examples of the commands usage for Auto Configuration Service manipulation.

Service Activation and Management

Chef activation and management

Chef-based Auto Configuration Service is available by default as soon as the project gets activated in Cloud. If you are using the default Chef server, no special setup is necessary, you can start using the Chef-related commands immediately.

If you need to disable auto configuration for VMs with specific OS types, rather than for the whole project, add the --customize parameter to the or2-manage-service (or2ms) command with the --activate flag:

or2ms -a -s chef -p project -r region --customize

To enable auto configuration back, run the same or2ms command with the --customize parameter, and select the NONE mode.

The new auto configuration disabling mode is applied only to the virtual instances that are launched after the mode is changed. The virtual instances created earlier, will stick to the mode in which they were created.

Ansible activation and management

To set up Ansible, run the or2ai command and specify your project and region:

or2ai -p project -r region

The command sets up all required configuration files in the current user directory. These are:

• ansible.cfg - Ansible configuration file
• default.properties - contains default values for the required CLI parameteres
• ansible_hosts.sh - the executable script for getting Ansible dynamic inventory

After Ansible environment is configured, you can proceed with Dynamic Inventory setup and further actions.

Manipulating Chef Server

There are three Chef auto configuration modes that determine which servers will be used by new Chef Infra clients:

• servers served by EPAM Cloud team,
• servers created specifically for the project by EPAM Cloud team,
• servers created specifically for the project by the project team itself.

Mode	Flag	Description
Default	-m default	The default mode for all projects in the EPAM Cloud. In this case, a common Chef server is used for all production environment machines.
EPC	-m epc	Use project-specific Chef server, created by EPAM Orchestrator for the specified project.
User	-m user	Use project-specific Chef server, created and properly configured by the user. When switching to this mode, the user should provide Chef server's instance ID (or instance IP) and manually upload validation.pem file to the Orchestrator's file storage. The user should also provide the path to validation.pem file during the command invocation.

Chef modes are managed with the or2-chef-mode (or2cm) command. To set up a non-default Chef server, you should start the Chef Service by running the or2cm command setting the Chef mode to EPC:

or2cm -p project -r region -m epc -y

This command starts a virtual machine to act as your project Chef server. All instances started under your project after the Chef mode switch, will be registered by the project server.

Setting the current Chef server to project will not apply any actions on the virtual machines running in this project. Re-configuration of the software on instances is not allowed.

To apply auto-configuration changes, you need to re-register existing VMs on the new Chef server.

You can share an existing EPC Chef server among several regions, projects and Cloud Platforms. To connect another region or project to an existing EPC Chef server, you will have to add its Service ID to the or2-manage-services command run for the project/region you want to add:

or2cm -p project -r region -m epc -S serviceid

In this way, you can share an existing Chef server between regions within one project or between different projectsthat are used for the same product or cutomer.

To establish a cross-project connection, you should have permissions to run the or2-change-mode (or2cm) command in the project which you will connect to the Chef server. In case you are not assigned to the project hosting the existing chef server, the Project Coordinator should provide you with the server service ID.

To disable auto-configuration for your project, use the or2-manage-service (or2ms) command with the --deactivate option:

or2ms -p project -r region -s chef --deactivate

Note that the command disabling auto-configuration will not terminate the project Chef server, if existing.

This command does not remove any resources created during the service performance.

To revert to the default Chef server, use the or2-chef-mode (or2cm) command to set the Chef mode to default. This command will automatically terminate the project Chef server.

It is impossible to start services in a project/region, if its Chef mode is set to USER.

You can use the or2-describe-chef (or2dchef) command to get the current Chef mode, Chef Server DNS and Chef Server state:

or2dchef -p project -r region

Reconfiguring Your VM with Chef Roles

In order to set a chef role or several roles for an instance, you can use the or2setp command with alias that allow to simplify the role assignment. The -c/--chefattribute parameter is used to specify the desired chef attribute to be used, and -h/--chefrole - the role to be set to the instance:

or2setp -i instance_id -h role1 -h role2 -c value1 -c "recipename1.attribute1=value2" -p project -r region

You can find the details about available auto configuration roles on our KB page.

Multiple CLI Workspaces

When working with Ansible auto configuration, it is important to have a set of pre-configured Maestro CLI parameters that would be automatically fetched and processed.

These parameters are set in %Maestro_CLI%lib/default.properties file according to property_name=property_value syntax. For example, default project and region specification would look like:

project=DEMOPRO region=EPAM-BY2

However, there is a typical need in using several environments in different locations and projects, or in having different sets of default parameters. This is possible by setting up different workspaces with different defaults collections.

To set up a multiple workspaces, perform the following steps:

• 1. Add the %Maestro_CLI%bin path to the PATH environment variable.
• 2. On your workstation, create a new folder for a new workspace. We would recommend to name it after project and region in which you are going to use this workspace.
• 3. Create a cli.properties file in the new folder and specify the properties for the new workspace in it. (if you initiate Ansible configuration, in this folder, this file will be automaticlly created).

After that, when working with Maestro CLI, just switch to the necessary workspace and run a command. The default properties will be automatically taken from the currently active folder. If the current folder includes no properties file, those specified in %Maestro_CLI%lib/cli.properties will be used.

Configuring with scripts

When running the VM, it is possible to specify one or several scripts which will be executed via cloud init after launching the VM, e.g. executing Bash for Linux, or a Batch/PowerShell file for Windows.

If a user wants to create a custom infrastructure for the project needs, and uses the same approach for reaching the desired state, the simplest way that still requires customization, is to execute scripts, while running a VM manually or using Maestro Stacks for running VMs, as well as launching stacks with pre-configured scripts.

Using scripts

In EPAM Cloud there is one single EPAM related script that performs all required functions and initiates downloading of custom scripts from EPAM Orchestrator for all cloud providers. This approach allowed to avoid all limitations and allows users to specify any number of supported scripts. If any of custom script has an error, it will not influence the whole VM configuration.

It is possible to specify custom parameterized scripts during virtual machine initialization using or2-run-instance(or2run) Maestro CLI command.

Specifics of the current implementation includes:

• for Linux-based systems – running Shell scripts will be executed at a Bash interpreter,
• for Windows-based systems – PowerShell scripts (with .ps1 extension) will be launched via PowerShell interpreter and Batch scripts (with .cmd extension) will be executed via cmd. Both of them will be launched with the highest privileges.

All scripts support passing the parameters, the values of which can be set during the instance start.

To run an instance with the custom script, invoke or2run (or2-run-instance) command and specify image, project, region, shape and script name.

or2run -i instance_id -p project -r region -s shape -t script_name

Response example:

References

More information on Chef Service can be found in the EPAM Cloud Services Guide. For detailed description of the Maestro CLI commands used to manage the Auto Configuration Service, refer to the Maestro CLI User Guide.
More information about Ansible usage can be found in the Ansible User Guide document, attached to this page.

Pricing

The service usage influences the project cost in case the Chef mode is switched to EPC. In this case, a special VM for project Chef Server is created. This VM has the following parameters:

• Shape: MEDIUM
• Image: Ubuntu18.04_64-bit

It is billed just as any other VM of such type. Therefore, the approximate monthly cost of such server in case of 100% and 24/7 load is about $30.64 in EPAM-BY2 region (as to July 2020). The price can vary depending on the region.
To get more detailed estimations, please, use our Cost Estimator tool.

When the Chef mode is switched from EPC to any other one, the VM launched for EPC mode is terminated.

Using Ansible does not apply additional charges on your project.

Documentation

Below is a list of documents related to this section. You can find the full list of our documents in the Documentation Storage.

Please select a required document: