Kubernetes as a Service (KUB)

Kubernetes is an open-source container management platform delivered by Google. In a containerized infrastructure, applications are deployed as isolated, independent entities, the so-called containers. Containerization has multiple benefits over the traditional deployment - faster and more efficient installation, environment consistency, portability, security.

EPAM Cloud provides installation of the latest Kubernetes version supported by the community.

In EPAM Cloud, by default, Kubernetes cluster includes two virtual machines function as master nodes, and one worker node.

Master node manages the workload and provides communication within the cluster as well as contains information about state of the cluster. Availability of two master nodes and more enables high service performance and ensures faultless operation. Worker node subordinates to the master node and serves as runner.

Application containers can be run on both master and worker nodes. You can make any changes in the predefined cluster configuration by changing Ansible inventory file generated by or2kc Maestro CLI command.

Have a Question?

The current page gives the general information on the service and the main workflows. However, while working with the services, our users encounter new questions they need assistance with. The most frequently asked questions on EPAM Cloud Services are gathered on the Cloud Services FAQ page.
Visit the page to check whether we have a ready answer for your question.

Related CLI Commands

The table below provides the list of service-related commands and their descriptions:

Command Description
or2-manage-service -a -s k8s -k key Activates Kubernetes services
or2-kubernetes-client (or2kc) Generates Ansible inventory. Location of the inventory file $MAESTRO_CLI_HOME/out/$PROJECT/$REGION/k8s/inventory

Service Activation

Master nodes and Worker node have the same parameters:

  • - Shape: LARGE (2 CPU, 7.5 RAM)
  • - Image: CoreOS_64-bit

To activate Kubernetes cluster, make sure you have installed Maestro CLI.

The following instruction is relevant to the POSIX compatible workstations. If you use Windows OS on your workstation, please run any Linux based instance.

STEPS FOR KUBERNETES CLUSTER ACTIVATION:

1. Declare the variables.

To make service set up and further usability easier and more user-friendly declare variables relevant for your project.

export PROJECT="Your project name"
export REGION="Your region"
export KEY_NAME="Name for ssh key pair"
export MAESTRO_CLI_HOME="Path to maestro-cli folder"
export LOG_FILE="cluster_deploy_$(date +"%Y_%m_%d_%I_%M_%p").log"

2. Create or add keys.

To create SSH keys, use or2addkey Maestro CLI command and specify your project, region and key name.

or2addkey -p $PROJECT -r $REGION -k $KEY_NAME

To add your own keys that already exist use or2ikey Maestro CLI command and specify path to your key, project, region and key name.

or2ikey -f /path/to/your/key -p $PROJECT -d $REGION -k $KEY_NAME

Move private key to the default folder with the command given below:

cp $MAESTRO_CLI_HOME/out/$PROJECT/$REGION/$KEY_NAME.pem ~/.ssh/
chmod 400 ~/.ssh/$KEY_NAME.pem

3. Install packages.

The following example is relevant to Ubuntu based distributives. If you use MacOS or other Linux distributives, please execute similar distributive specific package management command to install Ansible and kubectl. Ansible version should be higher than 2.9 x.

Use the following commands to install tools on your Ubuntu workstation to configure, manage and get info about your cluster. You can find more details on how to install kubeadm in the official documentation.

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
sudo apt install kubelet kubeadm kubectl git python3-pip -y

4. Add auto completion.

Add auto completion with the commands given below.

echo "source <(kubectl completion bash)" >> ~/.bashrc
source ~/.bashrc

5. Get kuberspray repo.

To get the kuberspray repo, use the link given below.

git clone https://github.com/kubernetes-sigs/kubespray.git

6. Install requirements.

To install requirements, use the command given below:

cd kubespray
mkdir inventory/mycluster
sudo pip3 install -r requirements.txt
cp -rfp inventory/sample inventory/mycluster

7. Run service for Kubernetes cluster.

To run service for the cluster, use or2ms Maestro CLI command and specify project, region, service name, -a flag and SSH key name. This command will run three Fedora CoreOS based instances of large shape (VM CPU 2; VM memory_MB 7680; SSD storage size_GB: 4)

or2ms -p $PROJECT -r $REGION -a -s k8s -k $KEY_NAME

8. Get stack status.

To get stack status, use or2dmstack Maestro CLI command and specify project, region, and service name.

or2dmstack -p $PROJECT -r $REGION -s $(or2dser -p $PROJECT -r $REGION -s k8s -P | awk '/stackId/ {print $3}')

9. Generate inventory file for Ansible.

To generate Ansible inventory, use or2kc Maestro CLI command and specify project and region.

or2kc -p $PROJECT -r $REGION

Please note that before the state of the stack goes to CREATE_COMPLETE, you will get error 32007 once you try to generate inventory. You just need to wait.

Here is an example of Ansible inventory file.

all:
 hosts:
  node1:
    ansible_host: 10.6.114.56
    ip: 10.6.114.56
    access_ip: 10.6.114.56
  node2:
    ansible_host: 10.6.114.65
    ip: 10.6.114.65
    access_ip: 10.6.114.65
  node3:
    ansible_host: 10.6.114.64
    ip: 10.6.114.64
    access_ip: 10.6.114.64
children:
 kube-master:
  hosts:
   node1:
   node2:
  kube-node:
  hosts:
   node1:
   node2:
   node3:
 etcd:
  hosts:
   node1:
   node2:
   node3:
  k8s-cluster:
  children:
   kube-master:
   kube-node:
  calico-rr:
   hosts: {}

10. Copy Ansible inventory file to the folder.

To copy inventory file to the folder, use the following command:

cp $MAESTRO_CLI_HOME/out/$PROJECT/$REGION/k8s/inventory inventory/mycluster/
cat << EOF >> inventory/mycluster/inventory
vars:
   kubectl_localhost: true
   kubeconfig_localhost: true
EOF

11. Run Ansible playbook.

To run Ansible playbook, invoke playbook cluster.yml

ansible-playbook -i ~/kubespray/inventory/mycluster/inventory --become --
become-user=root --user=core cluster.yml --key-
file "~/.ssh/$KEY_NAME.pem" | tee $LOG_FILE

Please note that you can face with error on the “Install required packages on Fedora CoreOS” step. This could happen due to the problem with package dependency. Now, we are looking for methods to eliminate this problem.

12. Create a symbolic link to default kubectl folder.

To create a symbolic link to default kubectl folder, use the command below:

amkdir -p ~/.kube
ln -s $(pwd)/kubespray/inventory/mycluster/artifacts/admin.conf
~/.kube/config

13. Create ServiceAccount (SA) for admin-user.

To create SA for admin-user, run the command given below:

kubectl create -n kube-system serviceaccount admin

14. Bind the role to admin SA.

cat > admin-role-binding.yml << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
   name: admin
roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: cluster-admin
subjects:
   - kind: ServiceAccount
   name: admin
   namespace: kube-system
EOF
kubectl apply -f admin-role-binding.yml

To get access to the Dashboard as "anonymous user", create the role and bind it:

cat > dashboard-anonymous-user.yml << EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
   name: kubernetes-dashboard-anonymous
rules:
- apiGroups: [""]
   resources: ["services/proxy"]
   resourceNames: ["https:kubernetes-dashboard:"]
   verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- nonResourceURLs: ["/ui", "/ui/*", "/api/v1/namespaces/kube-
system/services/https:kubernetes-dashboard:/proxy/*"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
   name: kubernetes-dashboard-anonymous
roleRef:
   apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
   name: kubernetes-dashboard-anonymous
subjects:
- kind: User
   name: system:anonymous
EOF
kubectl apply -f dashboard-anonymous-user.yml

15. Access to dashboard (optional)

To get access to the dashboard, on the first step check availability with the command given below:

kubectl cluster-info

The dashboard url is:

https://10.6.114.63:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

To see the information about nodes, workloads, etc. log in as admin. To get the token to log in as an admin, run the command given below:

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret
| grep admin | awk '{print $1}')

To get the token to log in as an anonymous, invoke the command given below:

ubectl -n kube-system describe secrets `kubectl -n kube-system get secrets
| awk '/clusterrole-aggregation-controller/ {print $1}'` | awk '/token:/
{print $2}'

To run a proxy to see the dashboard on the local host, use this command:

kubectl proxy &
curl http://127.0.0.1:8001/api/v1/namespaces/kubernetes-
dashboard/services/https:kubernetes-dashboard:/proxy/

Web UI

The Kubernetes service has a web UI which is automatically available as soon as the service is activated in the cluster. It is accessible via a URL over the https connection. The Web UI URL can be found from the Kubernetes node details returned by the kubectl cluster-info command.

The web UI has most of the functionality supported by the CLI in an intuitive format.

When the application is running, you can use the Kubernetes Dashboard to monitor its performance, debug errors and manage applications.

Pricing

The service usage price is defined by the price of the Kubernetes cluster. Kubernetes cluster contains 3 VMs of the following parameters:

  • - Shape: LARGE (2 CPU, 7.5 RAM)
  • - Image: CoreOS_64-bit

Therefore, the approximate monthly cost of a Kubernetes Server usage in case of 100% and 24/7 load is about $184 in EPAM-BY2 region (as to July 2020). Please pay attention, that if you run your Ubuntu workstation in EPAM Cloud, its cost will influence the total cost of the project based on the workstation capacity.

References

More information on the Kubernetes Service can be found in the EPAM Cloud Services Guide. For detailed description of the Maestro CLI commands used to manage the Kubernetes Service, refer to the Maestro CLI User Guide.