Installing VMware Tanzu Kubernetes Grid aka TKG

This Post helps you to create Tanzu Kubernetes Grid Clusters running on either VMware Cloud on AWS and/or vSphere 6.7 Update 3 infrastructure.

Note – At the time of this writing, Tanzu Kubernetes Grid Plus is the only supported version of VMware Cloud on AWS. You can deploy Kubernetes clusters on your VMC clusters using Tanzu Kubernetes Grid Plus. Please refer to KB 78173 for detailed support matrix.

Prerequisite

On your vSphere/VMware Cloud on AWS instance ensure that you have the following objects in place:

• A resource pool in which to deploy the Tanzu Kubernetes Grid Instance (TKG)
• A VM folder in which to collect the Tanzu Kubernetes Grid VMs (TKG)
• Create a network segment with DHCP enabled
• Firewall rules on compute segment.
• This is not must but i prefer a Linux based Virtual Machine called "cli-vm" which we will be using as a "Bootstrap Environment" to install Tanzu Kubernetes Grid CLI binaries for Linux

INSTALLING KUBECTL

Kubectl is a command line tool for controlling Kubernetes clusters that we will use for managing/controlling k8 cluster deployed by TKG. To install latest version of Kubectl follow below steps on "cli-vm":

#curl -LO /bin/linux/amd64/kubectl">https://storage.googleapis.com/kubernetes-release/release/stable.txt/bin/linux/amd64/kubectl

#chmod +x kubectl

#mv kubectl /usr/local/bin

INSTALLING DOCKER

Docker is a daemon-based container engine which allows us to deploy applications inside containers, since my VM is CentOS, I started the docker service, to start docker service, follow below steps on cli-vm:

#systemctl start docker

#systemctl enable docker

To view the status of the daemon, run the following command:

#systemctl status docker

INSTALL TANZU KUBERNETES GRID CLI

To use Tanzu Kubernetes Grid, you download and run the Tanzu Kubernetes Grid CLI on local system, in our case we will install on our "cli-vm"

• Get the tkg cli binary from GA build page like:
  • For Linux platforms, download tkg-linux-amd64-v1.0.0_vmware.1.gz.
  • For Mac OS platforms, download tkg-darwin-amd64-v1.0.0_vmware.1.gz.
• Unzip the file:
  • gunzip ./tkg-linux-amd64-v1.0.0_vmware.1.gz
  • Copy the unzip binary to /usr/local/bin:
  • chmod +x . /tkg-linux-amd64-v1.0.0_vmware.1
  • mv ./tkg-linux-amd64-v1.0.0_vmware.1 /usr/local/bin/tkg
• Check tkg env is ready:
  • #tkg version
    Client:
    Version: v1.0.0
    Git commit: 60f6fd5f40101d6b78e95a33334498ecca86176e
• /root/.tkg folder will be auto created for tkg config file

Create an SSH Key Pair

In order for Tanzu Kubernetes Grid VMs to run tasks in vSphere, you must provide the public key part of an SSH key pair to Tanzu Kubernetes Grid when you deploy the management cluster. You can use a tool such as ssh-keygen to generate a key pair.

• On the machine on which you will run the Tanzu Kubernetes Grid CLI, run the following ssh-keygen command.
  • #ssh-keygen -t rsa -b 4096 -C "email@example.com"
• At the prompt
  • Enter file in which to save the key (/root/.ssh/id_rsa):
  • Press Enter to accept the default.
• Enter and repeat a password for the key pair.
• Open the file .ssh/id_rsa.pub in a text editor copy and paste it when you deploy the management cluster.

Import OVA & Create Template in VC

Before we can deploy a Tanzu Kubernetes Grid management cluster or Tanzu Kubernetes clusters to vSphere, we must provide a base OS image template to vSphere. Tanzu Kubernetes Grid creates the management cluster and Tanzu Kubernetes cluster node VMs from this template. Tanzu Kubernetes Grid provides a base OS image template in OVA format for you to import into vSphere. After importing the OVA, you must convert the VM into a vSphere VM template

• Export TKG needs 2 ova, which are photon-3-v1.17.3+vmware.2.ova and photon-3-capv-haproxy-v0.6.3+vmware.1.ova.
• Convert above both ova VMs to templates and put them to the VM folder.
• Here are high level steps:
  • In the vSphere Client, right-click on cluster & select Deploy OVF template.
  • Choose Local file, click the button to upload files, and navigate to the photon-3-v1.17.3_vmware.2.ova file on your local machine.
  • Follow the on screen instruction to deploy VM from the OVA template.
  • Choose the appliance name.
  • Choose the destination datacenter or folder.
  • Choose the destination host, cluster, or resource pool.
  • Accept EULA
  • Select the disk format and datastore.
  • Select the network for the VM to connect to
  • Click Finish to deploy the VM
  • Right-click on the VM, select Template and click on Convert to Template.
  • Follow the same step for proxy OVA – photon-3-capv-haproxy-v0.6.3+vmware.1.ova

Installing TKG Management Cluster

Once pre-work is done, follow below steps to create Tanzu Management cluster:

• On the machine on which you downloaded and installed the Tanzu Kubernetes Grid CLI, run the following command
  • #tkg init –ui
• If your CLI VM running X11 desktop then it will open browser with loopback IP, otherwise you can setup using putty on your windows desktop, like this:

• Once you have successfully opened the connection, open you web browser and navigate to http://127.0.0.1:8081 and you should be seeing below page

• Enter your IAAS provider details where TKG can create K8s cluster and in this case we need to enter VMC vCenter Server information and then click "Connect" button, you will get notification etc accept all and then "connect button" becomes "connected" from here you just need to select the details about where do you want to deploy TKG, here fill the Datacenter and SSH Key which we created in previous steps.

• Select Development or Production Flavour and specify an instance type, then give the K8s Management Cluster a name and select API server Load Balancer (specify the HA Proxy VM Template, which we have uploaded in previous step)

• Select Resource Pool(TKG), VM Folder(TKG) and Workload Datastore

• Select Network Name and leave the other as default

• Select the K8s PhotonOS Template, this is the VM template that we uploaded in previous steps.

• Review all settings to ensure they match as per your selection and then click on "Deploy Management Cluster", which will begin the deployment process.

• This takes around 5 to 6 minutes to complete the entire process of setting up the TKG management cluster and once the Management Cluster has been deployed, you can go close the web browser and go back to your SSH session and stop the TKG UI rendering.

• You can verify that all pods are up & running by running the command:
  • #kubectl get pods -A

Deploy Tanzu Kubernetes workload Cluster

So let's deploy Tanzu Kubernetes Cluster as we have completed Tanzu Kubernetes grid Management cluster, we can use the "TKG" CLI to deploy Tanzu Kubernetes clusters from your management cluster to vSphere/VMware Cloud on AWS. Run the following command to deploy TKG Cluster called "avnish" or any other name you wish to use.

• #tkg create cluster avnish –plan=dev

This command will deploy a Tanzu Kubernetes cluster with the minimum default configuration. I am also deploying another cluster by specifying a few more parameters.

Deploy Highly Available Kubernetes Cluster

This command will deploy a highly available kubernetes cluster.

• #tkg create cluster my-cluster –plan dev –controlplane-machine-count 3 –worker-machine-count 5

• Once the TKG Cluster is up and running, we can run the following commands to get cluster information and credentials:
  • #tkg get clusters
  • #tkg get credential < clustername >

• To get the context of TKG clusters, run normal Kubernetes commands:
  • #kubectl config get-contexts
  • #kubectl config use-context avnish-admin@avnish

• To get the kubernetes node details, run kubernetes commands ...

Scale TKG Clusters

After we create a Tanzu Kubernetes cluster, we can scale it up or down by increasing or reducing the number of node VMs that it contains.

• Scale up – Scale a cluster, use the
  • #tkg scale cluster
  • to change the number of control plane nodes–controlplane-machine-count
  • to change the number of worker nodes –worker-machine-count

Installing Octant on TKG Clusters

Octant is an open source developer-centric web interface for Kubernetes that lets you inspect a Kubernetes cluster and its applications, it is simple to install and easy to use.

• Here is more information on Installation and configuration
• you need to install octant on your CLI VM and then proxy it so that you can open its web interface.

This completes the installation and configuration of Tanzu Kubernetes Grid. Once you have a management cluster ready, go ahead and deploy containerised applications on these clusters. TKG gives lots of flexibility of deploying, scaling and managing multiple TKG workload clusters. Such clusters can be deployed and configured on the basis of projects or departments.