Deploy Consul on Kubernetes

10min
|
Consul

Deployment

Consul is a service networking solution that enables teams to manage secure network connectivity between services and across on-prem and multi-cloud environments and runtimes. Consul offers service discovery, service mesh, traffic management, and automated updates to network infrastructure devices. Check out the What is Consul? page to learn more.

In this tutorial, you will deploy a Consul datacenter onto a Kubernetes cluster. After deploying Consul, you will interact with Consul using the UI and CLI.

In the following tutorials, you will deploy a demo application, integrate it with Consul service mesh, allow external traffic into the service mesh, and enhance observability into your service mesh.

In this tutorial, you will:

Deploy a HashiCorp Cloud Platform (HCP) Consul datacenter and an Elastic Kubernetes Service (EKS) cluster with Terraform
Configure your terminal to communicate with the Consul datacenter
View Consul services with the CLI, UI, and/or API

Prerequisites

Clone GitHub repository

Clone the GitHub repository containing the configuration files and resources.

$ git clone https://github.com/hashicorp-education/learn-consul-get-started-kubernetes.git

Change into the directory that contains the complete configuration files for this tutorial.

$ cd learn-consul-get-started-kubernetes/hcp-managed/eks

Clone the GitHub repository containing the configuration files and resources.

$ git clone https://github.com/hashicorp-education/learn-consul-get-started-kubernetes.git

Change into the directory that contains the complete configuration files for this tutorial.

$ cd learn-consul-get-started-kubernetes/hcp-managed/aks

Create infrastructure

With these Terraform configuration files, you are ready to deploy your infrastructure.

Issue the terraform init command from your working directory to download the necessary providers and initialize the backend.

$ terraform init

Initializing the backend...

Initializing provider plugins...
...

Terraform has been successfully initialized!
...

Then, deploy the resources. Confirm the run by entering yes.

$ terraform apply

## ...
Do you want to perform these actions?
 Terraform will perform the actions described above.
 Only 'yes' will be accepted to approve.

 Enter a value: yes

## ...

Apply complete! Resources: 69 added, 0 changed, 0 destroyed.

Note: The Terraform deployment could take up to 15 minutes to complete. Feel free to explore the next sections of this tutorial while waiting for the environment to complete initialization or learn more about the Raft protocol in a fun, interactive way.

With these Terraform configuration files, you are ready to deploy your infrastructure.

Issue the terraform init command from your working directory to download the necessary providers and initialize the backend.

$ terraform init

Initializing the backend...

Initializing provider plugins...
...

Terraform has been successfully initialized!
...

Then, deploy the resources. Confirm the run by entering yes.

$ terraform apply

## ...
Do you want to perform these actions?
 Terraform will perform the actions described above.
 Only 'yes' will be accepted to approve.

 Enter a value: yes

## ...

Apply complete! Resources: 24 added, 0 changed, 0 destroyed.

Connect to your infrastructure

Kubernetes stores cluster connection information in a file called kubeconfig. You can retrieve the Kubernetes configuration settings for your EKS cluster and merge them into your local kubeconfig file by issuing the following command:

$ aws eks --region $(terraform output -raw region) update-kubeconfig --name $(terraform output -raw kubernetes_cluster_id)

$ az aks get-credentials --resource-group $(terraform output -raw azure_rg_name) --name $(terraform output -raw aks_cluster_name)

Deploy Consul datacenter

HCP Consul is a secure, fully-managed solution for Consul and can be used immediately after the Terraform deployment completes, with no additional deployment/configuration steps. HCP Consul provides you with all Consul Enterprise features by default. Feel free to go directly to Configure your CLI to interact with Consul datacenter.

Configure your CLI to interact with Consul datacenter

In this section, you will set environment variables in your terminal so your Consul CLI can interact with your Consul datacenter. The Consul CLI reads these environment variables for behavior defaults and will reference these values when you run consul commands.

Tokens are artifacts in the ACL system used to authenticate users, services, and Consul agents. Since ACLs are enabled in this Consul datacenter, entities requesting access to a resource must include a token that is linked with a policy, service identity, or node identity that grants permission to the resource. The ACL system checks the token and grants or denies access to resources based on the associated permissions. A bootstrap token has unrestricted privileges to all resources and APIs.

Set the Consul ACL token as an environment variable.

$ export CONSUL_HTTP_TOKEN=$(terraform output -raw consul_token)

Set the Consul destination address. By default, Consul runs on port 8500 for http and 8501 for https.

$ export CONSUL_HTTP_ADDR=$(terraform output -raw consul_addr)

Set the Consul ACL token as an environment variable.

$ export CONSUL_HTTP_TOKEN=$(terraform output -raw consul_root_token)

Set the Consul destination address. By default, Consul runs on port 8500 for http and 8501 for https.

$ export CONSUL_HTTP_ADDR=$(terraform output -raw consul_url)

View Consul services

In this section, you will view your Consul services with the CLI, UI, and/or API to explore the details of your service mesh.

In your terminal, run the CLI command consul catalog services to return the list of services registered in Consul. Notice this returns only the consul service since it is the only running service in your Consul datacenter.

$ consul catalog services
consul

Agents run in either server or client mode. Server agents store all state information, including service and node IP addresses, health checks, and configuration. Client agents are lightweight processes that make up the majority of the datacenter. They report service health status to the server agents. Clients must run on every pod where services are running.

Run the CLI command consul members to return the list of Consul agents in your environment.

$ consul members
Node             Address            Status  Type    Build       Protocol  DC               Partition  Segment
ip-172-25-32-114 172.25.32.114:8301 alive   server  1.16.0+ent  2         learn-consul-gs  default    <all>

Output the HCP Consul URL value to your terminal and paste it in your browser.

$ echo $CONSUL_HTTP_ADDR
learn-hcp-eks-cluster.public.consul.00000000-0000-0000-0000-000000000000.aws.hashicorp.cloud

Output the token value to your terminal and copy the value to your clipboard. You will use this ACL token to authenticate in the Consul UI.

$ echo $CONSUL_HTTP_TOKEN
fe0dd5c3-f2e1-81e8-cde8-49d26cee5efc

On the left navigation pane, click Services to review your deployed services. At this time, you will only find the consul service.

Consul UI Services Page

By default, the anonymous ACL policy allows you to view the contents of Consul services, nodes, and intentions. To make changes and see more details within the Consul UI, click Log In in the top right and insert your ACL token.

Consul UI Login Page

After successfully authenticating with your ACL token, you are now able to view additional Consul components and make changes in the UI. Notice you can view and manage more options under the Access Controls section on the left navigation pane.

Consul UI Post Authentication

On the left navigation pane, click on Nodes.

Consul UI View Nodes

In your terminal, view the list of services registered in Consul.

$ curl -k \
    --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
    $CONSUL_HTTP_ADDR/v1/catalog/services\?pretty

Sample output:

{
    "consul": []
}

View the list of server and client Consul agents in your environment.

$ curl -k \
    --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
    $CONSUL_HTTP_ADDR/v1/agent/members\?pretty

Sample output:

[
    {
        "Name": "912fdb64-f013-535c-b564-c4351ecdb96a",
        "Addr": "172.25.16.4",
        "Port": 8301,
        "Tags": {
            "acls": "1",
            "bootstrap": "1",
            "build": "1.16.0+ent:e5a8325f",
            "dc": "learn-consul-gs",
            "ft_admpart": "1",
            "ft_fs": "1",
            "ft_ns": "1",
            "ft_si": "1",
            "grpc_tls_port": "8502",
            "id": "af11b884-3999-9214-7366-331bd1712fed",
            "port": "8300",
            "raft_vsn": "3",
            "role": "consul",
            "segment": "",
            "use_tls": "1",
            "vsn": "2",
            "vsn_max": "3",
            "vsn_min": "2",
            "wan_join_port": "8302"
        },
        "Status": 1,
        "ProtocolMin": 1,
        "ProtocolMax": 5,
        "ProtocolCur": 2,
        "DelegateMin": 2,
        "DelegateMax": 5,
        "DelegateCur": 4
    }
]

All services listed in your Consul catalog are empowered with Consul's service discovery capabilities that simplify scalability challenges and improve application resiliency. Review the Service Discovery overview page to learn more.

In this section, you will view your Consul services with the CLI, UI, and/or API to explore the details of your service mesh.

$ consul catalog services
consul

Run the CLI command consul members to return the list of Consul agents in your environment.

$ consul members
Node             Address           Status  Type    Build       Protocol  DC               Partition  Segment
912fdb64-f01...  172.25.16.4:8301  alive   server  1.16.0+ent  2         learn-consul-gs  default    <all>

Output the HCP Consul URL value to your terminal and paste it in your browser.

$ echo $CONSUL_HTTP_ADDR
https://servers-public-consul-8d4f721f.db0ea9b8.z1.hashicorp.cloud

Output the token value to your terminal and copy the value to your clipboard. You will use this ACL token to authenticate in the Consul UI.

$ echo $CONSUL_HTTP_TOKEN
fe0dd5c3-f2e1-81e8-cde8-49d26cee5efc

On the left navigation pane, click Services to review your deployed services. At this time, you will only find the consul service.

Consul UI Services Page

Consul UI Login Page

Consul UI Post Authentication

On the left navigation pane, click on Nodes.

Consul UI View Nodes

In your terminal, view the list of services registered in Consul.

$ curl -k \
    --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
    $CONSUL_HTTP_ADDR/v1/catalog/services\?pretty

Sample output:

{
    "consul": []
}

View the list of server and client Consul agents in your environment.

$ curl -k \
    --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
    $CONSUL_HTTP_ADDR/v1/agent/members\?pretty

Sample output:

[
    {
        "Name": "912fdb64-f013-535c-b564-c4351ecdb96a",
        "Addr": "172.25.16.4",
        "Port": 8301,
        "Tags": {
            "acls": "1",
            "bootstrap": "1",
            "build": "1.16.0+ent:e5a8325f",
            "dc": "learn-consul-gs",
            "ft_admpart": "1",
            "ft_fs": "1",
            "ft_ns": "1",
            "ft_si": "1",
            "grpc_tls_port": "8502",
            "id": "af11b884-3999-9214-7366-331bd1712fed",
            "port": "8300",
            "raft_vsn": "3",
            "role": "consul",
            "segment": "",
            "use_tls": "1",
            "vsn": "2",
            "vsn_max": "3",
            "vsn_min": "2",
            "wan_join_port": "8302"
        },
        "Status": 1,
        "ProtocolMin": 1,
        "ProtocolMax": 5,
        "ProtocolCur": 2,
        "DelegateMin": 2,
        "DelegateMax": 5,
        "DelegateCur": 4
    }
]

Next steps

In this tutorial, you integrated Consul into your Kubernetes environment. After deploying Consul, you interacted with Consul using the CLI, UI, and API.

In the next tutorial, you will deploy HashiCups, a demo application, onto your Kubernetes cluster to explore how to use Consul service mesh for service-to-service traffic management.

For more information about the topics covered in this tutorial, refer to the following resources:

Collection Overview

Consul on Kubernetes

Securely connect your services