We have talked about CKAN and its power as an integrated software numerous times. In this blog we will explain the procedure for deploying CKAN on the Kubernetes cluster on AWS EKS. But first, briefly about AWS. Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Now let’s move on further on how to deploy CKAN on Kubernetes Cluster on AWS.
The easiest way to create and run an AWS Kubernetes cluster is with the eksctl CLI tool.
Before we start using eksctl, we need to make sure we have all the required tools for our work.
- aws-cli
- Kubectl
aws-cli
First of all, we need to have aws-cli installed on our machine.
We can check if we have aws-cli with this command:
$ aws --version
aws-cli/2.1.31 Python/3.8.8 Linux/5.10.23-1-MANJARO exe/x86_64.manjaro.21 prompt/off
After that we will need to configure aws-cli with our AWS credentials (Access key ID and Secret access key)
$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: yaml
Also we will need to ensure if we have right IAM policies on our IAM user for creating cluster with eksctl.
Note: remember to replace <account_id> with your own.
AmazonEC2FullAccess (AWS Managed Policy)
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "ec2:*",
"Effect": "Allow",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "cloudwatch:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "autoscaling:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": [
"autoscaling.amazonaws.com",
"ec2scheduled.amazonaws.com",
"elasticloadbalancing.amazonaws.com",
"spot.amazonaws.com",
"spotfleet.amazonaws.com",
"transitgateway.amazonaws.com"
]
}
}
}
]
}
AWSCloudFormationFullAccess (AWS Managed Policy)
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"cloudformation:*"
],
"Resource": "*"
}
]
}
EksAllAccess
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "eks:*",
"Resource": "*"
},
{
"Action": [
"ssm:GetParameter",
"ssm:GetParameters"
],
"Resource": [
"arn:aws:ssm:*:<account_id>:parameter/aws/*",
"arn:aws:ssm:*::parameter/aws/*"
],
"Effect": "Allow"
},
{
"Action": [
"kms:CreateGrant",
"kms:DescribeKey"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
IamLimitedAccess
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:GetInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"iam:GetRole",
"iam:CreateRole",
"iam:DeleteRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:ListInstanceProfiles",
"iam:AddRoleToInstanceProfile",
"iam:ListInstanceProfilesForRole",
"iam:PassRole",
"iam:DetachRolePolicy",
"iam:DeleteRolePolicy",
"iam:GetRolePolicy",
"iam:GetOpenIDConnectProvider",
"iam:CreateOpenIDConnectProvider",
"iam:DeleteOpenIDConnectProvider",
"iam:ListAttachedRolePolicies",
"iam:TagRole"
],
"Resource": [
"arn:aws:iam::<account_id>:instance-profile/eksctl-*",
"arn:aws:iam::<account_id>:role/eksctl-*",
"arn:aws:iam::<account_id>:oidc-provider/*",
"arn:aws:iam::<account_id>:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup",
"arn:aws:iam::<account_id>:role/eksctl-managed-*"
]
},
{
"Effect": "Allow",
"Action": [
"iam:GetRole"
],
"Resource": [
"arn:aws:iam::<account_id>:role/*"
]
},
{
"Effect": "Allow",
"Action": [
"iam:CreateServiceLinkedRole"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": [
"eks.amazonaws.com",
"eks-nodegroup.amazonaws.com",
"eks-fargate.amazonaws.com"
]
}
}
}
]
}
kubectl
After installing and configuring aws-cli, we need to install kubectl on our computer.
Note: If you already have kubectl on your computer, skip this step.
There are many ways to install kubectl. One of them is through these commands:
$ curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
$ chmod +x ./kubectl
$ sudo mv ./kubectl /usr/local/bin/kubectl
$ kubectl version --client
eksctl
eksctl is a simple CLI tool for creating clusters on EKS – Amazon’s new managed Kubernetes service for EC2.
We will create cluster by using a config file. Just run:
$ eksctl create cluster -f cluster.yaml
to apply a cluster.yaml file:
# A simple example of ClusterConfig object:
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: ckan-cluster
region: us-east-1
nodeGroups:
- name: ng-1
instanceType: m5.large
desiredCapacity: 3
availabilityZones: ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1f"]
Note: We will have to wait a few minutes for this operation.
When the Kubernetes cluster creation operation is complete, we continue deploying CKAN on our Kubernetes cluster. If you are interested in learning more about how to deploy CKAN on Kubernetes, check out our previous article on this topic.
