CKAN on AWS

We have talked about CKAN and its power as an integrated software numerous times. In this blog we will explain the procedure for deploying CKAN on  the Kubernetes cluster on AWS EKS. But first, briefly about AWS. Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Now let’s move on further on how to deploy CKAN on Kubernetes Cluster on AWS. 

The easiest way to create and run an AWS Kubernetes cluster is with the eksctl CLI tool.

Before we start using eksctl, we need to make sure we have all the required tools for our work.

  • aws-cli
  • Kubectl

aws-cli

First of all, we need to have aws-cli installed on our machine.

We can check if we have aws-cli with this command: ​

$ aws --version
aws-cli/2.1.31 Python/3.8.8 Linux/5.10.23-1-MANJARO exe/x86_64.manjaro.21 prompt/off

​ After that we will need to configure aws-cli with our AWS credentials (Access key ID and Secret access key) ​

$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: yaml

​ Also we will need to ensure if we have right IAM policies on our IAM user for creating cluster with eksctl.

Note: remember to replace <account_id> with your own.

AmazonEC2FullAccess (AWS Managed Policy)

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": "ec2:*",
           "Effect": "Allow",
           "Resource": "*"
       },
       {
           "Effect": "Allow",
           "Action": "elasticloadbalancing:*",
           "Resource": "*"
       },
       {
           "Effect": "Allow",
           "Action": "cloudwatch:*",
           "Resource": "*"
       },
       {
           "Effect": "Allow",
           "Action": "autoscaling:*",
           "Resource": "*"
       },
       {
           "Effect": "Allow",
           "Action": "iam:CreateServiceLinkedRole",
           "Resource": "*",
           "Condition": {
               "StringEquals": {
                   "iam:AWSServiceName": [
                       "autoscaling.amazonaws.com",
                       "ec2scheduled.amazonaws.com",
                       "elasticloadbalancing.amazonaws.com",
                       "spot.amazonaws.com",
                       "spotfleet.amazonaws.com",
                       "transitgateway.amazonaws.com"
                   ]
               }
           }
       }
   ]
}

AWSCloudFormationFullAccess (AWS Managed Policy)

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": [
               "cloudformation:*"
           ],
           "Resource": "*"
       }
   ]
}

EksAllAccess

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": "eks:*",
           "Resource": "*"
       },
       {
           "Action": [
               "ssm:GetParameter",
               "ssm:GetParameters"
           ],
           "Resource": [
               "arn:aws:ssm:*:<account_id>:parameter/aws/*",
               "arn:aws:ssm:*::parameter/aws/*"
           ],
           "Effect": "Allow"
       },
       {
            "Action": [
              "kms:CreateGrant",
              "kms:DescribeKey"
            ],
            "Resource": "*",
            "Effect": "Allow"
       }
   ]
}

IamLimitedAccess

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": [
               "iam:CreateInstanceProfile",
               "iam:DeleteInstanceProfile",
               "iam:GetInstanceProfile",
               "iam:RemoveRoleFromInstanceProfile",
               "iam:GetRole",
               "iam:CreateRole",
               "iam:DeleteRole",
               "iam:AttachRolePolicy",
               "iam:PutRolePolicy",
               "iam:ListInstanceProfiles",
               "iam:AddRoleToInstanceProfile",
               "iam:ListInstanceProfilesForRole",
               "iam:PassRole",
               "iam:DetachRolePolicy",
               "iam:DeleteRolePolicy",
               "iam:GetRolePolicy",
               "iam:GetOpenIDConnectProvider",
               "iam:CreateOpenIDConnectProvider",
               "iam:DeleteOpenIDConnectProvider",
               "iam:ListAttachedRolePolicies",
               "iam:TagRole"
           ],
           "Resource": [
               "arn:aws:iam::<account_id>:instance-profile/eksctl-*",
               "arn:aws:iam::<account_id>:role/eksctl-*",
               "arn:aws:iam::<account_id>:oidc-provider/*",
               "arn:aws:iam::<account_id>:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup",
               "arn:aws:iam::<account_id>:role/eksctl-managed-*"
           ]
       },
       {
           "Effect": "Allow",
           "Action": [
               "iam:GetRole"
           ],
           "Resource": [
               "arn:aws:iam::<account_id>:role/*"
           ]
       },
       {
           "Effect": "Allow",
           "Action": [
               "iam:CreateServiceLinkedRole"
           ],
           "Resource": "*",
           "Condition": {
               "StringEquals": {
                   "iam:AWSServiceName": [
                       "eks.amazonaws.com",
                       "eks-nodegroup.amazonaws.com",
                       "eks-fargate.amazonaws.com"
                   ]
               }
           }
       }
   ]
}

kubectl

After installing and configuring aws-cli, we need to install kubectl on our computer.

Note: If you already have kubectl on your computer, skip this step.

There are many ways to install kubectl. One of them is through these commands: ​

$ curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"

$ chmod +x ./kubectl

$ sudo mv ./kubectl /usr/local/bin/kubectl

$ kubectl version --client

eksctl

eksctl is a simple CLI tool for creating clusters on EKS – Amazon’s new managed Kubernetes service for EC2. ​ We will create cluster by using a config file. Just run: ​

$ eksctl create cluster -f cluster.yaml

​ to apply a cluster.yaml file: ​

# A simple example of ClusterConfig object:
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
​
metadata:
 name: ckan-cluster
 region: us-east-1
​
nodeGroups:
 - name: ng-1
   instanceType: m5.large
   desiredCapacity: 3
​
availabilityZones: ["us-east-1a", "us-east-1b", "us-east-1c", "us-east-1d", "us-east-1f"]
​

Note: We will have to wait a few minutes for this operation.

When the Kubernetes cluster creation operation is complete, we continue deploying CKAN on our Kubernetes cluster. If you are interested in learning more about how to deploy CKAN on Kubernetes, check out our previous article on this topic. 

One Reply to “How to deploy CKAN on AWS”

Leave a Reply