Automating provisioning and configuration of Amazon ec2 instance Ansible.

In this article, you’ll come to know how we can automate entire setup of launching an ec2 instance to configuring it for webserver. And for this, we’ll be using one of the simplest, agentless, yet powerful automation tool called Ansible.

Ansible is an automation tool meant for configuration management. Although there are not so much features in Ansible for provisioning of OS, yet it can be used for provisioning. For this we’ll utilize a powerful feature of Ansible, dynamic inventory.

Now, question arises, how Ansible will interact with AWS?

A client can interact with AWS by three different ways: WebUI, AWS CLI and API (using SDKs). Ansible is integrated with API to make the configuration. As Ansible itself is based on Python, and uses Python libraries to do any sorts of configurations, so, here we’ll be using a Python library, boto3 to interact with AWS using Ansible.

Installing boto and boto3 library on controller node
#pip3 install boto boto3

To do any sort of configuration, Ansible controller node, need to login to managed node. In this case, ansible playbook performs configuration on same node and using http protocol, use API of AWS for provisioning on AWS. In APIs, we connect using http/s protocol using localhost IP of controller node, 127.0.0.1 i.e., playbook will run on the same node itself.

Ansible have a module named, ec2_instance, to create and manage AWS EC2 instances. We can see it using following command:

# ansible-doc ec2_instance

Before moving to provisioning part, you need to do login to AWS from Ansible. For AWS login you require two keys, aws_access_key and aws_secret_key and for this, you need an IAM user on AWS.

Note: The user should be IAM user with Programmatic access(SDK purpose).

Here, we’ll utilize the concept of Ansible vault to secure login credentials(access key and secret key). Ansible vault is a feature of ansible that allows to secure sensitive data such as security keys and password, by encrypting the file where it is stored.

To encrypt a file using Ansible vault, use command:
#ansible-vault encrypt <file_name>.yml
It will prompt for password which then further can be used to access the vault.

Now, you need to create an ansible playbook to launch an ec2 instance on AWS cloud. Attention!! As a pre-requisite here, you need a key pair and a security group pre-created on AWS. Below is the code of ansible playbook, for the same:

This playbook can be run using the following command. It will ask for vault password and will run the playbook for you.

ansible-playbook — ask-vault-pass <yml-file_name>

Ansible register used here, allows the user to get the output of the task and store it in a variables, here, ec2 , and can be used in different scenarios.

With the help of register variable, here we’re printing the public IP address of the instance from Ansible facts it gathers.

Now, we launched an ec2 instance and got its public IP address. Now what??Let’s move forward for configuring it as a webserver.

Here, ec2 instance is the managed node. Ansible framework allows administrators to run commands against defined hosts, Amazon EC2 instances, as soon as they are available, all over SSH. Then, in order to configure the instance, we need to do SSH login and for this we need login credentials dumped in inventory file. Let’s do it!

But not manually!!

We’ll use dynamic inventory to do so.

Ansible Dynamic Inventory

Dynamic Inventory is the inventory in which IP addresses are updated automatically/dynamically without human interruption. Here, IP address of target nodes are fetched dynamically from inventory/host file while running the playbooks.

For dynamic inventory, we use scripts that work as external APIs and pulls inventory information from dynamic sources, including cloud sources, using the supplied inventory plugins.

We can create own scripts or can use pre-created scripts by developers. Here, we’ll use some pre-created scripts shown below:

https://raw.githubusercontent.com/ansible/ansible/stable-1.9/plugins/inventory/ec2.py
https://raw.githubusercontent.com/ansible/ansible/stable-1.9/plugins/inventory/ec2.ini

You need to grab the ec2.py script and the ec2.ini config file. The ec2.py script is written using the Boto ec2 library and will query AWS for your running Amazon EC2 instances. The ec2.ini file is the config file for ec2.py, and can be used to limit the scope of Ansible’s reach. You can specify the regions, instance tags, or roles that the ec2.py script will find

Steps to achieve dynamic inventory for ec2 is given in below snippet:

STEP 1:
#wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.py
#wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.ini
STEP 2:
Make these files executable as,
#chmod +x ec2.py
#chmod +x ec2.ini
STEP 3:
Update shebang header in ec2.py file for Python3:
#!/usr/bin/python3
STEP 4:
Now update region and secret key or access key in ec2.ini file,
region= ap-south-1 #specify the regions that ec2.py script will find
aws_access_key= XXXX
aws_secret_key= XXXX
STEP 5:
After this, we need to export below commands in terminal:
export AWS_REGION='ap-south-1'
export AWS_ACCESS_KEY_ID=XXXX
export AWS_SECRET_ACCESS_KEY=XXXX

Now with the help of dynamic inventory, ansible will know how many ec2 instances are running on AWS and get their IPs. We can list the hosts IPs using below command:

# ansible  ec2 --list-hosts

Now, let’s quickly address SSH connectivity required from Ansible to ec2 instance. There are several options for how you’d like to handle authentication. Passwords, while not recommended, are allowed, and you can also use an SSH agent for credential forwarding.

Using an SSH agent is the best way to authenticate with your end nodes, as this alleviates the need to copy your .pem files around. To add an agent, do

# ssh-agent bash
#ssh-add ~/myAwsKey.pem #location of key file
We can check the ssh agent running using command:
#ssh-add -L

All set!! Now we’re ready to see Ansible shine. Let’s do the last step of configuring the instance for webserver. Below code will configure webserver in the managed node (Amazon ec2 instance here):

The playbook successfully run as we can see below:

As we know the public IP of the instance, we can see the webpage as:

Kudos!!!!!!

You did it!!

Let’s summarize what we did. Using Ansible, we reached to AWS and launched an Amazon ec2 instance, thereafter, it figured out their public IPs using dynamic inventory plugins, and re-reached to AWS connecting to the defined hosts for doing configuration part.

Here you can figure out, I did the entire setup with the help of automation tool. That’s the power of automation!!

You can get the codes, I’ve used above from below github link:

P-Bharti20/ansible_awsec2

Thank you for your patience! :)