Deploying to AWS with Ansible and Terraform
1. Set up
a. Install python and verify whether python is installed or not
$sudo apt-get update
$sudo apt-get install python3.6
$python --version
b. Install Python pip
$apt install python-pip
$pip install --upgrade pip
c. Download Terraform
$curl -o https://releases.hashicorp.com/terraform/0.12.4/terraform_0.12.4_linux_amd64.zip
it will generate .zip file
d.create a folder(/bin/terraform) and unzip the terraform.zip in that folder
$mkdir /bin/terraform
$unzip terraform.zip -d /bin/terraform
set the terraform path
$export PATH= $PATH:/bin/terraform
e.verfify whether terraform is installed or not
$terrafoem --version
f.Install AWS CLI
$pip install aws cli --upgrade
$apt-get update
g. Install software properties
$apt-get install software-properties-common
h. Add repository which will contain ansible
$apt-add-repository ppa:ansible/ansible
i.update the packages
$apt-get update
j.Install ansible
$apt-get install ansible
h.verify whether ansible is installed or not
$ansible --version
i.we need to generate the key to access the server for that
$ssh-keygen
j.Save the key in /root/.ssh/filename
k.To access the ssh agent
$ssh-agent bash
$ssh-add ~/.ssh/filename
l.To make sure the key is present
$ssh-add -l
m.Modify ansible configuration file
$vim /etc/ansible/ansible.conf
disable host_key_checking i.e host_key_checking= False
n.Create a working directory
$mkdir terransible
2. IAM and DNS setup
a. Go to IAM console Select IAM, Select Users, Add user i.e terransible
b.Attach administer policy to the user
c.Create User
Make sure download the credentials
d. Configure Route 53
e. COnfigure aws
$aws configure-profile superhero
To get info from domain, we need to set route53 reusable delegation set.
Route53 reusable delegation set allows to set any no of domains that we wish but we wish but keep the same name servers that we always able to create host zone. we use this to want set public and private hosted zones.
$aws create-reusable-delegation-set-caller-reference 1224 --profile superhero
e. Register domains : Select domain, add or edit name servers
name servers should match registrar name servers.
The files and set up the varaibles
a. The files we create are 1. main.tf 2. variables.tf 3. terraform.tfvars 4. userdata 5. wordpress.yml 6. s3 update.yml 7. aws_hosts
Here main.tf pulls the variables from variables.tf which pulls the values from variables.tfvars so to make this reusable we create two scripts for varaibles. variables.tf has empty variables and terraform.tfvars wil populate the variables. Terraform is going to create environment based on main.tf.and terraform will create userdata file which AWS is accesed and AWS will populate aws_hosts. Ansible uses wordpress.yml and S3update.yml for dev instance.
a.go to terransible folder
$cd terransible
b. create files
$touch main.tf variables.tf terraform.tfvars
$touch userdata aws_hosts wordpress.yml s3update.yml
c. Create main.tf
$vim main.tf
and code syntax is
provider “aws” {
region = "${var.aws_region}"
profile = "${var.aws_profile}"
}
#data “aws_availability_zones” “available” {}
d.Create variables.tf, here empty variables are created
$vim variables.tf
and synatx is
variable “aws_region” {}
variable “aws_profile” {}
data “aws_availability_zones” “available” {}
3. Creating IAM user and give s3 access role to user.
The code written as
**#------------IAM---------------- **
#S3_access
resource “aws_iam_instance_profile” “s3_access_profile” {
** name = “s3_access”**
** role = “${aws_iam_role.s3_access_role.name}”**
}
resource “aws_iam_role_policy” “s3_access_policy” {
** name = “s3_access_policy”**
** role = “${aws_iam_role.s3_access_role.id}”**
** policy = <<EOF**
{
** “Version”: “2012-10-17”,**
** “Statement”: [**
** {**
** “Effect”: “Allow”,**
** “Action”: “s3:",**
** “Resource”: "”**
** }**
** ]**
}
EOF
}
resource “aws_iam_role” “s3_access_role” {
** name = “s3_access_role”**
** assume_role_policy = <<EOF**
{
** “Version”: “2012-10-17”,**
** “Statement”: [**
** {**
** “Action”: “sts:AssumeRole”,**
** “Principal”: {**
** “Service”: “ec2.amazonaws.com”**
** },**
** “Effect”: “Allow”,**
** “Sid”: “”**
** }**
** ]**
}
EOF
}
4.Create VPC
The code is
#-------------VPC-----------
resource “aws_vpc” “wp_vpc” {
** cidr_block = “${var.vpc_cidr}”**
** enable_dns_hostnames = true**
** enable_dns_support = true**
** tags {**
** Name = “wp_vpc”**
** }**
}
5.Create Internet Gateway
The code is
#internet gateway
resource “aws_internet_gateway” “wp_internet_gateway” {
** vpc_id = “${aws_vpc.wp_vpc.id}”**
** tags {**
** Name = “wp_igw”**
** }**
}
6. Create Route tables with subnets
The Code is
Route tables
resource “aws_route_table” “wp_public_rt” {
vpc_id = “${aws_vpc.wp_vpc.id}”
route {
cidr_block = “0.0.0.0/0”
gateway_id = “${aws_internet_gateway.wp_internet_gateway.id}”
}
tags {
Name = “wp_public”
}
}
resource “aws_default_route_table” “wp_private_rt” {
default_route_table_id = “${aws_vpc.wp_vpc.default_route_table_id}”
tags {
Name = “wp_private”
}
}
resource “aws_subnet” “wp_public1_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“public1”]}”
map_public_ip_on_launch = true
availability_zone = “${data.aws_availability_zones.available.names[0]}”
tags {
Name = “wp_public1”
}
}
resource “aws_subnet” “wp_public2_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“public2”]}”
map_public_ip_on_launch = true
availability_zone = “${data.aws_availability_zones.available.names[1]}”
tags {
Name = “wp_public2”
}
}
resource “aws_subnet” “wp_private1_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“private1”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[0]}”
tags {
Name = “wp_private1”
}
}
resource “aws_subnet” “wp_private2_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“private2”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[1]}”
tags {
Name = “wp_private2”
}
}
7. Creating S3 endpoints
The code is
#create S3 VPC endpoint
resource “aws_vpc_endpoint” “wp_private-s3_endpoint” {
vpc_id = “${aws_vpc.wp_vpc.id}”
service_name = “com.amazonaws.${var.aws_region}.s3”
route_table_ids = ["${aws_vpc.wp_vpc.main_route_table_id}",
“${aws_route_table.wp_public_rt.id}”,
]
policy = <<POLICY
{
“Statement”: [
{
“Action”: “",
“Effect”: “Allow”,
“Resource”: "”,
“Principal”: “*”
}
]
}
POLICY
}
resource “aws_subnet” “wp_rds1_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“rds1”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[0]}”
tags {
Name = “wp_rds1”
}
}
resource “aws_subnet” “wp_rds2_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“rds2”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[1]}”
tags {
Name = “wp_rds2”
}
}
resource “aws_subnet” “wp_rds3_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“rds3”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[2]}”
tags {
Name = “wp_rds3”
}
}
8. Subnet Associations
The code is
Subnet Associations
resource “aws_route_table_association” “wp_public_assoc” {
subnet_id = “${aws_subnet.wp_public1_subnet.id}”
route_table_id = “${aws_route_table.wp_public_rt.id}”
}
resource “aws_route_table_association” “wp_public2_assoc” {
subnet_id = “${aws_subnet.wp_public2_subnet.id}”
route_table_id = “${aws_route_table.wp_public_rt.id}”
}
resource “aws_route_table_association” “wp_private1_assoc” {
subnet_id = “${aws_subnet.wp_private1_subnet.id}”
route_table_id = “${aws_default_route_table.wp_private_rt.id}”
}
resource “aws_route_table_association” “wp_private2_assoc” {
subnet_id = “${aws_subnet.wp_private2_subnet.id}”
route_table_id = “${aws_default_route_table.wp_private_rt.id}”
}
resource “aws_db_subnet_group” “wp_rds_subnetgroup” {
name = “wp_rds_subnetgroup”
subnet_ids = ["${aws_subnet.wp_rds1_subnet.id}",
“${aws_subnet.wp_rds2_subnet.id}”,
“${aws_subnet.wp_rds3_subnet.id}”,
]
tags {
Name = “wp_rds_sng”
}
}
9.Create Security Groups
The code is
**#Security groups
resource “aws_security_group” “wp_dev_sg” {
name = “wp_dev_sg”
description = “Used for access to the dev instance”
vpc_id = “${aws_vpc.wp_vpc.id}”
#SSH
ingress {
from_port = 22
to_port = 22
protocol = “tcp”
cidr_blocks = ["${var.localip}"]
}
#HTTP
ingress {
from_port = 80
to_port = 80
protocol = “tcp”
cidr_blocks = ["${var.localip}"]
}
egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}
#Public Security group
resource “aws_security_group” “wp_public_sg” {
name = “wp_public_sg”
description = “Used for public and private instances for load balancer access”
vpc_id = “${aws_vpc.wp_vpc.id}”
#HTTP
ingress {
from_port = 80
to_port = 80
protocol = “tcp”
cidr_blocks = [“0.0.0.0/0”]
}
#Outbound internet access
egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}
#Private Security Group
resource “aws_security_group” “wp_private_sg” {
name = “wp_private_sg”
description = “Used for private instances”
vpc_id = “${aws_vpc.wp_vpc.id}”
Access from other security groups
ingress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = ["${var.vpc_cidr}"]
}
egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}
#RDS Security Group
resource “aws_security_group” “wp_rds_sg” {
name = “wp_rds_sg”
description = “Used for DB instances”
vpc_id = “${aws_vpc.wp_vpc.id}”
SQL access from public/private security group
ingress {
from_port = 3306
to_port = 3306
protocol = “tcp”
security_groups = ["${aws_security_group.wp_dev_sg.id}",
"${aws_security_group.wp_public_sg.id}",
"${aws_security_group.wp_private_sg.id}",
]
}
}
10. Creating S3 code bucket
The code is
#S3 code bucket
resource “random_id” “wp_code_bucket” {
byte_length = 2
}
resource “aws_s3_bucket” “code” {
bucket = “${var.domain_name}-${random_id.wp_code_bucket.dec}”
acl = “private”
force_destroy = true
tags {
Name = “code bucket”
}
}
11.Create Database instance and dev instance
The code is
**#---------compute-----------
resource “aws_db_instance” “wp_db” {
allocated_storage = 10
engine = “mysql”
engine_version = “5.6.27”
instance_class = “${var.db_instance_class}”
name = “${var.dbname}”
username = “${var.dbuser}”
password = “${var.dbpassword}”
db_subnet_group_name = “${aws_db_subnet_group.wp_rds_subnetgroup.name}”
vpc_security_group_ids = ["${aws_security_group.wp_rds_sg.id}"]
skip_final_snapshot = true
}
#key pair
resource “aws_key_pair” “wp_auth” {
key_name = “${var.key_name}”
public_key = “${file(var.public_key_path)}”
}
#dev server
resource “aws_instance” “wp_dev” {
instance_type = “${var.dev_instance_type}”
ami = “${var.dev_ami}”
tags {
Name = “wp_dev”
}
key_name = “${aws_key_pair.wp_auth.id}”
vpc_security_group_ids = ["${aws_security_group.wp_dev_sg.id}"]
iam_instance_profile = “${aws_iam_instance_profile.s3_access_profile.id}”
subnet_id = “${aws_subnet.wp_public1_subnet.id}”
provisioner “local-exec” {
command = <<EOD
cat < aws_hosts
[dev]
${aws_instance.wp_dev.public_ip}
[dev:vars]
s3code=${aws_s3_bucket.code.bucket}
domain=${var.domain_name}
EOF
EOD
}
provisioner “local-exec” {
command = “aws ec2 wait instance-status-ok --instance-ids ${aws_instance.wp_dev.id} --profile superhero && ansible-playbook -i aws_hosts wordpress.yml”
}
}
12. Creating Load Balancers
The code is
**#load balancer
resource “aws_elb” “wp_elb” {
name = “${var.domain_name}-elb”
subnets = ["${aws_subnet.wp_public1_subnet.id}",
“${aws_subnet.wp_public2_subnet.id}”,
]
security_groups = ["${aws_security_group.wp_public_sg.id}"]
listener {
instance_port = 80
instance_protocol = “http”
lb_port = 80
lb_protocol = “http”
}
health_check {
healthy_threshold = “${var.elb_healthy_threshold}”
unhealthy_threshold = “${var.elb_unhealthy_threshold}”
timeout = “${var.elb_timeout}”
target = “TCP:80”
interval = “${var.elb_interval}”
}
cross_zone_load_balancing = true
idle_timeout = 400
connection_draining = true
connection_draining_timeout = 400
tags {
Name = “wp_${var.domain_name}-elb”
}
}
13.Creating AMI
The code is
**resource “random_id” “golden_ami” {
byte_length = 8
}
resource “aws_ami_from_instance” “wp_golden” {
name = “wp_ami-${random_id.golden_ami.b64}”
source_instance_id = “${aws_instance.wp_dev.id}”
provisioner “local-exec” {
command = <<EOT
cat < userdata
#!/bin/bash
/usr/bin/aws s3 sync s3://${aws_s3_bucket.code.bucket} /var/www/html/
/bin/touch /var/spool/cron/root
sudo /bin/echo ‘*/5 * * * * aws s3 sync s3://${aws_s3_bucket.code.bucket} /var/www/html/’ >> /var/spool/cron/root
EOF
EOT
}
}**
14. Create AWS lauch Configuration
The code is
resource “aws_launch_configuration” “wp_lc” {
name_prefix = “wp_lc-”
image_id = “${aws_ami_from_instance.wp_golden.id}”
instance_type = “${var.lc_instance_type}”
security_groups = ["${aws_security_group.wp_private_sg.id}"]
iam_instance_profile = “${aws_iam_instance_profile.s3_access_profile.id}”
key_name = “${aws_key_pair.wp_auth.id}”
user_data = “${file(“userdata”)}”
lifecycle {
create_before_destroy = true
}
}
15. Create Auto Scaling Group
The Code is
#ASG
#resource “random_id” “rand_asg” {
byte_length = 8.
#}
resource “aws_autoscaling_group” “wp_asg” {
name = “asg-${aws_launch_configuration.wp_lc.id}”
max_size = “${var.asg_max}”
min_size = “${var.asg_min}”
health_check_grace_period = “${var.asg_grace}”
health_check_type = “${var.asg_hct}”
desired_capacity = “${var.asg_cap}”
force_delete = true
load_balancers = ["${aws_elb.wp_elb.id}"]
vpc_zone_identifier = ["${aws_subnet.wp_private1_subnet.id}",
“${aws_subnet.wp_private2_subnet.id}”,
]
launch_configuration = “${aws_launch_configuration.wp_lc.name}”
tag {
key = “Name”
value = “wp_asg-instance”
propagate_at_launch = true
}
lifecycle {
create_before_destroy = true
}
}
16.Create Route53
The code is
#---------Route53-------------
#primary zone
resource “aws_route53_zone” “primary” {
name = “${var.domain_name}.com”
delegation_set_id = “${var.delegation_set}”
}
#www
resource “aws_route53_record” “www” {
zone_id = “${aws_route53_zone.primary.zone_id}”
name = “www.${var.domain_name}.com”
type = “A”
alias {
name = “${aws_elb.wp_elb.dns_name}”
zone_id = “${aws_elb.wp_elb.zone_id}”
evaluate_target_health = false
}
}
#dev
resource “aws_route53_record” “dev” {
zone_id = “${aws_route53_zone.primary.zone_id}”
name = “dev.${var.domain_name}.com”
type = “A”
ttl = “300”
records = ["${aws_instance.wp_dev.public_ip}"]
}
#secondary zone
resource “aws_route53_zone” “secondary” {
name = “${var.domain_name}.com”
vpc_id = “${aws_vpc.wp_vpc.id}”
}
#db
resource “aws_route53_record” “db” {
zone_id = “${aws_route53_zone.secondary.zone_id}”
name = “db.${var.domain_name}.com”
type = “CNAME”
ttl = “300”
records = ["${aws_db_instance.wp_db.address}"]
}
These codes are all written in main.tf
3.We create empty variables in variables.tf
$ vim variables.tf
write empty variables as
variable “aws_region” {}
variable “aws_profile” {}
data “aws_availability_zones” “available” {}
variable “localip” {}
variable “vpc_cidr” {}
variable “cidrs” {
type = “map”
}
variable “db_instance_class” {}
variable “dbname” {}
variable “dbuser” {}
variable “dbpassword” {}
variable “key_name” {}
variable “public_key_path” {}
variable “domain_name” {}
variable “dev_instance_type” {}
variable “dev_ami” {}
variable “elb_healthy_threshold” {}
variable “elb_unhealthy_threshold” {}
variable “elb_timeout” {}
variable “elb_interval” {}
variable “asg_max” {}
variable “asg_min” {}
variable “asg_grace” {}
variable “asg_hct” {}
variable “asg_cap” {}
variable “lc_instance_type” {}
variable “delegation_set” {}
4. In terraform.tfvars we define variables
$vim terraform.tfvars
localip = “104.173.212.11/32”
aws_profile = “superhero”
aws_region = “us-east-1”
vpc_cidr = “10.0.0.0/16”
cidrs = {
public1 = “10.0.1.0/24”
public2 = “10.0.2.0/24”
private1 = “10.0.3.0/24”
private2 = “10.0.4.0/24”
rds1 = “10.0.5.0/24”
rds2 = “10.0.6.0/24”
rds3 = “10.0.7.0/24”
}
db_instance_class = “db.t2.micro”
dbname = “superherodb”
dbuser = “superhero”
dbpassword = “superheropass”
key_name = “kryptonite”
public_key_path = “/root/.ssh/kryptonite.pub”
domain_name = “bravethecloud”
dev_instance_type = “t2.micro”
dev_ami = “ami-b73b63a0”
elb_healthy_threshold = “2”
elb_unhealthy_threshold = “2”
elb_timeout = “3”
elb_interval = “30”
asg_max = “2”
asg_min = “1”
asg_grace = “300”
asg_hct = “EC2”
asg_cap = “2”
lc_instance_type = “t2.micro”
delegation_set = “N1HDAZB52OQ3IV”
test = {}
5. We write Ansible playbooks for downloading wordpress and creating buckets in s3update files
$vim wordpress.yml
- hosts: dev
become: yes
remote_user: ec2-user
tasks:- name: Install Apache.
yum: name={{ item }} state=present
with_items:- httpd
- php
- php-mysql
- name: Download WordPress
get_url: url=http://wordpress.org/wordpress-latest.tar.gz dest=/var/www/html/wordpress.tar.gz force=yes - name: Extract WordPress
command: “tar xzf /var/www/html/wordpress.tar.gz -C /var/www/html --strip-components 1” - name: Make my directory tree readable
file:
path: /var/www/html/
mode: u=rwX,g=rX,o=rX
recurse: yes
owner: apache
group: apache - name: Make sure Apache is started now and at boot.
service: name=httpd state=started enabled=yes
…
- name: Install Apache.
$vim s3update.yml
- hosts: dev
become: yes
remote_user: ec2-user
tasks:- name: Update S3 code bucket
command: aws s3 sync /var/www/html/ s3://{{ s3code }}/ --delete - shell: echo “define(‘WP_SITEURL’,'http://dev.”{{ domain }}".com’);" >> wp-config.php
args:
chdir: /var/www/html/ - shell: echo “define(‘WP_HOME’,'http://dev.”{{ domain }}".com’);" >> wp-config.php
args:
chdir: /var/www/html/
…
- name: Update S3 code bucket
6. $vim aws_hosts
[dev]
34.239.130.247
[dev:vars]
s3code=bravethecloud_104
domain=bravethecloud
7.$vim userdata
#!/bin/bash
/usr/bin/aws s3 sync s3://bravethecloud_104 /var/www/html/
/bin/touch /var/spool/cron/root
sudo /bin/echo ‘*/5 * * * * aws s3 sync s3://bravethecloud_104 /var/www/html/’ >> /var/spool/cron/root