Deploying to AWS with Ansible and Terraform (2)


#1

Deploying to AWS with Ansible and Terraform
1. Set up
a. Install python and verify whether python is installed or not
$sudo apt-get update
$sudo apt-get install python3.6
$python --version
b. Install Python pip
$apt install python-pip
$pip install --upgrade pip
c. Download Terraform
$curl -o https://releases.hashicorp.com/terraform/0.12.4/terraform_0.12.4_linux_amd64.zip
it will generate .zip file
d.create a folder(/bin/terraform) and unzip the terraform.zip in that folder
$mkdir /bin/terraform
$unzip terraform.zip -d /bin/terraform
set the terraform path
$export PATH= $PATH:/bin/terraform
e.verfify whether terraform is installed or not
$terrafoem --version
f.Install AWS CLI
$pip install aws cli --upgrade
$apt-get update
g. Install software properties
$apt-get install software-properties-common
h. Add repository which will contain ansible
$apt-add-repository ppa:ansible/ansible
i.update the packages
$apt-get update
j.Install ansible
$apt-get install ansible
h.verify whether ansible is installed or not
$ansible --version
i.we need to generate the key to access the server for that
$ssh-keygen
j.Save the key in /root/.ssh/filename
k.To access the ssh agent
$ssh-agent bash
$ssh-add ~/.ssh/filename
l.To make sure the key is present
$ssh-add -l
m.Modify ansible configuration file
$vim /etc/ansible/ansible.conf
disable host_key_checking i.e host_key_checking= False
n.Create a working directory
$mkdir terransible
2. IAM and DNS setup
a. Go to IAM console Select IAM, Select Users, Add user i.e terransible
b.Attach administer policy to the user
c.Create User
Make sure download the credentials
d. Configure Route 53
e. COnfigure aws
$aws configure-profile superhero
To get info from domain, we need to set route53 reusable delegation set.
Route53 reusable delegation set allows to set any no of domains that we wish but we wish but keep the same name servers that we always able to create host zone. we use this to want set public and private hosted zones.
$aws create-reusable-delegation-set-caller-reference 1224 --profile superhero
e. Register domains : Select domain, add or edit name servers
name servers should match registrar name servers.
The files and set up the varaibles
a. The files we create are 1. main.tf 2. variables.tf 3. terraform.tfvars 4. userdata 5. wordpress.yml 6. s3 update.yml 7. aws_hosts
Here main.tf pulls the variables from variables.tf which pulls the values from variables.tfvars so to make this reusable we create two scripts for varaibles. variables.tf has empty variables and terraform.tfvars wil populate the variables. Terraform is going to create environment based on main.tf.and terraform will create userdata file which AWS is accesed and AWS will populate aws_hosts. Ansible uses wordpress.yml and S3update.yml for dev instance.
a.go to terransible folder
$cd terransible
b. create files
$touch main.tf variables.tf terraform.tfvars
$touch userdata aws_hosts wordpress.yml s3update.yml
c. Create main.tf
$vim main.tf
and code syntax is
provider “aws” {
region = "${var.aws_region}"
profile = "${var.aws_profile}"
}

#data “aws_availability_zones” “available” {}

d.Create variables.tf, here empty variables are created
$vim variables.tf
and synatx is
variable “aws_region” {}
variable “aws_profile” {}
data “aws_availability_zones” “available” {}

3. Creating IAM user and give s3 access role to user.
The code written as

**#------------IAM---------------- **

#S3_access

resource “aws_iam_instance_profile” “s3_access_profile” {
** name = “s3_access”**
** role = “${aws_iam_role.s3_access_role.name}”**
}

resource “aws_iam_role_policy” “s3_access_policy” {
** name = “s3_access_policy”**
** role = “${aws_iam_role.s3_access_role.id}”**

** policy = <<EOF**
{
** “Version”: “2012-10-17”,**
** “Statement”: [**
** {**
** “Effect”: “Allow”,**
** “Action”: “s3:",**
** “Resource”: "
”**
** }**
** ]**
}
EOF
}

resource “aws_iam_role” “s3_access_role” {
** name = “s3_access_role”**

** assume_role_policy = <<EOF**
{
** “Version”: “2012-10-17”,**
** “Statement”: [**
** {**
** “Action”: “sts:AssumeRole”,**
** “Principal”: {**
** “Service”: “ec2.amazonaws.com”**
** },**
** “Effect”: “Allow”,**
** “Sid”: “”**
** }**
** ]**
}
EOF
}

4.Create VPC
The code is
#-------------VPC-----------

resource “aws_vpc” “wp_vpc” {
** cidr_block = “${var.vpc_cidr}”**
** enable_dns_hostnames = true**
** enable_dns_support = true**

** tags {**
** Name = “wp_vpc”**
** }**
}
5.Create Internet Gateway
The code is
#internet gateway

resource “aws_internet_gateway” “wp_internet_gateway” {
** vpc_id = “${aws_vpc.wp_vpc.id}”**

** tags {**
** Name = “wp_igw”**
** }**
}
6. Create Route tables with subnets
The Code is

Route tables

resource “aws_route_table” “wp_public_rt” {
vpc_id = “${aws_vpc.wp_vpc.id}”

route {
cidr_block = “0.0.0.0/0”
gateway_id = “${aws_internet_gateway.wp_internet_gateway.id}”
}

tags {
Name = “wp_public”
}
}

resource “aws_default_route_table” “wp_private_rt” {
default_route_table_id = “${aws_vpc.wp_vpc.default_route_table_id}”

tags {
Name = “wp_private”
}
}

resource “aws_subnet” “wp_public1_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“public1”]}”
map_public_ip_on_launch = true
availability_zone = “${data.aws_availability_zones.available.names[0]}”

tags {
Name = “wp_public1”
}
}

resource “aws_subnet” “wp_public2_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“public2”]}”
map_public_ip_on_launch = true
availability_zone = “${data.aws_availability_zones.available.names[1]}”

tags {
Name = “wp_public2”
}
}

resource “aws_subnet” “wp_private1_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“private1”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[0]}”

tags {
Name = “wp_private1”
}
}

resource “aws_subnet” “wp_private2_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“private2”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[1]}”

tags {
Name = “wp_private2”
}
}
7. Creating S3 endpoints
The code is
#create S3 VPC endpoint
resource “aws_vpc_endpoint” “wp_private-s3_endpoint” {
vpc_id = “${aws_vpc.wp_vpc.id}”
service_name = “com.amazonaws.${var.aws_region}.s3”

route_table_ids = ["${aws_vpc.wp_vpc.main_route_table_id}",
“${aws_route_table.wp_public_rt.id}”,
]

policy = <<POLICY
{
“Statement”: [
{
“Action”: “",
“Effect”: “Allow”,
“Resource”: "
”,
“Principal”: “*”
}
]
}
POLICY
}

resource “aws_subnet” “wp_rds1_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“rds1”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[0]}”

tags {
Name = “wp_rds1”
}
}

resource “aws_subnet” “wp_rds2_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“rds2”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[1]}”

tags {
Name = “wp_rds2”
}
}

resource “aws_subnet” “wp_rds3_subnet” {
vpc_id = “${aws_vpc.wp_vpc.id}”
cidr_block = “${var.cidrs[“rds3”]}”
map_public_ip_on_launch = false
availability_zone = “${data.aws_availability_zones.available.names[2]}”

tags {
Name = “wp_rds3”
}
}

8. Subnet Associations
The code is

Subnet Associations

resource “aws_route_table_association” “wp_public_assoc” {
subnet_id = “${aws_subnet.wp_public1_subnet.id}”
route_table_id = “${aws_route_table.wp_public_rt.id}”
}

resource “aws_route_table_association” “wp_public2_assoc” {
subnet_id = “${aws_subnet.wp_public2_subnet.id}”
route_table_id = “${aws_route_table.wp_public_rt.id}”
}

resource “aws_route_table_association” “wp_private1_assoc” {
subnet_id = “${aws_subnet.wp_private1_subnet.id}”
route_table_id = “${aws_default_route_table.wp_private_rt.id}”
}

resource “aws_route_table_association” “wp_private2_assoc” {
subnet_id = “${aws_subnet.wp_private2_subnet.id}”
route_table_id = “${aws_default_route_table.wp_private_rt.id}”
}

resource “aws_db_subnet_group” “wp_rds_subnetgroup” {
name = “wp_rds_subnetgroup”

subnet_ids = ["${aws_subnet.wp_rds1_subnet.id}",
“${aws_subnet.wp_rds2_subnet.id}”,
“${aws_subnet.wp_rds3_subnet.id}”,
]

tags {
Name = “wp_rds_sng”
}
}

9.Create Security Groups
The code is
**#Security groups

resource “aws_security_group” “wp_dev_sg” {
name = “wp_dev_sg”
description = “Used for access to the dev instance”
vpc_id = “${aws_vpc.wp_vpc.id}”

#SSH

ingress {
from_port = 22
to_port = 22
protocol = “tcp”
cidr_blocks = ["${var.localip}"]
}

#HTTP

ingress {
from_port = 80
to_port = 80
protocol = “tcp”
cidr_blocks = ["${var.localip}"]
}
egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}

#Public Security group

resource “aws_security_group” “wp_public_sg” {
name = “wp_public_sg”
description = “Used for public and private instances for load balancer access”
vpc_id = “${aws_vpc.wp_vpc.id}”

#HTTP

ingress {
from_port = 80
to_port = 80
protocol = “tcp”
cidr_blocks = [“0.0.0.0/0”]
}

#Outbound internet access

egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}

#Private Security Group

resource “aws_security_group” “wp_private_sg” {
name = “wp_private_sg”
description = “Used for private instances”
vpc_id = “${aws_vpc.wp_vpc.id}”

Access from other security groups

ingress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = ["${var.vpc_cidr}"]
}
egress {
from_port = 0
to_port = 0
protocol = “-1”
cidr_blocks = [“0.0.0.0/0”]
}
}

#RDS Security Group
resource “aws_security_group” “wp_rds_sg” {
name = “wp_rds_sg”
description = “Used for DB instances”
vpc_id = “${aws_vpc.wp_vpc.id}”

SQL access from public/private security group

ingress {
from_port = 3306
to_port = 3306
protocol = “tcp”

security_groups = ["${aws_security_group.wp_dev_sg.id}",
  "${aws_security_group.wp_public_sg.id}",
  "${aws_security_group.wp_private_sg.id}",
]

}
}

10. Creating S3 code bucket
The code is
#S3 code bucket

resource “random_id” “wp_code_bucket” {
byte_length = 2
}

resource “aws_s3_bucket” “code” {
bucket = “${var.domain_name}-${random_id.wp_code_bucket.dec}”
acl = “private”
force_destroy = true

tags {
Name = “code bucket”
}
}

11.Create Database instance and dev instance
The code is
**#---------compute-----------

resource “aws_db_instance” “wp_db” {
allocated_storage = 10
engine = “mysql”
engine_version = “5.6.27”
instance_class = “${var.db_instance_class}”
name = “${var.dbname}”
username = “${var.dbuser}”
password = “${var.dbpassword}”
db_subnet_group_name = “${aws_db_subnet_group.wp_rds_subnetgroup.name}”
vpc_security_group_ids = ["${aws_security_group.wp_rds_sg.id}"]
skip_final_snapshot = true
}

#key pair

resource “aws_key_pair” “wp_auth” {
key_name = “${var.key_name}”
public_key = “${file(var.public_key_path)}”
}

#dev server

resource “aws_instance” “wp_dev” {
instance_type = “${var.dev_instance_type}”
ami = “${var.dev_ami}”

tags {
Name = “wp_dev”
}

key_name = “${aws_key_pair.wp_auth.id}”
vpc_security_group_ids = ["${aws_security_group.wp_dev_sg.id}"]
iam_instance_profile = “${aws_iam_instance_profile.s3_access_profile.id}”
subnet_id = “${aws_subnet.wp_public1_subnet.id}”

provisioner “local-exec” {
command = <<EOD
cat < aws_hosts
[dev]
${aws_instance.wp_dev.public_ip}
[dev:vars]
s3code=${aws_s3_bucket.code.bucket}
domain=${var.domain_name}
EOF
EOD
}

provisioner “local-exec” {
command = “aws ec2 wait instance-status-ok --instance-ids ${aws_instance.wp_dev.id} --profile superhero && ansible-playbook -i aws_hosts wordpress.yml”
}
}

12. Creating Load Balancers
The code is
**#load balancer

resource “aws_elb” “wp_elb” {
name = “${var.domain_name}-elb”

subnets = ["${aws_subnet.wp_public1_subnet.id}",
“${aws_subnet.wp_public2_subnet.id}”,
]

security_groups = ["${aws_security_group.wp_public_sg.id}"]

listener {
instance_port = 80
instance_protocol = “http”
lb_port = 80
lb_protocol = “http”
}

health_check {
healthy_threshold = “${var.elb_healthy_threshold}”
unhealthy_threshold = “${var.elb_unhealthy_threshold}”
timeout = “${var.elb_timeout}”
target = “TCP:80”
interval = “${var.elb_interval}”
}

cross_zone_load_balancing = true
idle_timeout = 400
connection_draining = true
connection_draining_timeout = 400

tags {
Name = “wp_${var.domain_name}-elb”
}
}

13.Creating AMI
The code is
**resource “random_id” “golden_ami” {
byte_length = 8
}

resource “aws_ami_from_instance” “wp_golden” {
name = “wp_ami-${random_id.golden_ami.b64}”
source_instance_id = “${aws_instance.wp_dev.id}”

provisioner “local-exec” {
command = <<EOT
cat < userdata
#!/bin/bash
/usr/bin/aws s3 sync s3://${aws_s3_bucket.code.bucket} /var/www/html/
/bin/touch /var/spool/cron/root
sudo /bin/echo ‘*/5 * * * * aws s3 sync s3://${aws_s3_bucket.code.bucket} /var/www/html/’ >> /var/spool/cron/root
EOF
EOT
}
}**

14. Create AWS lauch Configuration
The code is
resource “aws_launch_configuration” “wp_lc” {
name_prefix = “wp_lc-”
image_id = “${aws_ami_from_instance.wp_golden.id}”
instance_type = “${var.lc_instance_type}”
security_groups = ["${aws_security_group.wp_private_sg.id}"]
iam_instance_profile = “${aws_iam_instance_profile.s3_access_profile.id}”
key_name = “${aws_key_pair.wp_auth.id}”
user_data = “${file(“userdata”)}”

lifecycle {
create_before_destroy = true
}
}

15. Create Auto Scaling Group
The Code is
#ASG

#resource “random_id” “rand_asg” {
byte_length = 8.
#}

resource “aws_autoscaling_group” “wp_asg” {
name = “asg-${aws_launch_configuration.wp_lc.id}”
max_size = “${var.asg_max}”
min_size = “${var.asg_min}”
health_check_grace_period = “${var.asg_grace}”
health_check_type = “${var.asg_hct}”
desired_capacity = “${var.asg_cap}”
force_delete = true
load_balancers = ["${aws_elb.wp_elb.id}"]

vpc_zone_identifier = ["${aws_subnet.wp_private1_subnet.id}",
“${aws_subnet.wp_private2_subnet.id}”,
]

launch_configuration = “${aws_launch_configuration.wp_lc.name}”

tag {
key = “Name”
value = “wp_asg-instance”
propagate_at_launch = true
}

lifecycle {
create_before_destroy = true
}
}

16.Create Route53
The code is
#---------Route53-------------

#primary zone

resource “aws_route53_zone” “primary” {
name = “${var.domain_name}.com”
delegation_set_id = “${var.delegation_set}”
}

#www

resource “aws_route53_record” “www” {
zone_id = “${aws_route53_zone.primary.zone_id}”
name = “www.${var.domain_name}.com”
type = “A”

alias {
name = “${aws_elb.wp_elb.dns_name}”
zone_id = “${aws_elb.wp_elb.zone_id}”
evaluate_target_health = false
}
}

#dev

resource “aws_route53_record” “dev” {
zone_id = “${aws_route53_zone.primary.zone_id}”
name = “dev.${var.domain_name}.com”
type = “A”
ttl = “300”
records = ["${aws_instance.wp_dev.public_ip}"]
}

#secondary zone

resource “aws_route53_zone” “secondary” {
name = “${var.domain_name}.com”
vpc_id = “${aws_vpc.wp_vpc.id}”
}

#db

resource “aws_route53_record” “db” {
zone_id = “${aws_route53_zone.secondary.zone_id}”
name = “db.${var.domain_name}.com”
type = “CNAME”
ttl = “300”
records = ["${aws_db_instance.wp_db.address}"]
}

These codes are all written in main.tf

3.We create empty variables in variables.tf

$ vim variables.tf
write empty variables as
variable “aws_region” {}
variable “aws_profile” {}
data “aws_availability_zones” “available” {}
variable “localip” {}
variable “vpc_cidr” {}

variable “cidrs” {
type = “map”
}

variable “db_instance_class” {}
variable “dbname” {}
variable “dbuser” {}
variable “dbpassword” {}
variable “key_name” {}
variable “public_key_path” {}
variable “domain_name” {}
variable “dev_instance_type” {}
variable “dev_ami” {}
variable “elb_healthy_threshold” {}
variable “elb_unhealthy_threshold” {}
variable “elb_timeout” {}
variable “elb_interval” {}
variable “asg_max” {}
variable “asg_min” {}
variable “asg_grace” {}
variable “asg_hct” {}
variable “asg_cap” {}
variable “lc_instance_type” {}
variable “delegation_set” {}

4. In terraform.tfvars we define variables
$vim terraform.tfvars
localip = “104.173.212.11/32”
aws_profile = “superhero”
aws_region = “us-east-1”
vpc_cidr = “10.0.0.0/16”
cidrs = {
public1 = “10.0.1.0/24”
public2 = “10.0.2.0/24”
private1 = “10.0.3.0/24”
private2 = “10.0.4.0/24”
rds1 = “10.0.5.0/24”
rds2 = “10.0.6.0/24”
rds3 = “10.0.7.0/24”
}
db_instance_class = “db.t2.micro”
dbname = “superherodb”
dbuser = “superhero”
dbpassword = “superheropass”
key_name = “kryptonite”
public_key_path = “/root/.ssh/kryptonite.pub”
domain_name = “bravethecloud”
dev_instance_type = “t2.micro”
dev_ami = “ami-b73b63a0”
elb_healthy_threshold = “2”
elb_unhealthy_threshold = “2”
elb_timeout = “3”
elb_interval = “30”
asg_max = “2”
asg_min = “1”
asg_grace = “300”
asg_hct = “EC2”
asg_cap = “2”
lc_instance_type = “t2.micro”
delegation_set = “N1HDAZB52OQ3IV”
test = {}

5. We write Ansible playbooks for downloading wordpress and creating buckets in s3update files
$vim wordpress.yml

  • hosts: dev
    become: yes
    remote_user: ec2-user
    tasks:
    • name: Install Apache.
      yum: name={{ item }} state=present
      with_items:
      • httpd
      • php
      • php-mysql
    • name: Download WordPress
      get_url: url=http://wordpress.org/wordpress-latest.tar.gz dest=/var/www/html/wordpress.tar.gz force=yes
    • name: Extract WordPress
      command: “tar xzf /var/www/html/wordpress.tar.gz -C /var/www/html --strip-components 1”
    • name: Make my directory tree readable
      file:
      path: /var/www/html/
      mode: u=rwX,g=rX,o=rX
      recurse: yes
      owner: apache
      group: apache
    • name: Make sure Apache is started now and at boot.
      service: name=httpd state=started enabled=yes

$vim s3update.yml

  • hosts: dev
    become: yes
    remote_user: ec2-user
    tasks:
    • name: Update S3 code bucket
      command: aws s3 sync /var/www/html/ s3://{{ s3code }}/ --delete
    • shell: echo “define(‘WP_SITEURL’,'http://dev.”{{ domain }}".com’);" >> wp-config.php
      args:
      chdir: /var/www/html/
    • shell: echo “define(‘WP_HOME’,'http://dev.”{{ domain }}".com’);" >> wp-config.php
      args:
      chdir: /var/www/html/

6. $vim aws_hosts
[dev]
34.239.130.247
[dev:vars]
s3code=bravethecloud_104
domain=bravethecloud

7.$vim userdata

#!/bin/bash
/usr/bin/aws s3 sync s3://bravethecloud_104 /var/www/html/
/bin/touch /var/spool/cron/root
sudo /bin/echo ‘*/5 * * * * aws s3 sync s3://bravethecloud_104 /var/www/html/’ >> /var/spool/cron/root