Terraform in deployment pipeline
3 likes2,996 views
The document discusses the integration of Terraform in deployment pipelines, highlighting goals such as managing infrastructure and structuring code effectively. It provides an overview of infrastructure as code and deployment pipelines, along with examples of Terraform configurations and best practices to avoid common issues. Furthermore, it emphasizes the importance of versioning, reusability of code through modules, and automation in managing infrastructure environments.
![example2/main.tf
variable "subnet_id" {
description = "ID of subnet where resources will be created"
}
variable "security_groups" {
description = "ID of security group EC2 instance will use"
}
variable "instance_type" {
description = "Type of EC2 instance to launch"
}
data "aws_ami" "app" {
most_recent = true
filter {
name = "name"
values = ["fullstackfest-demo-*" ]
}
}
resource "aws_instance" "app" {
ami = "${data. aws_ami.app.id}"
instance_type = "${var.instance_type }"
subnet_id = "${var.subnet_id }"
vpc_security_group_ids = ["${var.security_groups }"]
}
output "app_public_ip" {
description = "Public IP of EC2 instance running an
application"
value = "${aws_instance .app.public_ip }"
}
packer/app.json
{
"builders" : [
{
"ami_name" : "fullstackfest-demo-{{uuid | clean_ami_name}}" ,
"ami_description" : "FullStackFest demo AMI based on Amazon
Linux",
"instance_type" : "t2.micro" ,
"region" : "eu-west-1" ,
"type": "amazon-ebs" ,
"ssh_username" : "ec2-user" ,
"source_ami_filter" : {
"filters" : {
"virtualization-type" : "hvm",
"name": "amzn-ami-hvm-*-x86_64-gp2" ,
"root-device-type" : "ebs"
},
"owners" : [
"137112412989"
],
"most_recent" : true
}
}
],
"provisioners" : [
{
"type": "shell",
"inline" : [
"# Install nginx, copy index.html into web-root"
]
}
]
}](https://image.slidesharecdn.com/terraformindeploymentpipeline1-170904151244/85/Terraform-in-deployment-pipeline-13-320.jpg)
![# Avoid hard-coded values in *.tf files, use data sources or
*.tfvars
data "aws_ami" "app" {
most_recent = true
filter {
name = "name"
values = ["fullstackfest-demo-*" ]
}
}
# Tag and name resources consistently
resource "aws_instance" "app" {
ami = "${data. aws_ami.app.id}"
instance_type = "${var.instance_type }"
subnet_id = "${var.subnet_id }"
vpc_security_group_ids = ["${var.security_groups }"]
tags {
Name = "fullstackfest-demo-${var. environment }"
}
}
variable "environment" {
description = "Name of environment to create infrastructure (eg,
staging, production)"
}
# terraform.tfvars
environment = "non-prod"
FTP (Frequent Terraform Problems)
● Avoid hard-coded values => use data
sources
● Tag and name resources consistently
Next: Terraform modules = reusability](https://image.slidesharecdn.com/terraformindeploymentpipeline1-170904151244/85/Terraform-in-deployment-pipeline-14-320.jpg)
![module "sg_web" {
source = "git@github.com:terraform-community-modules/tf_aws_sg.git//sg_web?ref=v0.2.3"
security_group_name = "fullstackfest-demo-web"
vpc_id = "vpc-12345678"
source_cidr_block = ["0.0.0.0/0" ]
}
resource "aws_instance" "app" {
# ...
vpc_security_group_ids = ["${module. sg_web.security_group_id_web }"]
# ...
}
Terraform modules
● Versioning
● Public/private access
● Local dir or hosted
● Allows:
○ code reuse
○ encapsulate groups of resources
○ testing](https://image.slidesharecdn.com/terraformindeploymentpipeline1-170904151244/85/Terraform-in-deployment-pipeline-15-320.jpg)
Terraform in deployment pipeline
- 1. Terraform in deployment pipeline by Anton Babenko
- 2. Hi! I am Anton Babenko, and I enjoy: ● AWS & DevOps ● AWS User Group Norway and DevOpsDays Oslo organizer ● Solve problems “Getting Started with Terraform”, terraform-community-modules, Terraform modules generator (Terrapin), and more… https://github.com/antonbabenko https://www.linkedin.com/in/antonbabenko
- 3. 1. Become more familiar with managing infrastructure using CD pipeline 2. See scenarios of integrating Terraform and Packer 3. How to structure infrastructure code? 4. How to version your infrastructure between environments and make it DRY? Goals of this talk
- 4. Do you know? What is: ● Infrastructure as code? ● Deployment pipeline? ● Pipeline as code?
- 5. Featuring... Write, plan, and create infrastructure as code Build automated machine images
- 7. Typical CI/CD pipeline source: https://dzone.com/articles/what-is-continuous-delivery-pipeline
- 8. Where are infrastructure changes here?
- 9. CI/CD pipeline (CircleCI 2.0)
- 10. Structure - all-in-one vs split ~/all-in-one-repo/ ├── packer # Packer configs │ └── app.json ├── terraform # Terraform configs │ ├── main.tf │ └── terraform.tfvars └── web # Application code └── index.html ~/infra-repo/ ├── packer # Packer configs │ └── app.json └── terraform # Terraform configs ├── main.tf └── terraform.tfvars ~/app-repo/ └── web # Application code └── index.html
- 11. Structure - evolving infrastructure repository ~/infra-repo/ ├── packer # Packer configs │ └── app.json └── terraform # Terraform configs ├── modules # Terraform modules │ ├── network │ │ └── main.tf │ └── service1 │ └── main.tf ├── main.tf └── terraform.tfvars ~/infra-repo/ ├── packer # Packer configs │ └── app.json └── terraform # Terraform configs ├── modules # Terraform modules │ ├── network │ │ └── main.tf │ └── service1 │ └── main.tf └── environments ├── non-prod │ └── us-east-1 │ ├── main.tf │ └── terraform.tfvars └── prod ├── eu-west-1 │ ├── main.tf │ └── terraform.tfvars └── us-east-1 ├── main.tf └── terraform.tfvars
- 12. example1/main.tf resource "random_pet" "bucket" {} resource "aws_s3_bucket" "app" { bucket = "fullstackfest-${ random_pet .bucket. id}" acl = "public-read" website { index_document = "index.html" } } data "template_file" "index" { template = "${file("../../web/index.html")}" vars { BUILD_DETAILS = "${aws_s3_bucket .app.website_endpoint }" } } resource "aws_s3_bucket_object" "object" { bucket = "${aws_s3_bucket .app.id}" key = "index.html" content = "${data. template_file .index.rendered }" etag = "${md5(data. template_file .index.rendered )}" content_type = "text/html" acl = "public-read" } output "app_website_endpoint" { value = "${aws_s3_bucket .app.website_endpoint }" } FullStackFest! ${BUILD_DETAILS} $ terraform init ... $ terraform plan ... $ terraform apply ... Apply complete! Resources: 3 added, 0 changed, 0 destroyed. Outputs: app_website_endpoint = fullstackfest-feasible-basilisk.s3-website-eu-west-1.amazonaws.c om
- 13. example2/main.tf variable "subnet_id" { description = "ID of subnet where resources will be created" } variable "security_groups" { description = "ID of security group EC2 instance will use" } variable "instance_type" { description = "Type of EC2 instance to launch" } data "aws_ami" "app" { most_recent = true filter { name = "name" values = ["fullstackfest-demo-*" ] } } resource "aws_instance" "app" { ami = "${data. aws_ami.app.id}" instance_type = "${var.instance_type }" subnet_id = "${var.subnet_id }" vpc_security_group_ids = ["${var.security_groups }"] } output "app_public_ip" { description = "Public IP of EC2 instance running an application" value = "${aws_instance .app.public_ip }" } packer/app.json { "builders" : [ { "ami_name" : "fullstackfest-demo-{{uuid | clean_ami_name}}" , "ami_description" : "FullStackFest demo AMI based on Amazon Linux", "instance_type" : "t2.micro" , "region" : "eu-west-1" , "type": "amazon-ebs" , "ssh_username" : "ec2-user" , "source_ami_filter" : { "filters" : { "virtualization-type" : "hvm", "name": "amzn-ami-hvm-*-x86_64-gp2" , "root-device-type" : "ebs" }, "owners" : [ "137112412989" ], "most_recent" : true } } ], "provisioners" : [ { "type": "shell", "inline" : [ "# Install nginx, copy index.html into web-root" ] } ] }
- 14. # Avoid hard-coded values in *.tf files, use data sources or *.tfvars data "aws_ami" "app" { most_recent = true filter { name = "name" values = ["fullstackfest-demo-*" ] } } # Tag and name resources consistently resource "aws_instance" "app" { ami = "${data. aws_ami.app.id}" instance_type = "${var.instance_type }" subnet_id = "${var.subnet_id }" vpc_security_group_ids = ["${var.security_groups }"] tags { Name = "fullstackfest-demo-${var. environment }" } } variable "environment" { description = "Name of environment to create infrastructure (eg, staging, production)" } # terraform.tfvars environment = "non-prod" FTP (Frequent Terraform Problems) ● Avoid hard-coded values => use data sources ● Tag and name resources consistently Next: Terraform modules = reusability
- 15. module "sg_web" { source = "[email protected]:terraform-community-modules/tf_aws_sg.git//sg_web?ref=v0.2.3" security_group_name = "fullstackfest-demo-web" vpc_id = "vpc-12345678" source_cidr_block = ["0.0.0.0/0" ] } resource "aws_instance" "app" { # ... vpc_security_group_ids = ["${module. sg_web.security_group_id_web }"] # ... } Terraform modules ● Versioning ● Public/private access ● Local dir or hosted ● Allows: ○ code reuse ○ encapsulate groups of resources ○ testing
- 16. Demo - infrastructure as code and deployment pipeline https://github.com/antonbabenko/terraform-deployment-pipeline-talk
- 17. Further thoughts… ● Use linters (tflint), coding styles (fmt), pre-commit hooks ● Automate (no excuses) ● Terraform workspaces, Terragrunt, Atlantis by Hootsuite (super!) ● Version & release infrastructure as the app code ● Using pipelines to manage environments with infrastructure as code by Kief Morris
- 18. Thank you!