Download as PDF, PPTX
![Packer
source "azure-arm" "basic-example" {
…
vm_size = "Standard_A2"
}
build {
sources = ["sources.azure-arm.basic-example"]
provisioner "shell" {
scripts = ["scripts/common/puppet_install.sh"]
}
}
$ packer build template. hcl
https://github.com/petems/puppet-install-shell](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-8-2048.jpg)
![Vagrant & Puppet
cat Vagrantfile
case node["provision_type"]
when 'masterless'
srv.vm.synced_folder "#{environment}/hieradata",
"/etc/puppetlabs/code/environments/#{environment}/hieradata"
srv.vm.provision :puppet do |puppet|
puppet.environment_path = "."
puppet.hiera_config_path = "#{environment}/hiera.yaml"
end
$ vagrant (up,halt,destroy)](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-13-2048.jpg)
![Vagrant & Puppet
cat Vagrantfile
case node["provision_type"]
when 'puppet_agent'
srv.vm.provision "puppet_server" do |puppet|
puppet.options = "-t --environment #{environment}"
puppet.puppet_server = "puppetmaster.#{environment}.vagrant"
end
srv.trigger.after :destroy do |trigger|
trigger.name = "Cleaning puppet certificate"
trigger.run = {inline: "vagrant ssh puppetmaster -c 'sudo /opt/puppetlabs/bin/puppetserver ca clean --certname
#{node["hostname"]}'"}
end
$ vagrant (up,halt,destroy)](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-14-2048.jpg)
![Lets magic a node
resource "azurerm_virtual_machine" "puppetmaster" {
name = "vm-${var.node_name}"
location = "${var.azure_region_name}"
resource_group_name = "${var.resource_group}"
network_interface_ids = ["${azurerm_network_interface.puppetmaster.id}"]
vm_size = "${var.vm_size}"
storage_image_reference {
id = "${var.packerimage_id}" # ← your packer image id
}
os_profile {
...
}
}](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-17-2048.jpg)
![Geolocation Stuff
consul_prepared_query { 'puppetmaster':
ensure => 'present',
service_name => 'puppetmaster',
service_failover_n => 1,
service_failover_dcs => [ 'failover', 'dr' ],
service_only_passing => true,
ttl => 10,
}
dig @127.0.0.1 -p 8600 puppetmaster.query.consul ANY](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-25-2048.jpg)
![Fast Exported Resources
::consul::watch { 'detect_backend_changes':
type => 'service',
handler => '/usr/bin/runpuppet.sh',
service => 'node_exporter',
passingonly => true,
require => File['/usr/bin/runpuppet.sh'],
}](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-28-2048.jpg)
![Pesky Passwords
class profiles::security::vault (
Variant[Hash, Array[Hash]] $listener = {
'tcp' => {
'address' => '127.0.0.1:8200',
'cluster_address' => '127.0.0.1:8201',
},
},
Hash $storage = { 'consul' => { 'address' => '127.0.0.1:8500', 'path' => 'vault/' }},
){
class {'vault':
enable_ui => true,
listener => $listener,
storage => $storage,
}
https://forge.puppet.com/jsok/vault](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-30-2048.jpg)
![Puppet > 6
$password = Deferred('vault_lookup::lookup',["password/mysql"], 'https://active.vault.service.consul:8200',)
class { '::mysql::server':
root_password => $password,
}](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-33-2048.jpg)
![Waypoint
• Modern workflow to build, deploy, and release across platforms.
• Build
• Deploy
• Release
https://www.waypointproject.io/
[WIP] https://github.com/attachmentgenie/attachmentgenie-waypoint](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-36-2048.jpg)
![Boundary
• Identity-based access for zero trust security
• Authenticate & authorize
• Connect
• Access
https://www.boundaryproject.io/
[WIP] https://github.com/attachmentgenie/attachmentgenie-boundary](https://image.slidesharecdn.com/paths-211002221807/75/Puppet-and-the-HashiStack-37-2048.jpg)
Uploaded byBram Vogelaar
Puppet and the HashiStack
Puppet is an open source configuration management tool that can be used to automate the configuration and management of infrastructure and applications. It uses a client-server architecture and declarative language to define and enforce the desired state of systems. Other HashiCorp tools like Packer, Terraform, Vault and Nomad can integrate with Puppet for tasks like infrastructure provisioning, secrets management and workload orchestration. Bolt is a task orchestration tool from Puppet that can be used to automate operational tasks across infrastructure defined by tools like Terraform. Consul provides service discovery and configuration for the Puppet infrastructure.
More Related Content
Puppet and the HashiStack
- 1. Puppet and theHashiStack Bram Vogelaar @attachmentgenie
- 2. $ whoami • Usedto be a Molecular Biologist • Then became a Dev • Now an Ops • Currently Cloud Engineer @ The Factory
- 3.
- 4. Puppet l Open Sourceconfiguration management tool since 2005 l Desired State DSL on top of ruby l Client – Server architecture l Runs on pretty much any OS l Combines node info (Facter), node config (Hiera), node model (DSL) into a catalogue that is applies at regular intervals https://puppet.com/open-source/#osp
- 5.
- 6. Packer l Open Sourcetool to make OS images l Supports Cloud Providers, Docker, Virtualbox, … (builders) l Has hooks to provision the base images (provisioners) l Create artifacts (Post-Processors) https://www.packer.io/
- 7. HCL2 l HashiCorp ConfigurationLanguage l Yet another config management DSL l Desired state l Used by multiple HashiCorp tools but also 3rd party tools https://github.com/hashicorp/hcl
- 8. Packer source "azure-arm" "basic-example"{ … vm_size = "Standard_A2" } build { sources = ["sources.azure-arm.basic-example"] provisioner "shell" { scripts = ["scripts/common/puppet_install.sh"] } } $ packer build template. hcl https://github.com/petems/puppet-install-shell
- 9. Packer & Puppet provisioner"puppet-masterless" { manifest_file = "site.pp" } provisioner "puppet-server" { puppet_server = ”puppet.example.com" }
- 10. Vagrant l Open Sourcetool to bootstrap local vms - Build by packer! l Supports many vm Providers, Docker, Virtualbox, … l Has hooks to provision the base images (provisioners), Puppet, Chef, Ansible, Bash https://www.vagrantup.com/
- 11. Vagrantfile Vagrant.configure("2") do |config| config.vm.box= "base" # config.vm.box_check_update = false # config.vm.network "forwarded_port", guest: 80, host: 8080 # config.vm.network "private_network", ip: "192.168.33.10" # config.vm.network "public_network" # config.vm.synced_folder "../data", "/vagrant_data" # config.vm.provider "virtualbox" do |vb| # vb.gui = true # vb.memory = "1024" # end # config.vm.provision "shell", inline: <<-SHELL # apt-get update # apt-get install -y apache2 # SHELL end
- 12. Vagrant & Puppet $vagrant plugin install vagrant-puppet-install https://github.com/petems/vagrant-puppet-install
- 13. Vagrant & Puppet catVagrantfile case node["provision_type"] when 'masterless' srv.vm.synced_folder "#{environment}/hieradata", "/etc/puppetlabs/code/environments/#{environment}/hieradata" srv.vm.provision :puppet do |puppet| puppet.environment_path = "." puppet.hiera_config_path = "#{environment}/hiera.yaml" end $ vagrant (up,halt,destroy)
- 14. Vagrant & Puppet catVagrantfile case node["provision_type"] when 'puppet_agent' srv.vm.provision "puppet_server" do |puppet| puppet.options = "-t --environment #{environment}" puppet.puppet_server = "puppetmaster.#{environment}.vagrant" end srv.trigger.after :destroy do |trigger| trigger.name = "Cleaning puppet certificate" trigger.run = {inline: "vagrant ssh puppetmaster -c 'sudo /opt/puppetlabs/bin/puppetserver ca clean --certname #{node["hostname"]}'"} end $ vagrant (up,halt,destroy)
- 15. Terraform l Open SourceAutomation Tool l “cloud” oriented l Cloud are API’s l API’s oriented l Terraform is an open source automation tool which can deal with any kind of CRUD api’s – including major cloud providers https://www.terraform.io/
- 16. The Terraform Model lYou model your infrastructure l You make a plan l If ok, you apply that plan l Current state is saved for future changes $ terraform (fmt, validate, plan, apply)
- 17. Lets magic anode resource "azurerm_virtual_machine" "puppetmaster" { name = "vm-${var.node_name}" location = "${var.azure_region_name}" resource_group_name = "${var.resource_group}" network_interface_ids = ["${azurerm_network_interface.puppetmaster.id}"] vm_size = "${var.vm_size}" storage_image_reference { id = "${var.packerimage_id}" # ← your packer image id } os_profile { ... } }
- 18. Lets magic anode resource "azurerm_virtual_machine" "puppetmaster" { ... os_profile { custom_data = "${data.template_file.bootstrap_sh.rendered}" } } output "private_ip" { value = "${azurerm_network_interface.network_interface.private_ip_address}" }
- 19. Bootstrap your PuppetServer #!/usr/bin/env bash apt-get install puppetserver puppet config set --section agent environment ${environment} systemctl start puppetserver
- 20. Move it allto a module module "web" { source = "../modules/azure/instance" domain = "${var.domain}" environment = "${var.environment}" node_name = "web" private_subnet = "${module.inuits_play_bootstrap.subnet_id}" puppet_master = "${var.puppet_master}" }
- 21. It’s a DNSproblem
- 22. Consul l Open SourceService Discovery Tool l Build-in KV store l Service Mesh tool https://www.consul.io/
- 23. Consul & Puppet class{ '::consul': config_hash => $config, version => $version, } ::consul::service { 'puppetmaster': port => 8140, } ::consul::check { 'puppetmaster_tcp': interval => '10s', tcp => 'localhost:8140', notes => 'Puppetmasters listen on port 8140', service_id => 'puppetmaster’, } dig @127.0.0.1 -p 8600 puppetmaster.service.consul ANY https://forge.puppet.com/KyleAnderson/consul
- 24. Consul~Icinga(2) Exit Codes ::consul::check{ 'puppetmaster': interval => '10s', script => '/usr/lib64/nagios/plugins/puppetmaster', notes => 'Puppetmasters listen on port 8140', service_id => 'puppetmaster', }
- 25. Geolocation Stuff consul_prepared_query {'puppetmaster': ensure => 'present', service_name => 'puppetmaster', service_failover_n => 1, service_failover_dcs => [ 'failover', 'dr' ], service_only_passing => true, ttl => 10, } dig @127.0.0.1 -p 8600 puppetmaster.query.consul ANY
- 26. X509 is hard #!/usr/bin/envbash apt-get install puppetserver puppet config set --section agent environment ${environment} puppet config set --section master dns_alt_names puppet,puppetmaster.service.consul,puppetmaster.query.consul puppet config set --section master autosign /etc/puppetlabs/puppet/autosign-policy.rb systemctl start puppetserver
- 27. So what aboutthat chicken? #!/usr/bin/env bash consul agent -retry-join "provider=azure tag_name=consul tenant_id=${tid} client_id=${cid} subscription_id=${sid} secret_access_key=${ro_key}” cat > /etc/puppetlabs/puppet/csr_attributes.yaml << YAML --- extension_requests: pp_preshared_key: ${psk} pp_role: ${role} YAML puppet config set --section agent environment ${environment} puppet config set --section agent server puppet.query.consul /opt/puppetlabs/bin/puppet agent -t
- 28. Fast Exported Resources ::consul::watch{ 'detect_backend_changes': type => 'service', handler => '/usr/bin/runpuppet.sh', service => 'node_exporter', passingonly => true, require => File['/usr/bin/runpuppet.sh'], }
- 29. Vault l Open Sourcetool to do secrets management l Secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API. l Certificate management l Password rotation https://www.vaultproject.io/
- 30. Pesky Passwords class profiles::security::vault( Variant[Hash, Array[Hash]] $listener = { 'tcp' => { 'address' => '127.0.0.1:8200', 'cluster_address' => '127.0.0.1:8201', }, }, Hash $storage = { 'consul' => { 'address' => '127.0.0.1:8500', 'path' => 'vault/' }}, ){ class {'vault': enable_ui => true, listener => $listener, storage => $storage, } https://forge.puppet.com/jsok/vault
- 31. Pesky Passwords $ vaultunseal $ vault write kv/my-secret value="s3c(eT" $ vault read kv/mysecret Key Value --- ----- refresh_interval 768h mysecret s3c(eT
- 32. Vault & Hiera cathiera.yaml --- version: 5 hierarchy: - name: "Hiera-vault lookup” lookup_key: hiera_vault options: confine_to_keys: - '^sensitive_.*’ - '^password.*’ address: http://active.vault.service.consul:8200 mounts: generic: - secret/puppet/%{::trusted.certname}/ - secret/puppet/common/ https://github.com/petems/hiera_backend_vault
- 33. Puppet > 6 $password= Deferred('vault_lookup::lookup',["password/mysql"], 'https://active.vault.service.consul:8200',) class { '::mysql::server': root_password => $password, }
- 34. Nomad l Open Sourcetool to do dynamic workload scheduling l Batch, containerized, and non-containerized applications. l Has native Consul and Vault integrations. https://www.nomadproject.io/
- 35. Nomad & Puppet class{ '::nomad': config_hash = { 'client’ => { 'enabled' => true, }, 'consul’ => { 'address' => '127.0.0.1:8500’, },ß 'server' => { 'enabled' => true 'bootstrap_expect' => 3, }, 'vault’ => { 'enabled’ => true, 'address’ => 'https://active.vault.service.consul:8200', 'create_from_role' => 'nomad-cluster', 'token’ => 's.krHYYcd8PRzpSBO59AE6sawO', }, } } https://forge.puppet.com/modules/puppet/nomad
- 36. Waypoint • Modern workflowto build, deploy, and release across platforms. • Build • Deploy • Release https://www.waypointproject.io/ [WIP] https://github.com/attachmentgenie/attachmentgenie-waypoint
- 37. Boundary • Identity-based accessfor zero trust security • Authenticate & authorize • Connect • Access https://www.boundaryproject.io/ [WIP] https://github.com/attachmentgenie/attachmentgenie-boundary
- 38. Bolt l Open Sourcetool to do task orchestration l Aimed at fire and forget 2nd day operations l Support tasks written in Puppet or YAML https://puppet.com/open-source/bolt/
- 39. Bolt # bolt-project.yaml modulepath: - modules modules: -name: puppetlabs/apt # inventory.yaml groups: - name: all-of-azure targets: - uri: 1.2.3.4:22 name: prod config: transport: ssh ssh: user: ubuntu
- 40. Bolt bolt project init boltmodule install bolt command run apt action=update --targets servers
- 41. Bolt & Terraform groups: -name: all-of-azure targets: - _plugin: terraform dir: /path/to/terraform/project resource_type: azurerm_virtual_machine target_mapping: uri: public_ip plan do_important_stuff (TargetSpec $targets){ run_task('terraform::initialize', 'dir' => '/path/to/terraform/project') $apply_result = run_plan('terraform::apply', 'dir' => '/path/to/terraform/project', 'return_output' => true) run_task('important::stuff', $targets, 'task_var' => $apply_result) } https://forge.puppet.com/modules/puppetlabs/terraform
- 42. Bolt & Vault targets: -... config: ssh: user: root private-key: key-data: _plugin: vault server_url: https://active.vault.service.consul:8200 auth: method: token token: xxxxx-xxxxx path: secrets/bolt field: private-key https://forge.puppet.com/modules/puppetlabs/vault
- 43.
- 44. The Floor isyours… Questions ?