Ansible Automation Engine
Find a couple of examples how to use Ansible and Vagrant on GitHub:
https://github.com/vpenso/vagrant-playground/tree/master/ansible
- …useful for self educational to learn how Ansible works
- …quick tests of configuration options and Ansible modules
- …serve as skeletons do build development and test environments
Ansible manages IT infrastructure using secure shell (SSH) for access…
- …software deployment, system configuration, and orchestrate more advanced IT tasks
- …provides a scalable engine to create consistency across IT infrastructure
- Used by… system administrators, release engineers, IT managers, developers
Essential features…
- …simplicity, easy-to-learn
- …language designed around auditability by humans
- …agentless …no central server required
- …written in Python …easily extendable with modules
- …push architecture …orchestrate configuration changes on a selection of targets
Terminology…
- …configuration changes …“actions” defined in a playbook…
- …pushed to a number of clients referenced by a host inventory file
- …playbooks consist of plays or tasks that call modules
- …computer used to run playbooks called “Ansible management node”
- …ad hoc commands …single linear commands …perform a one-time activity
Documentation
Recommended introduction Ansible 101 video series on Youtube by Jeff Geerling
- …Ansible for DevOps Book
- …corresponding manuscript on GitHub
Entry point in the Ansible community documentation…
- …plugin indexes and collection index
- …
ansible-doccommand displays information on plugins
# ...lists module plugins (by default)
ansible-doc -l
# ...choose plugin type, for example...
ansible-doc -l -t inventory
# ...ansible core modules
ansible-doc -l | grep ansible.builtin
# ...documentation for a given plugin, for example...
ansible-doc ansible.builtin.user
ansible-doc user # ...in shotsCommunity AnsibleFest, The Inside Playbook
Install
Source code available in the Github repository ansible/ansible…
Installation from the command-line …cf. installation guide
# ...on all platforms
pip3 install ansible
# on Enterprise Linux
sudo dnf install -y ansible-core
sudo dnf install -y ansible # ...batteries includedFedora ansible-collection…
# ...install the community.general collection
sudo dnf install -y ansible-collection-community-generalCheck Ansible versions and configuration paths…
ansible --versionConfiguration
- …settings adjustable via file
ansible.cfg - …search paths for the configuration file
# ...in order of precedents
export ANSIBLE_CONFIG=${path}
$PWD/.ansible.cfg # in path config
~/.ansible.cfg # user config
/etc/ansible/ansible.cfg # global configansible-config …list available options …inspect the current values…
ansible-config init --disabled -t all…reference the comprehensive list of the configuration settings
ansible.builtin.ssh connection …configure in ansible.cfg…
[ssh_connection]
host_key_checking = false
ssh_common_args = "-o ForwardAgent=yes"Host Inventory
Host inventory definition…
- …track list of servers and devices that you want to automate
- …groups …patterns to automate specific sub-sets of an inventory
- …create your inventory file in one of many formats ..INI and YAML
# ...passing multiple inventory sources
ansible-playbook -i staging -i production #...- …list of connection variables for SSH
- Default groups…
all…contains every hostungrouped…hosts that don’t have another group
INI format example…
# variables applies to all groups
[all:vars]
ansible_user=root
# defining a group called example
[example]
node0
node1.example.org
node[2:8].example.org
node10:2222
node11 ansible_port=22 ansible_host=10.20.30.40 ansible_user=root
# variables for a specific group
[example:vars]
ansible_password=secretVerify the inventory configuration with ansible-inventory…
ansible-inventory --listPlaybooks
- …repeatable, re-usable configuration management
- …store playbooks under source control for reproducibility
- …expressed in YAML format with a minimum of syntax
- …composed of one or more ‘plays’ in an ordered list
Structure of a playbook (basically a list of plays)…
- …keyword
import_playbook…imports a list of plays - …play …ties tasks to host list
- …
hosts…hosts from the inventory to apply the configuration to - …
tasks…definition of a call to a module - …may include pre-tasks, post-tasks and handlers
- …
---
- name: […] # ...of the play
hosts: <pattern> # groups or host patterns, separated by colons
<key>: <value> # ...options configurations
[…]
tasks: # list of tasks
- name: <comment> # ...task description
<module>: [<options>] # ...module to call
<key>: <value> # ...configuration for the module
[…]
notify: # ...actions triggered on change
- <key>: <value> # ...list of handlers
[…]
[…]
- name: […] # another task...
handlers: # handler task list
- name: <comment>
service:
<key>: <value
- name: […] # ...of another play
hosts: […]
tasks: […]Tasks are executed in order…
- …one at a time (…against machines defined in the host inventory)
- …tasks execute a module with specific arguments
- …idempotents …allows repeated execution
- …notify is triggered at the end of each task for each defined handler
- …handlers are lists of tasks executed by notifiers
Execute a playbook…
- …
ansible-playbook…executes playbooks on targeted hosts - …
ansible-pull…read from a remote version control systems
Best practices…
- …use fully-qualified collection names for example
ansible.builtin.copy - …generous use of whitespace, blank lines before blocks/tasks
- …always
name:plays and tasks …always mentionstate: - …use comments …any line starting with
# - …use native YAML syntax, not the
key=valueshorthand - …use prefixes for variables …for example
httpd_port
ansible-playbook
Example playbook to install a web-server (called http-server.yaml in the following):
---
- name: HTTP server installation and configuration
hosts: alpha,beta
become: true
tasks:
- name: install httpd package
ansible.builtin.dnf:
name:
- httpd
- httpd-devel
state: latest
- name: start and enable httpd service
ansible.builtin.service:
name: httpd
enabled: true
state: started
- name: create a custom index.html file
ansible.builtin.copy:
dest: /var/www/html/index.html
content: |
Hallo Worldansible-playbook executes playbooks on targeted hosts
--limit
…--limit …selected hosts with an additional pattern
- …list all hosts in inventory
ansible-inventory --list - …can be a host, IP-address or group
- …multiple hosts
alpha,beta - …negation
all:!beta - …cf. patterns in ad-hoc commands
- …reads the list of hosts from a file if prefixed with
@
# ...on all nodes in the hosts list
ansible-playbook http-server.yml
# ...limit run to specific targets
ansible-playbook http-server.yml --limit alpha
# ...list host in file
ansible-playbook --limit @retry_hosts.txt--check & --diff
Check mode …runs without making any changes on remote systems
- …combined with diff mode adds before-and-after comparisons
- …prevent tasks from being run in check or diff mode with
check_mode: falseanddiff: false - …skip a task when run in check mode with
when: not ansible_check_mode - …ignore errors in check mode with
ignore_errors: "{{ ansible_check_mode }}"
ansible-playbook --check --diff http_server.yml --limit alphaVariables
Define variables in a variety of places…
- …inventory
- …playbook
- …reusable files
- …roles
…read understanding variable precedence
# ...defining varibales at run-time
ansible-playbook --extra-vars "http_port=80" #...Defining variables in a playbook play…
- name: HTTP server configuration
vars:
http_port: 80Defining variables in included files…
- name: HTTP server configuration
vars_files:
- vars.yamlReferencing simple variables…
- …using double curly braces
{foo}(Jinja2 syntax) - …starting a value with a variable require quotes
- …boolean typically use
true/falseoryes/no
- name: test the index page
uri:
# ...most basic form of variable substitution
url: http://{{ansible_hostname}}/index.php
status_code: 200Strings
Strings over multiple lines…
- …
|literal block scalar …preserve the newlines in the string - …
>folded block scalar …collapse all of the newlines in the string into a single space
var1: |
exactly as you see
will appear these three
lines of text
var2: >
this is really a
single line of text
despite appearancesLists
…store as an itemized list or in square brackets []
# ...define variables with multiple values
foo:
- zero
- one
- two
# ...referencing list variables
region: "{{ foo[0] }}"Dictionary
…store the data in key-value pairs
# ...dictionary maps keys to values
foo:
a: one
b: two
c: three
# ...referencing a key
foo.a
foo['b']Use bracket notation if…
- …keys start and end with two underscores
- …key collides with known Python attribute
register
…creates variables from the output of an Ansible task…
- …may be simple variables, list variables, dictionary variables, or complex nested data structures
- …only valid on the host for the rest of the current playbook run
- …stored in memory …not cached for use in future playbook runs
- …
changed_when: falsedisables task change if appropriate
# ...store output of a command in a variable
- ansible.builtin.command: uname -r
register: uname_r
changed_when: false
# ...print variable in verbose mode
- ansible.builtin.debug:
var: uname_r
verbosity: 2Environment Variables
Setting the remote environment …environment keyword …play, block, or task level
#...
tasks:
- name: download a file over an http proxy
ansbile.builtin.get_url:
url: #...
dest: /tmp
# ...add environment variables for this task
environment:
HTTP_PROXY: http://proxy.example.com:8080
HTTPS_PROXY: http://proxy.example.com:8080
#...…load environment variables in a task
# ...
vars:
proxy_vars:
HTTP_PROXY: http://proxy.example.com:8080
HTTPS_PROXY: http://proxy.example.com:8080
tasks:
- name: download a file over an http proxy
#... add environment variables from the playbook
environment: proxy_varsModify the shell environment on a target node…
#...
tasks:
- name: permanently add an enviromment varibale to a users shell
become: false
ansible.builtin.lineinfile:
dest: '~/.bash_profile'
regexp: '^ENV_VAR='
line: 'ENV_VAR=value'
#...…using the ansible.builtin.lineinfile module
#...
tasks:
- name: read the value of an environment variable
become: false
ansible.builtin.shell: 'source ~/.bash_profile && echo $ENV_VAR'
regsiter: env_var
- ansible.builtin.debug:
msg: "Variable ENV_VAR={{env_var.stdout}}"
#...Gather Facts
ansible.builtin.gather_facts …gathers facts about remote hosts
- …facts are data related to your remote systems …for example IP address, operating system, etc.
- …access this data in the
ansible_factsvariable - …some can be access as top-level variable with
ansible_*prefix
#...
- name: Print all available facts
ansible.builtin.debug:
var: ansible_facts
#...Disable fact gathering in a playbook with the gather_facts: key…
---
- name: #...
hosts: #...
gather_facts: false
#...hosts
Targeting hosts and groups…
- …choose which managed nodes or groups you want to execute against
- Patterns can refer to
- …a single host
- …an IP address
- …an inventory group
- …a set of groups
- …either a comma
,or a colon:to separate a list
all # all hosts in the inventory
node
node1,node2
group # from the inventory
group1,group2 # multiple groups
group2:!group2 # ...exclude a group
group1:&group2 # ...nodes in both groupsConditionals
Conditionals controls the workflow of a playbook…
{pre,post}_tasks
Conditional execution block that runs before/after running the play…
---
- hosts: #...
# ...
pre_tasks:
- name: update apt cache if needed
apt:
update_cache: true
cache_valid_time: 3600
#...
post_tasks:
- name: notify via mattermost
community.general.mattermost:
url: http://mattermost.example.com
api_key: #...
attachments:
title: #...when
…evaluates if a condition is met or satisfied…
- …run a task depending on Ansible facts
- …run a task based on the output of another task
- …control the execution when a certain value is reached in Loop
true/false in a when clause…
# ...if the variable pam_access_all is false
- name: Create /etc/security/access.conf
ansible.builtin.copy:
dest: /etc/security/access.conf
content: |
#...
when: not pam_access_all
# ...if the variable pam_access_all is true
- name: Create /etc/security/access.conf
ansible.builtin.copy:
dest: /etc/security/access.conf
content: |
#...
when: pam_access_all{changed,failed}_when
changed_when defines change for a particular task or if a handler should be triggered…
- …based on return codes or output
- …lists of multiple conditions are joined with an implicit
and - …otherwise define the conditions in a string with an explicit
oroperator
…similar failed_when defines failure state for a task…
# ...searching for a word or phrase in the output of a command
- ansible.builtin.command: echo FAILED
register: result
failed_when: "'FAILED' in result.stderr"
# ...based on the return code
- name: Fail task when both files are identical
ansible.builtin.command: false
register: result
failed_when: result.rc == 1handlers
…running operations on change. Handlers…
- …named in order for tasks to be able to notify them
- …alternative …utilize the
listenkeyword - …handlers run at the end of the playbook by default
#...
handlers:
- name: restart http server
ansible.builtin.service:
name: httpd
state: restarted
listen: httpd_restart
tasks:
- name: configure the http server
ansible.builtin.copy:
src: files/site.conf
dest: /etc/httpd/conf.d/site.conf
notify: httpd_restart
#...- …tasks instruct handlers to execute using the
notifykeyword ansible-playbook --force-handlers…run handlers even if a task fails- …
meta: flush_handlerstask triggers any handlers that have been notified at that point in the play
#...
tasks:
- name: flush handlers immediatly
meta: flush_handlers
#...lookup
Lookup plugins retrieve data from outside sources…
- …such as files, databases, key/value stores, APIs, and other services
- …Ansible-specific extension to the Jinja2 templating language
- …evaluated on the Ansible control machine
- …executed with a working directory relative to the role or play
- …return values are marked as unsafe for security reasons
- …can be combined with filters, tests
vars:
# ...populate variables using lookups
file_contents: "{{ lookup('file', 'path/to/file.txt') }}"
users:
jdoe:
name: "John Doe"
mail: j.doe@example.org
tasks:
# ...read an environment variable
- ansible.builtin.debug:
msg: "'{{ lookup('env', 'HOME') }}' is the HOME environment variable."
# ...read a URL
- ansible.builtin.debug:
msg: "{{ lookup('url', '<http://localhost:80/index.html>')}}"
# ...read a dictionary
- ansible.builtin.debug:
msg: "User {{ item.key }} is {{ item.value.name }} ({{ item.value.mail }})"
loop: "{{ lookup('dict', users) }}"
# ...read a DNS record
- ansible.builting.debug:
msg: "{{ lookup ('dig', 'google.com')}}"Documentation…
ansible-doc -t lookup -l…lists available lookup pluginsansible-doc -t lookup $plugin_name…documentation for a specific plugins
Structure
Structural organisation of playbooks…
{import,include}_tasks
ansible.builtin.import_tasks module…
- …imports a list of tasks to be added to the current playbook
- …statically includes from a referenced file before execution
- …ansible.builtin.include_tasks module …dynamically includes
---
- hosts: #...
#...
handlers:
- ansible.builtin.import_tasks: handlers/httd.yaml
tasks:
- ansible.builtin.import_tasks: tasks/httpd.ymlimport_playbook
ansible.builtin.import_playbook module…
- …includes a file with a list of plays to be executed
- …can only be included at the top level (not inside a play)
---
- hosts: #...
# ....
- import_playbook: playbook/httpd.ymlAnsible Vault
Ansible Vault is an encryption/decryption utility for data files
- …
ansible-vaultencrypts any structured data file - Uses with…
- …
group_vars/orhost_vars/inventory variables - …variables loaded by
include_varsorvars_files - …playbook options
-e @file.yml
- …
Encrypt
Example vars/key.yaml file containing a secret…
---
key: "g4PIqsTK7xZjbc0vrvgWqP0SaR84Jj2rUK2motwjws9GUrjJpN"ansible-vault encrypt ... supplied file…
>>> ansible-vault encrypt vars/key.yaml
New Vault password:
Confirm New Vault password:
Encryption successfulOther sub-commands…
- …
viewdecrypt and view an existing vaulted file - …
editopen and decrypt an existing vaulted file in an editor - …
rekeychange the password
Decrypt
Following playbook main.yaml exemplifies using encrypted data…
---
- hosts: localhost
connection: local
gather_facts: no
become: false
vars_files:
- vars/key.yaml
tasks:
- ansible.builtin.command: "echo {{ key }}"
register: echo_key
- ansible.builtin.debug:
var: echo_key.stdout…run the playbook with option --ask-vault-pass…
>>> ansible-playbook main.yaml --ask-vault-pass
Vault password:
#....
TASK [ansible.builtin.debug] *******************************************************
ok: [localhost] => {
"echo_key.stdout": "g4PIqsTK7xZjbc0vrvgWqP0SaR84Jj2rUK2motwjws9GUrjJpN"
}
#...…or use a (temporary) password file with option --vault-password-file…
echo $password > pass.txt
ansible-playbook main.yaml --vault-password-file pass.txtRoles
Roles bundle Ansible artifacts into a standardized structure…
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── README.md
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.ymlMinimal meta/main.yml required
- …role dependencies and optional Galaxy metadata
- …see using role dependencies
---
dependencies: []Include a role at the play level with the roles: keyword…
- …treats the roles as static imports and processes them during playbook
- …locate to the
roles/directory in the current working path- …by default
main.ymlfiles in all directories are loaded - …other files need to be referenced from
main.ymlfiles
- …by default
- …the
role:option allows to definevars:andtags:
---
- hosts: #...
roles:
- time
# ...add variables to a role
- role: ldap
vars:
base: 'dc=foo,dc=bar'
tls_cacert: /etc/ssl/certs/site.crt
# ...fully qualified path to role
- role: '/path/to/my/roles/common'Configure roles_path in your ansible.cfg file…
- …prevents lookup of roles from a global location
- …
roles installwill download roles into the configured path
[defaults]
#...
roles_path=./roles- …multiple roles can be installed by listing them in a
requirements.ymlfile - …
init…initialize new role with the base structure of a role
Collections
Collections are a distribution format for Ansible content…
- …bundles of modules, plug-ins, roles, and documentation
- …released and maintained separately from main Ansible product releases
- …naming conventions
a-z,0-9and_…must be valid Python identifiers - …Ansible certified and validated Content
Migrating Roles to Roles in Collections on Galaxy…
- …distribute roles in a single cohesive unit of re-usable automation
- …share custom plugins across all roles in the collection
- FQCN (Fully Qualified Collection Name)…
- …combination of the collection namespace, collection name
- …for example
mynamespace.mycollection.role1
Collection directory structure…
mynamespace/
└── mycollection/
├── docs/
├── galaxy.yml # ...role dependencies
├── plugins/
├── README.md
├── roles/ # ...one or more roles
├── playbooks/
└── tests/requirements.yml
Install dependency collections from a configuration file requirements.yml…
---
roles:
#...
collections:
- name: community.kubernetes
version: 0.9.0…using option -r to reference the file:
ansible-galaxy install -r requirements.ymlBest practice …generally recommends against installing collections into a global location
[defaults]
# ...collections relative to current path ./ansible_collections/namespace/collection_name
collections_paths = ./
# ...roles relative to current path ./roles/namespace.rolename
roles_path = ./roles- …requirements dedicated to a playbook (won’t affect any other playbooks)
- …commit dependency roles and collections to your project code repository
ansible-galaxy
- …hub for finding and sharing Ansible content
- …pre-packaged units of work known to Ansible as Roles
- …finding Ansible roles using the search function
- How to evaluate community Ansible roles for your playbooks
ansible-galaxy …perform various Role and Collection related operations
# ...name and version of each collection installed
ansible-galaxy collection list
# ..install a collection from Ansible Galaxy
ansible-galaxy collection install community.general
# ...local installation path for collections
ansible-config dump | grep -i ^collections_pathTesting & Linting
Testing spectrum …listed by increasing complexity…
yamllint…check YAML syntaxansible-playbook --syntax-check…check if playbook is correctansible-lint…check compatibility to best practicesmolecule test…integration test …run playbook from scratchansible-playbook --check…run against production infrastructure- …parallel test infrastructure
debug
ansible.builtin.debug ..print statements during execution
msg…customized message that is printedvar…variable name to debug …mutually exclusive with themsgverbosity…for example set to 3 prints with command-line option-vvvor above
---
- hosts: localhost
gather_facts: no
connection: localhost
tasks:
- name: register output of uptime command
ansible.builtin.command: uptime
register: system_uptime
- name: print registered output
ansible.builtin.debug:
var: system_uptime.stdout
- name: print message on command result changing
ansible.builtin.debug:
msg: "Uptime has changed!"
when: system_uptime is changedassert & fail
ansible.builtin.assert …asserts given expressions are true
that…list of string expressions- …with an optional custom message…
- …
fail_msg…message used for a failing assertion - …
success_msg…message used for a successful assertion
- …
---
- hosts: localhost
gather_facts: no
connection: localhost
vars:
false_value: false
tasks:
- name: stop if assertion false
assert:
that: "false_value == true"ansible.builtin.fail …fail with custom message
yamllint
# ...install from RPM package
sudo dnf install -y yamllint
# ...as python package
pip3 install yamllintRun the yamllint command to check for…
- …syntax validity …by checking a series of rules
- …key repetition
- …cosmetic problems (line length, trailing spaces, indentation, etc.)
# ...all YAML files in a directory
yamllint .
# ...lint one or more files
yamllint my_file.yml my_other_file.yaml ...Extend the configuration with a file ~/.yamllint…
---
extends: default
rules:
truthy:
allowed-values:
- 'true'
- 'false'
- 'yes'
- 'no'ansible-lint
# ...install with RPM package
sudo dnf install -y python3-ansible-lintansible-lint …tool for linting playbooks, roles and collections…
- …promote proven practices …avoiding common pitfalls
- …opinionated …rules are the result of community contribution
- …skip rules with an configuration file
.ansible-lint-ignore
ansible-lint my_file.yml my_other_file.yaml ...Molecule
# ...install as Python package
pip3 install moleculeMolecule …testing framework built specifically to test Ansible roles and playbooks within a collection
- …launches an environment to run your playbook and then performs tests over it
- …scenario …test suite for roles or playbooks within a collection
# ...create a new role
>>> ansible-galaxy init somerole ; cd somerole
- Role somerole was created successfully
# ...create the Molecule configuration
molecule init scenario
# ...create test environment ...run tests ...destory test environment
molecule test
# ...create test environment ..run tests ...stop
molecule converge
# ...access the test environment
molecule login
# ...destroy the test environment
molecule destroyDefault configuration in molecule/default …scenario layout
molecule.yml…describes how integration test are executedcreate.yml…used to create the instancesdestroy.yml…remove the test instanceconverge.yml…playbook run against the test instance
…optionally add more scenario with molecule init scenario $name
# molecule.yml
---
dependency:
name: galaxy
options:
requirements-file: requirements.yml
platforms:
- name: rockylinux
image: quay.io/rockylinux/rockylinuxMolecule uses Ansible to manage instances to operate on…
- …work is offloaded to the
provisioner - …
driverspecified inmolecule.yml.. - Cf. using Podman containers
ansible-pull
Pull configuration for a version control system and execute on localhost…
- …inverts the default push architecture …near-limitless scaling potential
- …utilizing the
ansible-pullcommand - …run via cron and update playbook source via a source repository