Ansible Automation Engine
Find a couple of examples how to use Ansible and Vagrant on GitHub:
https://github.com/vpenso/vagrant-playground/tree/master/ansible
- …useful for self educational to learn how Ansible works
- …quick tests of configuration options and Ansible modules
- …serve as skeletons do build development and test environments
Ansible manages IT infrastructure using secure shell (SSH) for access…
- …software deployment, system configuration, and orchestrate more advanced IT tasks
- …provides a scalable engine to create consistency across IT infrastructure
- Used by… system administrators, release engineers, IT managers, developers
Essential features…
- …simplicity, easy-to-learn
- …language designed around auditability by humans
- …agentless …no central server required
- …written in Python …easily extendable with modules
- …push architecture …orchestrate configuration changes on a selection of targets
Terminology…
- …configuration changes …“actions” defined in a playbook…
- …pushed to a number of clients referenced by a host inventory file
- …playbooks consist of plays or tasks that call modules
- …computer used to run playbooks called “Ansible management node”
- …ad hoc commands …single linear commands …perform a one-time activity
Documentation
Recommended introduction Ansible 101 video series on Youtube by Jeff Geerling
- …Ansible for DevOps Book
- …corresponding manuscript on GitHub
Entry point in the Ansible community documentation…
- …plugin indexes and collection index
- …
ansible-doc
command displays information on plugins
# ...lists module plugins (by default)
ansible-doc -l
# ...choose plugin type, for example...
ansible-doc -l -t inventory
# ...ansible core modules
ansible-doc -l | grep ansible.builtin
# ...documentation for a given plugin, for example...
ansible-doc ansible.builtin.user
ansible-doc user # ...in shots
Community AnsibleFest, The Inside Playbook
Install
Source code available in the Github repository ansible/ansible
…
Installation from the command-line …cf. installation guide
# ...on all platforms
pip3 install ansible
# on Enterprise Linux
sudo dnf install -y ansible-core
sudo dnf install -y ansible # ...batteries included
Fedora ansible-collection
…
# ...install the community.general collection
sudo dnf install -y ansible-collection-community-general
Check Ansible versions and configuration paths…
ansible --version
Configuration
- …settings adjustable via file
ansible.cfg
- …search paths for the configuration file
# ...in order of precedents
export ANSIBLE_CONFIG=${path}
$PWD/.ansible.cfg # in path config
~/.ansible.cfg # user config
/etc/ansible/ansible.cfg # global config
ansible-config
…list available options …inspect the current values…
ansible-config init --disabled -t all
…reference the comprehensive list of the configuration settings
ansible.builtin.ssh connection …configure in ansible.cfg
…
[ssh_connection]
host_key_checking = false
ssh_common_args = "-o ForwardAgent=yes"
Host Inventory
Host inventory definition…
- …track list of servers and devices that you want to automate
- …groups …patterns to automate specific sub-sets of an inventory
- …create your inventory file in one of many formats ..INI and YAML
# ...passing multiple inventory sources
ansible-playbook -i staging -i production #...
- …list of connection variables for SSH
- Default groups…
all
…contains every hostungrouped
…hosts that don’t have another group
INI format example…
# variables applies to all groups
[all:vars]
ansible_user=root
# defining a group called example
[example]
node0
node1.example.org
node[2:8].example.org
node10:2222
node11 ansible_port=22 ansible_host=10.20.30.40 ansible_user=root
# variables for a specific group
[example:vars]
ansible_password=secret
Verify the inventory configuration with ansible-inventory
…
ansible-inventory --list
Playbooks
- …repeatable, re-usable configuration management
- …store playbooks under source control for reproducibility
- …expressed in YAML format with a minimum of syntax
- …composed of one or more ‘plays’ in an ordered list
Structure of a playbook (basically a list of plays)…
- …keyword
import_playbook
…imports a list of plays - …play …ties tasks to host list
- …
hosts
…hosts from the inventory to apply the configuration to - …
tasks
…definition of a call to a module - …may include pre-tasks, post-tasks and handlers
- …
---
- name: […] # ...of the play
hosts: <pattern> # groups or host patterns, separated by colons
<key>: <value> # ...options configurations
[…]
tasks: # list of tasks
- name: <comment> # ...task description
<module>: [<options>] # ...module to call
<key>: <value> # ...configuration for the module
[…]
notify: # ...actions triggered on change
- <key>: <value> # ...list of handlers
[…]
[…]
- name: […] # another task...
handlers: # handler task list
- name: <comment>
service:
<key>: <value
- name: […] # ...of another play
hosts: […]
tasks: […]
Tasks are executed in order…
- …one at a time (…against machines defined in the host inventory)
- …tasks execute a module with specific arguments
- …idempotents …allows repeated execution
- …notify is triggered at the end of each task for each defined handler
- …handlers are lists of tasks executed by notifiers
Execute a playbook…
- …
ansible-playbook
…executes playbooks on targeted hosts - …
ansible-pull
…read from a remote version control systems
Best practices…
- …use fully-qualified collection names for example
ansible.builtin.copy
- …generous use of whitespace, blank lines before blocks/tasks
- …always
name:
plays and tasks …always mentionstate:
- …use comments …any line starting with
#
- …use native YAML syntax, not the
key=value
shorthand - …use prefixes for variables …for example
httpd_port
ansible-playbook
Example playbook to install a web-server (called http-server.yaml
in the following):
---
- name: HTTP server installation and configuration
hosts: alpha,beta
become: true
tasks:
- name: install httpd package
ansible.builtin.dnf:
name:
- httpd
- httpd-devel
state: latest
- name: start and enable httpd service
ansible.builtin.service:
name: httpd
enabled: true
state: started
- name: create a custom index.html file
ansible.builtin.copy:
dest: /var/www/html/index.html
content: |
Hallo World
ansible-playbook executes playbooks on targeted hosts
--limit
…--limit
…selected hosts with an additional pattern
- …list all hosts in inventory
ansible-inventory --list
- …can be a host, IP-address or group
- …multiple hosts
alpha,beta
- …negation
all:!beta
- …cf. patterns in ad-hoc commands
- …reads the list of hosts from a file if prefixed with
@
# ...on all nodes in the hosts list
ansible-playbook http-server.yml
# ...limit run to specific targets
ansible-playbook http-server.yml --limit alpha
# ...list host in file
ansible-playbook --limit @retry_hosts.txt
--check
& --diff
Check mode …runs without making any changes on remote systems
- …combined with diff mode adds before-and-after comparisons
- …prevent tasks from being run in check or diff mode with
check_mode: false
anddiff: false
- …skip a task when run in check mode with
when: not ansible_check_mode
- …ignore errors in check mode with
ignore_errors: "{{ ansible_check_mode }}"
ansible-playbook --check --diff http_server.yml --limit alpha
Variables
Define variables in a variety of places…
- …inventory
- …playbook
- …reusable files
- …roles
…read understanding variable precedence
# ...defining varibales at run-time
ansible-playbook --extra-vars "http_port=80" #...
Defining variables in a playbook play…
- name: HTTP server configuration
vars:
http_port: 80
Defining variables in included files…
- name: HTTP server configuration
vars_files:
- vars.yaml
Referencing simple variables…
- …using double curly braces
{foo}
(Jinja2 syntax) - …starting a value with a variable require quotes
- …boolean typically use
true
/false
oryes
/no
- name: test the index page
uri:
# ...most basic form of variable substitution
url: http://{{ansible_hostname}}/index.php
status_code: 200
Strings
Strings over multiple lines…
- …
|
literal block scalar …preserve the newlines in the string - …
>
folded block scalar …collapse all of the newlines in the string into a single space
var1: |
exactly as you see
will appear these three
lines of text
var2: >
this is really a
single line of text despite appearances
Lists
…store as an itemized list or in square brackets []
# ...define variables with multiple values
foo:
- zero
- one
- two
# ...referencing list variables
region: "{{ foo[0] }}"
Dictionary
…store the data in key-value pairs
# ...dictionary maps keys to values
foo:
a: one
b: two
c: three
# ...referencing a key
foo.a
foo['b']
Use bracket notation if…
- …keys start and end with two underscores
- …key collides with known Python attribute
register
…creates variables from the output of an Ansible task…
- …may be simple variables, list variables, dictionary variables, or complex nested data structures
- …only valid on the host for the rest of the current playbook run
- …stored in memory …not cached for use in future playbook runs
- …
changed_when: false
disables task change if appropriate
# ...store output of a command in a variable
- ansible.builtin.command: uname -r
register: uname_r
changed_when: false
# ...print variable in verbose mode
- ansible.builtin.debug:
var: uname_r
verbosity: 2
Environment Variables
Setting the remote environment …environment
keyword …play, block, or task level
#...
tasks:
- name: download a file over an http proxy
ansbile.builtin.get_url:
url: #...
dest: /tmp
# ...add environment variables for this task
environment:
HTTP_PROXY: http://proxy.example.com:8080
HTTPS_PROXY: http://proxy.example.com:8080
#...
…load environment variables in a task
# ...
vars:
proxy_vars:
HTTP_PROXY: http://proxy.example.com:8080
HTTPS_PROXY: http://proxy.example.com:8080
tasks:
- name: download a file over an http proxy
#... add environment variables from the playbook
environment: proxy_vars
Modify the shell environment on a target node…
#...
tasks:
- name: permanently add an enviromment varibale to a users shell
become: false
ansible.builtin.lineinfile:
dest: '~/.bash_profile'
regexp: '^ENV_VAR='
line: 'ENV_VAR=value'
#...
…using the ansible.builtin.lineinfile module
#...
tasks:
- name: read the value of an environment variable
become: false
ansible.builtin.shell: 'source ~/.bash_profile && echo $ENV_VAR'
regsiter: env_var
- ansible.builtin.debug:
msg: "Variable ENV_VAR={{env_var.stdout}}"
#...
Gather Facts
ansible.builtin.gather_facts …gathers facts about remote hosts
- …facts are data related to your remote systems …for example IP address, operating system, etc.
- …access this data in the
ansible_facts
variable - …some can be access as top-level variable with
ansible_*
prefix
#...
- name: Print all available facts
ansible.builtin.debug:
var: ansible_facts
#...
Disable fact gathering in a playbook with the gather_facts:
key…
---
- name: #...
hosts: #...
gather_facts: false
#...
hosts
Targeting hosts and groups…
- …choose which managed nodes or groups you want to execute against
- Patterns can refer to
- …a single host
- …an IP address
- …an inventory group
- …a set of groups
- …either a comma
,
or a colon:
to separate a list
all # all hosts in the inventory
node
node1,node2
group # from the inventory
group1,group2 # multiple groups
group2:!group2 # ...exclude a group
group1:&group2 # ...nodes in both groups
Conditionals
Conditionals controls the workflow of a playbook…
{pre,post}_tasks
Conditional execution block that runs before/after running the play…
---
- hosts: #...
# ...
pre_tasks:
- name: update apt cache if needed
apt:
update_cache: true
cache_valid_time: 3600
#...
post_tasks:
- name: notify via mattermost
community.general.mattermost:
url: http://mattermost.example.com
api_key: #...
attachments:
title: #...
when
…evaluates if a condition is met or satisfied…
- …run a task depending on Ansible facts
- …run a task based on the output of another task
- …control the execution when a certain value is reached in Loop
true
/false
in a when
clause…
# ...if the variable pam_access_all is false
- name: Create /etc/security/access.conf
ansible.builtin.copy:
dest: /etc/security/access.conf
content: |
#... when: not pam_access_all
# ...if the variable pam_access_all is true
- name: Create /etc/security/access.conf
ansible.builtin.copy:
dest: /etc/security/access.conf
content: |
#... when: pam_access_all
{changed,failed}_when
changed_when
defines change for a particular task or if a handler should be triggered…
- …based on return codes or output
- …lists of multiple conditions are joined with an implicit
and
- …otherwise define the conditions in a string with an explicit
or
operator
…similar failed_when
defines failure state for a task…
# ...searching for a word or phrase in the output of a command
- ansible.builtin.command: echo FAILED
register: result
failed_when: "'FAILED' in result.stderr"
# ...based on the return code
- name: Fail task when both files are identical
ansible.builtin.command: false
register: result
failed_when: result.rc == 1
handlers
…running operations on change. Handlers…
- …named in order for tasks to be able to notify them
- …alternative …utilize the
listen
keyword - …handlers run at the end of the playbook by default
#...
handlers:
- name: restart http server
ansible.builtin.service:
name: httpd
state: restarted
listen: httpd_restart
tasks:
- name: configure the http server
ansible.builtin.copy:
src: files/site.conf
dest: /etc/httpd/conf.d/site.conf
notify: httpd_restart
#...
- …tasks instruct handlers to execute using the
notify
keyword ansible-playbook --force-handlers
…run handlers even if a task fails- …
meta: flush_handlers
task triggers any handlers that have been notified at that point in the play
#...
tasks:
- name: flush handlers immediatly
meta: flush_handlers
#...
lookup
Lookup plugins retrieve data from outside sources…
- …such as files, databases, key/value stores, APIs, and other services
- …Ansible-specific extension to the Jinja2 templating language
- …evaluated on the Ansible control machine
- …executed with a working directory relative to the role or play
- …return values are marked as unsafe for security reasons
- …can be combined with filters, tests
vars:
# ...populate variables using lookups
file_contents: "{{ lookup('file', 'path/to/file.txt') }}"
users:
jdoe:
name: "John Doe"
mail: j.doe@example.org
tasks:
# ...read an environment variable
- ansible.builtin.debug:
msg: "'{{ lookup('env', 'HOME') }}' is the HOME environment variable."
# ...read a URL
- ansible.builtin.debug:
msg: "{{ lookup('url', '<http://localhost:80/index.html>')}}"
# ...read a dictionary
- ansible.builtin.debug:
msg: "User {{ item.key }} is {{ item.value.name }} ({{ item.value.mail }})"
loop: "{{ lookup('dict', users) }}"
# ...read a DNS record
- ansible.builting.debug:
msg: "{{ lookup ('dig', 'google.com')}}"
Documentation…
ansible-doc -t lookup -l
…lists available lookup pluginsansible-doc -t lookup $plugin_name
…documentation for a specific plugins
Structure
Structural organisation of playbooks…
{import,include}_tasks
ansible.builtin.import_tasks module…
- …imports a list of tasks to be added to the current playbook
- …statically includes from a referenced file before execution
- …ansible.builtin.include_tasks module …dynamically includes
---
- hosts: #...
#...
handlers:
- ansible.builtin.import_tasks: handlers/httd.yaml
tasks:
- ansible.builtin.import_tasks: tasks/httpd.yml
import_playbook
ansible.builtin.import_playbook module…
- …includes a file with a list of plays to be executed
- …can only be included at the top level (not inside a play)
---
- hosts: #...
# ....
- import_playbook: playbook/httpd.yml
Ansible Vault
Ansible Vault is an encryption/decryption utility for data files
- …
ansible-vault
encrypts any structured data file - Uses with…
- …
group_vars/
orhost_vars/
inventory variables - …variables loaded by
include_vars
orvars_files
- …playbook options
-e @file.yml
- …
Encrypt
Example vars/key.yaml
file containing a secret…
---
key: "g4PIqsTK7xZjbc0vrvgWqP0SaR84Jj2rUK2motwjws9GUrjJpN"
ansible-vault encrypt ...
supplied file…
>>> ansible-vault encrypt vars/key.yaml
New Vault password:
Confirm New Vault password:
Encryption successful
Other sub-commands…
- …
view
decrypt and view an existing vaulted file - …
edit
open and decrypt an existing vaulted file in an editor - …
rekey
change the password
Decrypt
Following playbook main.yaml
exemplifies using encrypted data…
---
- hosts: localhost
connection: local
gather_facts: no
become: false
vars_files:
- vars/key.yaml
tasks:
- ansible.builtin.command: "echo {{ key }}"
register: echo_key
- ansible.builtin.debug:
var: echo_key.stdout
…run the playbook with option --ask-vault-pass
…
>>> ansible-playbook main.yaml --ask-vault-pass
Vault password:
#....
TASK [ansible.builtin.debug] *******************************************************
ok: [localhost] => {
"echo_key.stdout": "g4PIqsTK7xZjbc0vrvgWqP0SaR84Jj2rUK2motwjws9GUrjJpN"
}
#...
…or use a (temporary) password file with option --vault-password-file
…
echo $password > pass.txt
ansible-playbook main.yaml --vault-password-file pass.txt
Roles
Roles bundle Ansible artifacts into a standardized structure…
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── README.md
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars └── main.yml
Minimal meta/main.yml
required
- …role dependencies and optional Galaxy metadata
- …see using role dependencies
---
dependencies: []
Include a role at the play level with the roles:
keyword…
- …treats the roles as static imports and processes them during playbook
- …locate to the
roles/
directory in the current working path- …by default
main.yml
files in all directories are loaded - …other files need to be referenced from
main.yml
files
- …by default
- …the
role:
option allows to definevars:
andtags:
---
- hosts: #...
roles:
- time
# ...add variables to a role
- role: ldap
vars:
base: 'dc=foo,dc=bar'
tls_cacert: /etc/ssl/certs/site.crt
# ...fully qualified path to role
- role: '/path/to/my/roles/common'
Configure roles_path
in your ansible.cfg
file…
- …prevents lookup of roles from a global location
- …
roles install
will download roles into the configured path
[defaults]
#...
roles_path=./roles
- …multiple roles can be installed by listing them in a
requirements.yml
file - …
init
…initialize new role with the base structure of a role
Collections
Collections are a distribution format for Ansible content…
- …bundles of modules, plug-ins, roles, and documentation
- …released and maintained separately from main Ansible product releases
- …naming conventions
a-z
,0-9
and_
…must be valid Python identifiers - …Ansible certified and validated Content
Migrating Roles to Roles in Collections on Galaxy…
- …distribute roles in a single cohesive unit of re-usable automation
- …share custom plugins across all roles in the collection
- FQCN (Fully Qualified Collection Name)…
- …combination of the collection namespace, collection name
- …for example
mynamespace.mycollection.role1
Collection directory structure…
mynamespace/
└── mycollection/
├── docs/
├── galaxy.yml # ...role dependencies
├── plugins/
├── README.md
├── roles/ # ...one or more roles
├── playbooks/ └── tests/
requirements.yml
Install dependency collections from a configuration file requirements.yml
…
---
roles:
#...
collections:
- name: community.kubernetes
version: 0.9.0
…using option -r
to reference the file:
ansible-galaxy install -r requirements.yml
Best practice …generally recommends against installing collections into a global location
[defaults]
# ...collections relative to current path ./ansible_collections/namespace/collection_name
collections_paths = ./
# ...roles relative to current path ./roles/namespace.rolename
roles_path = ./roles
- …requirements dedicated to a playbook (won’t affect any other playbooks)
- …commit dependency roles and collections to your project code repository
ansible-galaxy
- …hub for finding and sharing Ansible content
- …pre-packaged units of work known to Ansible as Roles
- …finding Ansible roles using the search function
- How to evaluate community Ansible roles for your playbooks
ansible-galaxy
…perform various Role and Collection related operations
# ...name and version of each collection installed
ansible-galaxy collection list
# ..install a collection from Ansible Galaxy
ansible-galaxy collection install community.general
# ...local installation path for collections
ansible-config dump | grep -i ^collections_path
Testing & Linting
Testing spectrum …listed by increasing complexity…
yamllint
…check YAML syntaxansible-playbook --syntax-check
…check if playbook is correctansible-lint
…check compatibility to best practicesmolecule test
…integration test …run playbook from scratchansible-playbook --check
…run against production infrastructure- …parallel test infrastructure
debug
ansible.builtin.debug ..print statements during execution
msg
…customized message that is printedvar
…variable name to debug …mutually exclusive with themsg
verbosity
…for example set to 3 prints with command-line option-vvv
or above
---
- hosts: localhost
gather_facts: no
connection: localhost
tasks:
- name: register output of uptime command
ansible.builtin.command: uptime
register: system_uptime
- name: print registered output
ansible.builtin.debug:
var: system_uptime.stdout
- name: print message on command result changing
ansible.builtin.debug:
msg: "Uptime has changed!"
when: system_uptime is changed
assert
& fail
ansible.builtin.assert …asserts given expressions are true
that
…list of string expressions- …with an optional custom message…
- …
fail_msg
…message used for a failing assertion - …
success_msg
…message used for a successful assertion
- …
---
- hosts: localhost
gather_facts: no
connection: localhost
vars:
false_value: false
tasks:
- name: stop if assertion false
assert:
that: "false_value == true"
ansible.builtin.fail …fail with custom message
yamllint
# ...install from RPM package
sudo dnf install -y yamllint
# ...as python package
pip3 install yamllint
Run the yamllint
command to check for…
- …syntax validity …by checking a series of rules
- …key repetition
- …cosmetic problems (line length, trailing spaces, indentation, etc.)
# ...all YAML files in a directory
yamllint .
# ...lint one or more files
yamllint my_file.yml my_other_file.yaml ...
Extend the configuration with a file ~/.yamllint
…
---
extends: default
rules:
truthy:
allowed-values:
- 'true'
- 'false'
- 'yes'
- 'no'
ansible-lint
# ...install with RPM package
sudo dnf install -y python3-ansible-lint
ansible-lint
…tool for linting playbooks, roles and collections…
- …promote proven practices …avoiding common pitfalls
- …opinionated …rules are the result of community contribution
- …skip rules with an configuration file
.ansible-lint-ignore
ansible-lint my_file.yml my_other_file.yaml ...
Molecule
# ...install as Python package
pip3 install molecule
Molecule …testing framework built specifically to test Ansible roles and playbooks within a collection
- …launches an environment to run your playbook and then performs tests over it
- …scenario …test suite for roles or playbooks within a collection
# ...create a new role
>>> ansible-galaxy init somerole ; cd somerole
- Role somerole was created successfully
# ...create the Molecule configuration
molecule init scenario
# ...create test environment ...run tests ...destory test environment
molecule test
# ...create test environment ..run tests ...stop
molecule converge
# ...access the test environment
molecule login
# ...destroy the test environment
molecule destroy
Default configuration in molecule/default
…scenario layout
molecule.yml
…describes how integration test are executedcreate.yml
…used to create the instancesdestroy.yml
…remove the test instanceconverge.yml
…playbook run against the test instance
…optionally add more scenario with molecule init scenario $name
# molecule.yml
---
dependency:
name: galaxy
options:
requirements-file: requirements.yml
platforms:
- name: rockylinux
image: quay.io/rockylinux/rockylinux
Molecule uses Ansible to manage instances to operate on…
- …work is offloaded to the
provisioner
- …
driver
specified inmolecule.yml
.. - Cf. using Podman containers
ansible-pull
Pull configuration for a version control system and execute on localhost…
- …inverts the default push architecture …near-limitless scaling potential
- …utilizing the
ansible-pull
command - …run via cron and update playbook source via a source repository