Ansible Automation Engine

Ansible
SSH
Published

August 9, 2018

Modified

April 10, 2024

Ansible Vagrant Examples

Find a couple of examples how to use Ansible and Vagrant on GitHub:

https://github.com/vpenso/vagrant-playground/tree/master/ansible

  • …useful for self educational to learn how Ansible works
  • …quick tests of configuration options and Ansible modules
  • …serve as skeletons do build development and test environments

Ansible manages IT infrastructure using secure shell (SSH) for access…

Essential features…

Terminology…

Documentation

Recommended introduction Ansible 101 video series on Youtube by Jeff Geerling

Entry point in the Ansible community documentation

# ...lists module plugins (by default)
ansible-doc -l

# ...choose plugin type, for example...
ansible-doc -l -t inventory

# ...ansible core modules
ansible-doc -l | grep ansible.builtin

# ...documentation for a given plugin, for example...
ansible-doc ansible.builtin.user
ansible-doc user                    # ...in shots

Community AnsibleFest, The Inside Playbook

Install

Source code available in the Github repository ansible/ansible

Installation from the command-line …cf. installation guide

# ...on all platforms
pip3 install ansible

# on Enterprise Linux
sudo dnf install -y ansible-core
sudo dnf install -y ansible          # ...batteries included

Fedora ansible-collection

# ...install the community.general collection
sudo dnf install -y ansible-collection-community-general

Check Ansible versions and configuration paths…

ansible --version

Configuration

Configuring Ansible

  • …settings adjustable via file ansible.cfg
  • …search paths for the configuration file
# ...in order of precedents
export ANSIBLE_CONFIG=${path}
$PWD/.ansible.cfg                                    # in path config
~/.ansible.cfg                                       # user config
/etc/ansible/ansible.cfg                             # global config

ansible-config …list available options …inspect the current values…

ansible-config init --disabled -t all

…reference the comprehensive list of the configuration settings

ansible.builtin.ssh connection …configure in ansible.cfg

[ssh_connection]
host_key_checking = false
ssh_common_args = "-o ForwardAgent=yes"

Host Inventory

Host inventory definition…

  • …track list of servers and devices that you want to automate
  • …groups …patterns to automate specific sub-sets of an inventory
  • …create your inventory file in one of many formats ..INI and YAML
# ...passing multiple inventory sources
ansible-playbook -i staging -i production #...
  • …list of connection variables for SSH
  • Default groups…
    • all …contains every host
    • ungrouped …hosts that don’t have another group

INI format example…

# variables applies to all groups
[all:vars]
ansible_user=root

# defining a group called example
[example]
node0
node1.example.org
node[2:8].example.org
node10:2222
node11 ansible_port=22 ansible_host=10.20.30.40 ansible_user=root

# variables for a specific group
[example:vars]
ansible_password=secret

Verify the inventory configuration with ansible-inventory

ansible-inventory --list

Playbooks

Ansible Playbooks

  • …repeatable, re-usable configuration management
  • …store playbooks under source control for reproducibility
  • …expressed in YAML format with a minimum of syntax
  • …composed of one or more ‘plays’ in an ordered list

Structure of a playbook (basically a list of plays)…

  • …keyword import_playbook …imports a list of plays
  • play …ties tasks to host list
    • hosts …hosts from the inventory to apply the configuration to
    • tasks …definition of a call to a module
    • …may include pre-tasks, post-tasks and handlers
---
- name: []                                       # ...of the play
  hosts: <pattern>                                 # groups or host patterns, separated by colons
  <key>: <value>                                  # ...options configurations
  []

  tasks:                                          # list of tasks
    - name: <comment>                             # ...task description
      <module>: [<options>]                       # ...module to call
        <key>: <value>                            # ...configuration for the module
        []
      notify:                                     # ...actions triggered on change
      - <key>: <value>                            # ...list of handlers
      []
    []
    - name: []                                   # another task...

  handlers:                                       # handler task list
    - name: <comment>
      service:
        <key>: <value

- name: []                                       # ...of another play
  hosts: []
  tasks: []

Tasks are executed in order…

  • …one at a time (…against machines defined in the host inventory)
  • …tasks execute a module with specific arguments
  • idempotents …allows repeated execution
  • notify is triggered at the end of each task for each defined handler
  • handlers are lists of tasks executed by notifiers

Execute a playbook…

  • ansible-playbook …executes playbooks on targeted hosts
  • ansible-pull …read from a remote version control systems

Best practices…

  • …use fully-qualified collection names for example ansible.builtin.copy
  • …generous use of whitespace, blank lines before blocks/tasks
  • …always name: plays and tasks …always mention state:
  • …use comments …any line starting with #
  • …use native YAML syntax, not the key=value shorthand
  • …use prefixes for variables …for example httpd_port

ansible-playbook

Example playbook to install a web-server (called http-server.yaml in the following):

---
- name: HTTP server installation and configuration
  hosts: alpha,beta
  become: true
  tasks:
    - name: install httpd package
      ansible.builtin.dnf:
        name:
          - httpd
          - httpd-devel
        state: latest
    - name: start and enable httpd service
      ansible.builtin.service:
        name: httpd
        enabled: true
        state: started
    - name: create a custom index.html file
      ansible.builtin.copy:
        dest: /var/www/html/index.html
        content: |
          Hallo World

ansible-playbook executes playbooks on targeted hosts

--limit

--limit …selected hosts with an additional pattern

  • …list all hosts in inventory ansible-inventory --list
  • …can be a host, IP-address or group
  • …multiple hosts alpha,beta
  • …negation all:!beta
  • …cf. patterns in ad-hoc commands
  • …reads the list of hosts from a file if prefixed with @
# ...on all nodes in the hosts list
ansible-playbook http-server.yml

# ...limit run to specific targets
ansible-playbook http-server.yml --limit alpha

# ...list host in file 
ansible-playbook --limit @retry_hosts.txt

--check & --diff

Check mode …runs without making any changes on remote systems

  • …combined with diff mode adds before-and-after comparisons
  • …prevent tasks from being run in check or diff mode with check_mode: false and diff: false
  • …skip a task when run in check mode with when: not ansible_check_mode
  • …ignore errors in check mode with ignore_errors: "{{ ansible_check_mode }}"
ansible-playbook --check --diff http_server.yml --limit alpha

Variables

Define variables in a variety of places…

  • …inventory
  • …playbook
  • …reusable files
  • …roles

…read understanding variable precedence

# ...defining varibales at run-time
ansible-playbook --extra-vars "http_port=80" #...

Defining variables in a playbook play…

- name: HTTP server configuration
  vars:
    http_port: 80

Defining variables in included files…

- name: HTTP server configuration
  vars_files:
    - vars.yaml

Referencing simple variables…

  • …using double curly braces {foo} (Jinja2 syntax)
  • …starting a value with a variable require quotes
  • …boolean typically use true/false or yes/no
- name: test the index page
  uri:
    # ...most basic form of variable substitution
    url: http://{{ansible_hostname}}/index.php
    status_code: 200

Strings

Strings over multiple lines…

  • | literal block scalar …preserve the newlines in the string
  • > folded block scalar …collapse all of the newlines in the string into a single space
var1: |
 exactly as you see
 will appear these three
 lines of text

var2: >
 this is really a
 single line of text
 despite appearances

Lists

…store as an itemized list or in square brackets []

# ...define variables with multiple values
foo:
  - zero
  - one
  - two

# ...referencing list variables
region: "{{ foo[0] }}"

Dictionary

…store the data in key-value pairs

# ...dictionary maps keys to values
foo:
  a: one
  b: two
  c: three

# ...referencing a key
foo.a
foo['b']

Use bracket notation if…

  • …keys start and end with two underscores
  • …key collides with known Python attribute

register

…creates variables from the output of an Ansible task…

  • …may be simple variables, list variables, dictionary variables, or complex nested data structures
  • …only valid on the host for the rest of the current playbook run
  • …stored in memory …not cached for use in future playbook runs
  • changed_when: false disables task change if appropriate
# ...store output of a command in a variable
- ansible.builtin.command: uname -r
  register: uname_r
  changed_when: false

# ...print variable in verbose mode
- ansible.builtin.debug:
    var: uname_r
    verbosity: 2

Environment Variables

Setting the remote environmentenvironment keyword …play, block, or task level

#...
  tasks:
    - name: download a file over an http proxy
      ansbile.builtin.get_url:
        url: #...
        dest: /tmp
      # ...add environment variables for this task
      environment:
        HTTP_PROXY: http://proxy.example.com:8080
        HTTPS_PROXY: http://proxy.example.com:8080
#...

…load environment variables in a task

# ...
  vars:
    proxy_vars: 
      HTTP_PROXY: http://proxy.example.com:8080
      HTTPS_PROXY: http://proxy.example.com:8080

  tasks:
    - name: download a file over an http proxy
      #... add environment variables from the playbook
      environment: proxy_vars

Modify the shell environment on a target node…

#...
  tasks:
    - name: permanently add an enviromment varibale to a users shell
      become: false
      ansible.builtin.lineinfile:
        dest: '~/.bash_profile'
        regexp: '^ENV_VAR='
        line: 'ENV_VAR=value'
#...

…using the ansible.builtin.lineinfile module

#...
  tasks:
    - name: read the value of an environment variable
      become: false
      ansible.builtin.shell: 'source ~/.bash_profile && echo $ENV_VAR'
      regsiter: env_var
    - ansible.builtin.debug: 
        msg: "Variable ENV_VAR={{env_var.stdout}}"
#...

Gather Facts

ansible.builtin.gather_facts …gathers facts about remote hosts

  • …facts are data related to your remote systems …for example IP address, operating system, etc.
  • …access this data in the ansible_facts variable
  • …some can be access as top-level variable with ansible_* prefix
#...
  - name: Print all available facts
    ansible.builtin.debug:
      var: ansible_facts
#...

Disable fact gathering in a playbook with the gather_facts: key…

---
- name: #...
  hosts: #...
  gather_facts: false
#...

hosts

Targeting hosts and groups…

  • …choose which managed nodes or groups you want to execute against
  • Patterns can refer to
    • …a single host
    • …an IP address
    • …an inventory group
    • …a set of groups
  • …either a comma , or a colon : to separate a list
all                   # all hosts in the inventory
node
node1,node2
group                 # from the inventory
group1,group2         # multiple groups
group2:!group2        # ...exclude a group
group1:&group2        # ...nodes in both groups

Conditionals

Conditionals controls the workflow of a playbook…

{pre,post}_tasks

Conditional execution block that runs before/after running the play…

---
- hosts: #...
  # ...
  pre_tasks:
    - name: update apt cache if needed
      apt:
        update_cache: true
        cache_valid_time: 3600
  #...
  post_tasks:
    - name: notify via mattermost
      community.general.mattermost:
        url: http://mattermost.example.com
        api_key: #...
        attachments:
          title: #...

when

…evaluates if a condition is met or satisfied…

  • …run a task depending on Ansible facts
  • …run a task based on the output of another task
  • …control the execution when a certain value is reached in Loop

true/false in a when clause…

# ...if the variable pam_access_all is false
- name: Create /etc/security/access.conf
  ansible.builtin.copy:
    dest: /etc/security/access.conf
    content: |
      #...
  when: not pam_access_all

# ...if the variable pam_access_all is true
- name: Create /etc/security/access.conf
  ansible.builtin.copy:
    dest: /etc/security/access.conf
    content: |
      #...
  when: pam_access_all

{changed,failed}_when

changed_when defines change for a particular task or if a handler should be triggered…

  • …based on return codes or output
  • …lists of multiple conditions are joined with an implicit and
  • …otherwise define the conditions in a string with an explicit or operator

…similar failed_when defines failure state for a task…

# ...searching for a word or phrase in the output of a command
- ansible.builtin.command: echo FAILED
  register: result
  failed_when: "'FAILED' in result.stderr"

# ...based on the return code
- name: Fail task when both files are identical
  ansible.builtin.command: false
  register: result
  failed_when: result.rc == 1

handlers

…running operations on change. Handlers

  • …named in order for tasks to be able to notify them
  • …alternative …utilize the listen keyword
  • …handlers run at the end of the playbook by default
#...
  handlers:
    - name: restart http server
      ansible.builtin.service:
        name: httpd
        state: restarted
      listen: httpd_restart
  
  tasks:
    - name: configure the http server
      ansible.builtin.copy:
        src: files/site.conf
        dest: /etc/httpd/conf.d/site.conf
      notify: httpd_restart
#...
  • …tasks instruct handlers to execute using the notify keyword
  • ansible-playbook --force-handlers …run handlers even if a task fails
  • meta: flush_handlers task triggers any handlers that have been notified at that point in the play
#...
  tasks: 
    - name: flush handlers immediatly
      meta: flush_handlers
#...

Tags

Tags used to run only specific parts of a playbook …using the tags: keyword

---
- hosts: #...
  tags: # add tags to a play

- hosts: #...
  tasks:
    - name: #...
      tags: # add tags to a task

    - name: #...
      tags: # add tags to a block
      block:
        - name: #...
          # ...
        - name: #...
          # ...

Tags reserved for special behaviour…

  • never …skip that task or play unless you specifically request it
  • always …run that task or play unless you specifically skip it

Command-line options for ansible-playbook

  • --list-tags …generate a list of available tags
  • --tags all …run all tasks, ignore tags (default behavior)
  • --tags [tag1, tag2] …run only tasks with either the tag tag1 or the tag tag2
  • --skip-tags [tag3, tag4] …run all tasks except those with either the tag tag3 or the tag tag4
  • --tags tagged …run only tasks with at least one tag
  • --tags untagged …run only tasks with no tags

Environment variables ANSIBLE_RUN_TAGS and ANSIBLE_SKIP_TAGS

lookup

Lookup plugins retrieve data from outside sources…

  • …such as files, databases, key/value stores, APIs, and other services
  • …Ansible-specific extension to the Jinja2 templating language
  • …evaluated on the Ansible control machine
    • executed with a working directory relative to the role or play
    • …return values are marked as unsafe for security reasons
  • …can be combined with filters, tests
vars:
  # ...populate variables using lookups
  file_contents: "{{ lookup('file', 'path/to/file.txt') }}"
  users:
    jdoe:
      name: "John Doe"
      mail: j.doe@example.org
 
tasks:
  # ...read an environment variable
  - ansible.builtin.debug:
    msg:  "'{{ lookup('env', 'HOME') }}' is the HOME environment variable."

  # ...read a URL
  - ansible.builtin.debug:
      msg: "{{ lookup('url', '<http://localhost:80/index.html>')}}"

  # ...read a dictionary
  - ansible.builtin.debug:
      msg: "User {{ item.key }} is {{ item.value.name }} ({{ item.value.mail }})"
    loop: "{{ lookup('dict', users) }}"

  # ...read a DNS record
  - ansible.builting.debug:
      msg: "{{ lookup ('dig', 'google.com')}}"

Documentation…

  • ansible-doc -t lookup -l …lists available lookup plugins
  • ansible-doc -t lookup $plugin_name …documentation for a specific plugins

Structure

Structural organisation of playbooks…

{import,include}_tasks

ansible.builtin.import_tasks module…

  • …imports a list of tasks to be added to the current playbook
  • statically includes from a referenced file before execution
  • ansible.builtin.include_tasks module …dynamically includes
---
- hosts: #...
  #...
  handlers:
    - ansible.builtin.import_tasks: handlers/httd.yaml
  tasks:
    - ansible.builtin.import_tasks: tasks/httpd.yml

import_playbook

ansible.builtin.import_playbook module…

  • …includes a file with a list of plays to be executed
  • can only be included at the top level (not inside a play)
---
- hosts: #...
  # ....

- import_playbook: playbook/httpd.yml

Ansible Vault

Ansible Vault is an encryption/decryption utility for data files

  • ansible-vault encrypts any structured data file
  • Uses with…
    • group_vars/ or host_vars/ inventory variables
    • …variables loaded by include_vars or vars_files
    • …playbook options -e @file.yml

Encrypt

Example vars/key.yaml file containing a secret…

---
key: "g4PIqsTK7xZjbc0vrvgWqP0SaR84Jj2rUK2motwjws9GUrjJpN"

ansible-vault encrypt ... supplied file

>>> ansible-vault encrypt vars/key.yaml 
New Vault password: 
Confirm New Vault password: 
Encryption successful

Other sub-commands…

  • view decrypt and view an existing vaulted file
  • edit open and decrypt an existing vaulted file in an editor
  • rekey change the password

Decrypt

Following playbook main.yaml exemplifies using encrypted data…

---
- hosts: localhost
  connection: local
  gather_facts: no
  become: false
  vars_files:
    - vars/key.yaml
  tasks:
    - ansible.builtin.command: "echo {{ key }}"
      register: echo_key
    - ansible.builtin.debug:
        var: echo_key.stdout

…run the playbook with option --ask-vault-pass

>>> ansible-playbook main.yaml --ask-vault-pass
Vault password:
#....
TASK [ansible.builtin.debug] *******************************************************
ok: [localhost] => {
    "echo_key.stdout": "g4PIqsTK7xZjbc0vrvgWqP0SaR84Jj2rUK2motwjws9GUrjJpN"
}
#...

…or use a (temporary) password file with option --vault-password-file

echo $password > pass.txt
ansible-playbook main.yaml --vault-password-file pass.txt

Roles

Roles bundle Ansible artifacts into a standardized structure…

├── defaults
│   └── main.yml
├── files
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── README.md
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

Minimal meta/main.yml required

---
dependencies: []

Include a role at the play level with the roles: keyword…

  • …treats the roles as static imports and processes them during playbook
  • …locate to the roles/ directory in the current working path
    • …by default main.yml files in all directories are loaded
    • …other files need to be referenced from main.yml files
  • …the role: option allows to define vars: and tags:
---
- hosts: #...
  roles:
    - time

    # ...add variables to a role
    - role: ldap
      vars:
        base: 'dc=foo,dc=bar'
        tls_cacert: /etc/ssl/certs/site.crt

    # ...fully qualified path to role
    - role: '/path/to/my/roles/common'

Configure roles_path in your ansible.cfg file…

  • …prevents lookup of roles from a global location
  • roles install will download roles into the configured path
[defaults]
#...
roles_path=./roles

Collections

Collections are a distribution format for Ansible content…

  • …bundles of modules, plug-ins, roles, and documentation
  • …released and maintained separately from main Ansible product releases
  • …naming conventions a-z, 0-9 and _ …must be valid Python identifiers
  • …Ansible certified and validated Content

Migrating Roles to Roles in Collections on Galaxy

  • …distribute roles in a single cohesive unit of re-usable automation
  • …share custom plugins across all roles in the collection
  • FQCN (Fully Qualified Collection Name)…
    • …combination of the collection namespace, collection name
    • …for example mynamespace.mycollection.role1

Collection directory structure…

mynamespace/
└── mycollection/
  ├── docs/
  ├── galaxy.yml    # ...role dependencies
  ├── plugins/
  ├── README.md
  ├── roles/        # ...one or more roles
  ├── playbooks/
  └── tests/

requirements.yml

Install dependency collections from a configuration file requirements.yml

---
roles:
  #... 

collections:
  - name: community.kubernetes
    version: 0.9.0

…using option -r to reference the file:

ansible-galaxy install -r requirements.yml

Best practice …generally recommends against installing collections into a global location

[defaults]
# ...collections relative to current path ./ansible_collections/namespace/collection_name
collections_paths = ./
# ...roles relative to current path ./roles/namespace.rolename
roles_path = ./roles
  • …requirements dedicated to a playbook (won’t affect any other playbooks)
  • …commit dependency roles and collections to your project code repository

ansible-galaxy

Ansible Galaxy

ansible-galaxy …perform various Role and Collection related operations

# ...name and version of each collection installed
ansible-galaxy collection list

# ..install a collection from Ansible Galaxy
ansible-galaxy collection install community.general

# ...local installation path for collections
ansible-config dump | grep -i ^collections_path

Testing & Linting

Testing spectrum …listed by increasing complexity…

  • yamllint …check YAML syntax
  • ansible-playbook --syntax-check …check if playbook is correct
  • ansible-lint …check compatibility to best practices
  • molecule test …integration test …run playbook from scratch
  • ansible-playbook --check …run against production infrastructure
  • …parallel test infrastructure

debug

ansible.builtin.debug ..print statements during execution

  • msg …customized message that is printed
  • var …variable name to debug …mutually exclusive with the msg
  • verbosity …for example set to 3 prints with command-line option -vvv or above
---
- hosts: localhost
  gather_facts: no
  connection: localhost

  tasks:
    - name: register output of uptime command
      ansible.builtin.command: uptime
      register: system_uptime

    - name: print registered output 
      ansible.builtin.debug:
        var: system_uptime.stdout

    - name: print message on command result changing
      ansible.builtin.debug:
        msg: "Uptime has changed!"
      when: system_uptime is changed

assert & fail

ansible.builtin.assert …asserts given expressions are true

  • that …list of string expressions
  • …with an optional custom message…
    • fail_msg …message used for a failing assertion
    • success_msg …message used for a successful assertion
---
- hosts: localhost
  gather_facts: no
  connection: localhost

  vars:
    false_value: false

  tasks:
    - name: stop if assertion false
      assert:
        that: "false_value == true"

ansible.builtin.fail …fail with custom message

yamllint

# ...install from RPM package
sudo dnf install -y yamllint

# ...as python package
pip3 install yamllint

Run the yamllint command to check for…

  • …syntax validity …by checking a series of rules
  • …key repetition
  • …cosmetic problems (line length, trailing spaces, indentation, etc.)
# ...all YAML files in a directory
yamllint .

# ...lint one or more files
yamllint my_file.yml my_other_file.yaml ...

Extend the configuration with a file ~/.yamllint

---
extends: default
rules:
  truthy: 
    allowed-values:
      - 'true'
      - 'false'
      - 'yes'
      - 'no'

ansible-lint

# ...install with RPM package
sudo dnf install -y python3-ansible-lint

ansible-lint …tool for linting playbooks, roles and collections…

  • …promote proven practices …avoiding common pitfalls
  • …opinionated …rules are the result of community contribution
  • …skip rules with an configuration file .ansible-lint-ignore
ansible-lint my_file.yml my_other_file.yaml ...

Molecule

# ...install as Python package
pip3 install molecule

Molecule …testing framework built specifically to test Ansible roles and playbooks within a collection

  • …launches an environment to run your playbook and then performs tests over it
  • scenario …test suite for roles or playbooks within a collection
# ...create a new role
>>> ansible-galaxy init somerole ; cd somerole
- Role somerole was created successfully

# ...create the Molecule configuration
molecule init scenario 

# ...create test environment ...run tests ...destory test environment
molecule test

# ...create test environment ..run tests ...stop
molecule converge

# ...access the test environment
molecule login

# ...destroy the test environment
molecule destroy

Default configuration in molecule/defaultscenario layout

  • molecule.yml …describes how integration test are executed
  • create.yml …used to create the instances
  • destroy.yml …remove the test instance
  • converge.yml …playbook run against the test instance

…optionally add more scenario with molecule init scenario $name

# molecule.yml
---
dependency:
  name: galaxy
  options:
    requirements-file: requirements.yml
platforms:
  - name: rockylinux
    image: quay.io/rockylinux/rockylinux

Molecule uses Ansible to manage instances to operate on…

ansible-pull

Pull configuration for a version control system and execute on localhost…

  • …inverts the default push architecture …near-limitless scaling potential
  • …utilizing the ansible-pull command
  • …run via cron and update playbook source via a source repository