Skip to content

Project

Your Ansible project needs more than just a playbook and an inventory. Before starting it is useful to ship a custom configuration. You should keep files to indicate necessary Python requirements and Ansible collections. The following topics also describe filename syntax and how to structure the project folder. Some basic YAML syntax is given which every developer should follow.

Version Control

Keep your playbooks and inventory file in git (or another version control system), and commit when you make changes to them. This way you have an audit trail describing when and why you changed the rules that are automating your infrastructure.

Tip

Always use version control!

Take a look at the Development section for additional information.

Directory structure

The most basic directory structure for a classic Ansible project should look like the following, if your project should not be used as a collection itself. 

---
config:
  treeView:
    rowIndent: 20
    defaultIconPack: vscode-icons
    showIcons: true
    filenameIcons:
      hosts: file-type-config
      requirements.txt: file-type-pypi
    extensionIcons:
      .cfg: file-type-config
      .j2: file-type-jinja
      .json: file-type-json
      .md: file-type-markdown
      .sh: file-type-shell
      .yml: file-type-light-yaml-official
---
treeView-beta
├── ansible.cfg ## # small configuration file, e.g. for setting the inventory or other callback plugins
├── hosts ## # inventory file
├── k8s_install.yml ## # main playbook file
├── README.md
├── requirements.txt ## # Python dependencies
├── requirements.yml ## # Collection dependencies
└── roles/ ## # roles directory containing ONLY folders with content
    ├── k8s_bootstrap/
    │   ├── files/
    │   │   ├── daemon.json
    │   │   └── k8s.conf
    │   ├── tasks/
    │   │   ├── install_kubeadm.yml
    │   │   ├── main.yml
    │   │   └── prerequisites.yml
    │   └── templates/
    │       └── kubernetes.repo.j2
    ├── k8s_control_plane/
    │   ├── files/
    │   │   └── kubeconfig.sh
    │   └── tasks/
    │       └── main.yml
    └── k8s_worker_nodes/
        └── tasks/
            └── main.yml

Filenames

Folder- and file-names consisting of multiple words are separated with underscores (e.g. roles/grafana_deployment/tasks/grafana_installation.yml).
YAML files are saved with the extension .yml.
Use descriptive names that are human-readable and do not shorten more than necessary. A pattern object[_feature]_action has proven useful as it guarantees a proper sorting in the file system for roles and playbooks. Ansible supports long identifier names, so use them!

---
config:
  treeView:
    rowIndent: 20
    defaultIconPack: vscode-icons
    showIcons: true
    extensionIcons:
      .conf: file-type-config
      .j2: file-type-jinja
      .json: file-type-json
      .md: file-type-markdown
      .sh: file-type-shell
      .yml: file-type-light-yaml-official
---
treeView-beta
├── ansible.cfg
├── hosts
├── k8s_install.yml
├── README.md
├── requirements.yml
└── roles/
    ├── k8s_bootstrap/
    │   ├── files/
    │   │   ├── daemon.json
    │   │   └── k8s.conf
    │   ├── tasks/
    │   │   ├── install_kubeadm.yml
    │   │   ├── main.yml
    │   │   └── prerequisites.yml
    │   └── templates/
    │       └── kubernetes.repo.j2
    ├── k8s_control_plane/
    │   ├── files/
    │   │   └── kubeconfig.sh
    │   └── tasks/
    │       └── main.yml
    └── k8s_worker_nodes/
        └── tasks
            └── main.yml

Playbook-name without underscores and wrong file extension, role folders or task files inconsistent, with underscores and wrong extension.

---
config:
  treeView:
    rowIndent: 20
    defaultIconPack: vscode-icons
    showIcons: true
    extensionIcons:
      .conf: file-type-config
      .j2: file-type-jinja
      .json: file-type-json
      .md: file-type-markdown
      .sh: file-type-shell
      .yml: file-type-light-yaml-official
---
treeView-beta
├── ansible.cfg
├── hosts
├── k8s-install.yaml
├── README.md
└── roles
    ├── k8s-bootstrap
    │   ├── files
    │   │   ├── daemon.json
    │   │   └── k8s.conf
    │   ├── tasks
    │   │   ├── installKubeadm.yaml
    │   │   ├── main.yml
    │   │   └── prerequisites.yaml
    │   └── templates
    │       └── kubernetes.repo.j2
    ├── k8sControlPlane
    │   ├── files
    │   │   └── kubeconfig.sh
    │   └── tasks
    │       └── main.yaml
    └── k8s_worker-nodes
        └── tasks
            └── main.yaml

YAML Syntax

Following a basic YAML coding style across the whole team improves readability and reusability.

Indentation

Two spaces are used to indent everything, e.g. list items or dictionary keys.

Playbook:

---
- name: Initialize Control-Plane Nodes
  hosts: kubemaster
  become: true
  roles:
    - k8s_control_plane

- name: Install and configure Worker Nodes
  hosts: kubeworker
  become: true
  roles:
    - k8s_worker_nodes

Variable-file:

ntp_server_list:
  - 0.de.pool.ntp.org
  - 1.de.pool.ntp.org
  - 2.de.pool.ntp.org
  - 3.de.pool.ntp.org

Playbook with roles not indented by two whitespaces.

- name: Demo play
  hosts: database_servers
  roles:
  - common
  - postgres

List in variable-file indented with four whitespaces:

ntp_server_list:
    - 0.de.pool.ntp.org
    - 1.de.pool.ntp.org
    - 2.de.pool.ntp.org
    - 3.de.pool.ntp.org

The so-called YAML "one-line" syntax is not used, neither for passing parameters in tasks, nor for lists or dictionaries.

---
- name: Install Apache from the testing repo
  ansible.builtin.package:
    name: httpd
    enablerepo: testing
    state: present

- name: Install a list of packages
  ansible.builtin.package:
    name:
      - nginx
      - postgresql
      - postgresql_server
    state: present

Task with One-line syntax:

- name: Install the latest version of Apache from the testing repo
  package: name=httpd enablerepo=testing state=present

List in task with One-line syntax:

- name: Install a list of packages
  package:
    name: ['nginx', 'postgresql', 'postgresql-server']
    state: present

Booleans

Use true and false for boolean values in playbooks.
Do not use the Ansible-specific yes and no as boolean values in YAML as these are completely custom extensions used by Ansible and are not part of the YAML spec. Also, avoid the use of the Python-style True and False for boolean values.

- name: Start and enable service httpd
  ansible.builtin.service:
    name: httpd
    enabled: true
    state: started

- name: Start and enable service httpd
  ansible.builtin.service:
    name: httpd
    enabled: yes
    state: started

YAML 1.1 allows all variants whereas YAML 1.2 allows only true/false, you can avoid a massive migration effort for when it becomes the default.

Use the | bool filter when using bare variables (expressions consisting of just one variable reference without any operator) in when conditions.

Using a variable upgrade_allowed with the default value false, task is executed when overwritten with true value.

- name: Upgrade all packages, excluding kernel & foo related packages # noqa package-latest
  ansible.builtin.package:
    name: "*"
    state: latest
    exclude: kernel*,foo*
  when: upgrade_allowed | bool

- name: Upgrade all packages, excluding kernel & foo related packages
  ansible.builtin.package:
    name: "*"
    state: latest
    exclude: kernel*,foo*
  when: upgrade_allowed

Quoting

Do not use quotes unless you have to, especially for short module-keyword-like strings like present, absent, etc.
When using quotes, use the same type of quotes throughout your playbooks. Always use double quotes ("), whenever possible.

Comments

Use loads of comments!
Well, the name parameter should describe your task in detail, but if your task uses multiple filters or regex's, comments should be used for further explanation.
Commented code is generally to be avoided. Playbooks or task files are not committed, if they contain commented out code.

Bad

Why is the second task commented?
Is it not necessary anymore?
Does it not work as expected?

- name: Change port to {{ grafana_port }}
    community.general.ini_file:
        path: /etc/grafana/grafana.ini
        section: server
        option: http_port
        value: "{{ grafana_port }}"
    become: true
    notify: restart grafana

# - name: Change theme to {{ grafana_theme }}
#   ansible.builtin.lineinfile:
#     path: /etc/grafana/grafana.ini
#     regexp: '.*default_theme ='
#     line: "default_theme = {{ grafana_theme }}"
#   become: yes
#   notify: restart grafana

Comment commented tasks

If you really have to comment the whole task, add a description why, when and by whom it was commented.

Ansible configuration

Always use a project-specific ansible.cfg in the parent directory of your project. The following configuration can be used as a starting point:

[defaults]
# Define inventory, no need to provide '-i' anymore.
inventory = inventory/production.ini

# Playbook-Output in YAML instead of JSON
callback_result_format = yaml

# New hosts in playbook run are scanned, but same hosts in subsequent play(s) not again
gathering = smart # (1)!

  1. Default setting is implicit which always gathers facts, even multiple times when host is addressed again in subsequent plays.
    Following settings are available:

    Setting Description
    implicit Facts will be gathered per play unless gather_facts: false is set. Cache plugins are ignored!
    explicit Facts will not be gathered unless directly requested in the play.
    smart Each new host that has no facts discovered will be scanned, but if the same host is addressed in multiple plays it will not be contacted again in the run.

    To retain Ansible facts for repeated use, select a different cache plugin. Take a look at the Fact caching section for details.

To validate your configuration file use the ansible-config utility:

ansible-config validate -t all # (1)!

  1. Without -t all only the base settings (-t base) are validated!

    In case of unknown/wrong settings keys, the output looks like this:

    [ERROR]: Found unknown key 'result_format' in section 'defaults' in '/home/timgrt/demo/ansible.cfg.
[ERROR]: Found unknown key 'custom_stats' in section 'defaults' in '/home/timgrt/demo/ansible.cfg.

Tip

The following shows a couple of useful configuration settings, for a full view of all base settings use the documentation.
To create a new config file with all available (but commented) settings, use the following command:

ansible-config init -t all --disabled > ansible.cfg

Show check mode

The following parameter enables displaying markers when running in check mode.

[defaults]
check_mode_markers = true

The markers are DRY RUN at the beginning and ending of playbook execution (when calling ansible-playbook --check) and CHECK MODE as a suffix at every play and task that is run in check mode.

Example output 
$ ansible-playbook -i inventory.ini playbook.yml -C

DRY RUN ******************************************************************

PLAY [Install and configure Worker Nodes] [CHECK MODE] *******************

TASK [Gathering Facts] [CHECK MODE] **************************************
ok: [k8s-worker1]
ok: [k8s-worker2]
ok: [k8s-worker2]

...

Show task path when failed

For easier development when handling with very big playbooks, it may be useful to know which file holds the failed task. To display the path to the file containing the failed task and the line number, add this parameter:

[defaults]
show_task_path_on_failure = true
Example output

When set to true:

TASK [Set motd message for k8s worker node] **************************************************
task path: /home/timgrt/kubernetes_installation/roles/kube_worker/tasks/configure.yml:39
fatal: [k8s-worker1]: FAILED! =>
...

When set to false:

TASK [Set motd message for k8s worker node] ****************************************************
fatal: [k8s-worker1]: FAILED! =>
...

Even if you don't set this, the path is displayed automatically for every task when running with -vv or greater verbosity, but you'll need to run the playbook again.

Silence warnings about no inventory or localhost

Ansible will issue a warning when no inventory was loaded and notes that it will use an implicit localhost-only inventory. Additionally, when running against localhost only (for example, when automating against API endpoints), you'll receive a warning as well.

$ ansible-playbook aci_automation.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Automate Cisco ACI] *******************************************************************************************

TASK [aci-automation : Create tenant] *******************************************************************************
changed: [localhost]

...

You can silence both warnings with the following configuration:

[defaults]
localhost_warning = false # (1)!

[inventory]
inventory_unparsed_warning = false # (2)!
  1. This will silence [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
  2. This will silence [WARNING]: No inventory was parsed, only implicit localhost is available

Do not show skipped hosts

When running against a huge number of host with tasks executed on select hosts only and most hosts are skipped because of certain conditions, it can be useful to not display skipped hosts to decrease visual clutter.

[defaults]
display_skipped_hosts = false
Example output

The Play Recap will always show the number of skipped tasks for every host, regardless of the configuration set.

Only showing hosts where tasks are executed

$ ansible-playbook kafka_configuration.yml

PLAY [Kafka configuration] ******************************************************************************************

...

TASK [Initial creation of Kafka topics] *****************************************************************************
ok: [kafka-node1]

TASK [Ensure logical-to-physical topic mapping is correct] **********************************************************
ok: [kafka-node1]

...

PLAY RECAP **********************************************************************************************************
kafka-node1                : ok=7    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
kafka-node2                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0
kafka-node3                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0
kafka-node4                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0
kafka-node5                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

Showing all hosts, most of them skipped

$ ansible-playbook kafka_configuration.yml

...

TASK [Initial creation of Kafka topics] *****************************************************************************
skipping: [kafka-node2]
skipping: [kafka-node3]
skipping: [kafka-node4]
skipping: [kafka-node5]
ok: [kafka-node1]

TASK [Ensure logical-to-physical topic mapping is correct] **********************************************************
skipping: [kafka-node2]
skipping: [kafka-node3]
ok: [kafka-node1]
skipping: [kafka-node4]
skipping: [kafka-node5]

...

PLAY RECAP **********************************************************************************************************
kafka-node1                : ok=7    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
kafka-node2                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0
kafka-node3                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0
kafka-node4                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0
kafka-node5                : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

Fact caching

Facts are stored in memory by default. To retain Ansible facts for repeated use, select a different cache plugin.
Not only does fact caching improve performance and playbook runtime, you have access to variables and information about all hosts even when you are only managing a small number of servers.

The builtin jsonfile cache plugin uses JSON formatted files, one for each host, saved to the filesystem.

Info

Useful for local, per-user fact caching. Does get cluttered/hard to navigate when targeting a huge number of hosts.

ansible.cfg
[defaults]
fact_caching = ansible.builtin.jsonfile
fact_caching_connection = ~/.ansible/facts # (1)!
fact_caching_timeout = 3600 # (2)!
fact_caching_prefix = ansible_facts_
  1. Saves all fact cache files in a sub-folder of the default Ansible cache/tmp folder.
  2. Expiration timeout of one hour.
Example 
$ grep distribution  ~/.ansible/facts/ansible_facts_instance1
"ansible_distribution": "RedHat",
"ansible_distribution_file_parsed": true,
"ansible_distribution_file_path": "/etc/redhat-release",
"ansible_distribution_file_search_string": "Red Hat",
"ansible_distribution_file_variety": "RedHat",
"ansible_distribution_major_version": "9",
"ansible_distribution_release": "Plow",
"ansible_distribution_version": "9.8",

Facts are available, even when not gathered directly as with the debug module used in an ad-hoc command:

$ ansible instance2 -m debug -a var=ansible_facts.distribution
instance2 | SUCCESS => {
    "ansible_facts.distribution": "RedHat"
}

JSON file caching stores facts on the local filesystem. This has some limitations:

  • Each control node has its own cache, so running a playbook from a different controller does not benefit from gathered facts
  • File I/O can become a bottleneck with thousands of hosts

Redis can be used as the fact caching backend, it is an in-memory key–value database, used as a distributed cache and message broker.

Info

Redis is a network service, multiple users and even Ansible control nodes can share the same cache.

ansible.cfg
[defaults]
fact_caching = community.general.redis # (1)!
fact_caching_connection = localhost:6379:0 # (2)!
fact_caching_timeout = 3600 # (3)!
fact_caching_prefix = ansible_facts_

  1. Install the necessary collection:

    ansible-galaxy collection install community.general

  2. Here, Redis is running locally on Port 6379, facts are stored in DB 0. The connection string format is host:port:db:password.
    For local development, start a container with Redis:

    podman run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

    The redis/redis-stack image also provides a user interface (on port 8001) which makes viewing and searching facts and fact values easy.

  3. Expiration timeout of one hour.

Example 
$ redis-cli -h localhost -p 6379 get "ansible_facts_instance1" | python3 -m json.tool | grep distribution
"ansible_distribution": "RedHat",
"ansible_distribution_file_parsed": true,
"ansible_distribution_file_path": "/etc/redhat-release",
"ansible_distribution_file_search_string": "Red Hat",
"ansible_distribution_file_variety": "RedHat",
"ansible_distribution_major_version": "9",
"ansible_distribution_release": "Plow",
"ansible_distribution_version": "9.8",

Configure ansible-galaxy

By default, ansible-galaxy uses https://galaxy.ansible.com as the Galaxy server. To configure additional servers like Automation Hub or Private Automation Hub, you'll need to configure these in the galaxy section.

Create a new section for each server name, set the url option for each server name and set the API token for each server name, if necessary.

[galaxy]
server_list = release_galaxy,automation_hub # (1)!

[galaxy_server.release_galaxy]
url = https://galaxy.ansible.com/ # (2)!

[galaxy_server.automation_hub]
url = https://console.redhat.com/api/automation-hub/content/published/ # (3)!
auth_url = https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token # (4)!
token = <secret-token>
  1. Comma-separated list of server identifiers in a prioritized order (in the example galaxy.ansible.com is searched first, then Automation Hub).
    The name of the server does not matter, but needs to match the corresponding galaxy_server.* section.
  2. galaxy.ansible.com doesn't need authentication to download collections (or roles).
    If you want to publish stuff, you'll need an API Token (or username and password).
  3. Gets content from the published repository, to get stuff from the validated repository, adjust or add a new section and add it to the server_list.
  4. The URL of a Keycloak server token_endpoint if using SSO authentication. Requires token.

Danger

Take extra care when using tokens (or passwords) in your ansible.cfg, as they might get added to Git by accident.
Add the file to your .gitignore.

Take a look at the Ansible documentation for additional information.

Dependencies

Your project will have certain dependencies, make sure to provide a requirements.yml for necessary Ansible collections and a requirements.txt for necessary Python packages.
Consider using Execution Environments where all dependencies are combined in a Container Image.

Collections

Always provide a requirements.yml with all collections used within your project.
This makes sure that required collections can be installed, if only the ansible-core binary is installed.

---
collections:
  - community.general
  - ansible.posix
  - cisco.ios
Add the collection source URL in case galaxy.ansible.com is down 
collections:
    - name: ansible.posix
      # source: https://github.com/ansible-collections/ansible.posix
      # type: git
    - name: containers.podman
      version: 1.20.2
      # source: https://github.com/containers/ansible-podman-collections
      # type: git

If Galaxy is unavailable, uncomment the source and type lines to install the collection directly from GitHub.

Install all collections from the requirements-file:

ansible-galaxy collection install -r requirements.yml

If you want to install Collections not available on Galaxy or Automation Hub, use the following configuration:

collections:
  - name: https://github.com/TimGrt/Ansible-Bootstrap.git # (1)!
    type: git
    version: main # (2)!
  1. The Git-URL to the Collection content, ensure that you can pull/clone from this address without authentication. The collection name will be deduced from the galaxy.yml of the collection (the content is cloned to a temporary location and then installed as if done with the ansible-galaxy utility).
  2. The branch to use, if you want to test with a feature branch, adjust this.

Take a look at the Ansible documentation for all available options in the requirements.yml file.

Python packages

Always provide a requirements.txt with all Python packages need by modules used within your project.

boto
openshift>=0.6
PyYAML>=3.11

Install all dependencies from the requirements-file:

pip3 install -r requirements.txt