Ansible - How to create a recipe

Posted on March 12, 2018, 8:15 am by about-dev.com


Recently, I've started to play with Ansible and I've discoverd some awesome things which can be used to deploy, configure or manage remote machines.

I've struggled some time to discover all the things I needed and that's the reason why I want to share all the things I've learned.

Below I try to expose the steps that I've walked to create a recipe with Ansible, not before a few words about Ansible.

 

What is Ansible?

Ansible is a configuration, management, deployment and orchestration tool. By default it manages machines over the SSH protocol (both remote or local; by default manages remote machines). 

From the documentation website: "Once Ansible is installed, it will not add a database, and there will be no daemons to start or keep running. You only need to install it on one machine (which could easily be a laptop) and it can manage an entire fleet of remote machines from that central point. When Ansible manages remote machines, it does not leave software installed or running on them, so there’s no real question about how to upgrade Ansible when moving to a new version."
 
Ansible is similar to Puppet and Chef but much faster and easier to learn and manipulate. Here is a comparison.

 

What problem do I want to solve?

I've had problems with Gearman servers changing all the time. During a release or after a release when I needed to restart some workers I've had the surprise to discover these changes.

With Ansible I've created one command which does all the things I needed (without the need to connect to any server, just using my local machine - a Ubuntu machine):

  • gets the list of servers from the deploy machine
  • connects to each of those servers (by default Ansible runs in parallel processes)
  • runs commands on all servers (for my case restart|start|stop a worker)

 

Ansible terms

Inventory

  • a target host or a collection of target hosts against which we execute our commands
  • these may be grouped
  • default path is: /etc/ansible/hosts
  • all ansible or ansible-playbook commands must contain an inventory file (to skip inventory you may use ",")
  • to specify the inventory file use the "-i" option
  • more info

 
Task

  • a call to an ansible module (a command to be executed)

 
Role

  • a task or a collection of tasks acting as a reusable unit of work
  • can be used to easily apply common configurations in different scenarios
  • ansible will look for the main.yml task file to execute (if you see in a playbook that you want to execute the gearman role, ansible will find in the /roles/tasks/ path the main.yml file)
  • more info

 
Playbook(s)

  • Playbooks are Ansible’s configuration, deployment, and orchestration language
  • map a group of hosts to some well defined roles or tasks (if we dont't want to group tasks into roles)
  • are specified in YML
  • describe a policy you want your remote systems to enforce, or a set of steps 
  • in a playbook you specify the target host or group and the commands / steps you want to execute on this (the commands are grouped in roles)
  • more info

 
INFO: Ansible works with INI or YML files. All ansible configuration may be controlled from the default /etc/ansible/ansible.cfg file or a custom /home/my_path/ansible.cfg file if you create (and I recommend you to do so) a custom folder structure for your recipes.


Installation

Because the Ansible tutorial is very explanatory in this topic I won't insist very much on this. I just want to tell you that I used a local machine with Ubuntu installed.

  • Install Ubuntu on a Windows machine (I've installed a Ubuntu version for Windows from here).
  • Install ansible using the tutorial from here

Ansible commands

>>ansible <host-pattern> [options]

  • used to run ad-hoc commands (something that you want to run really quick, but don’t want to save for later)
  • reads the host from the hosts file (default path: /etc/ansible/hosts; custom path: /home/MY_PATH/hosts)
  • use "ansible -h" for more information


Example: restart a worker from my local gearman 

// run ansible ad-hoc command to restart a worker
user@104IT56:/etc/ansible$ ansible my-dev -m shell -a "supervisorctl -s http://127.0.0.1:8989 -u an_user -p parola restart update_job:*"
 
// CLI response
my.network | SUCCESS | rc=0 >>
update_job:update_job_0: started

Explanation: 
<host-pattern> = my-dev

  • this information is gathered from the /etc/ansible/hosts file and represents a group from that file

[options] = -m shell -a "supervisorctl -s http://127.0.0.1:8989 -u admin -p parola restart update_job:*"

  • -m NAME
    • this is the call to an Ansible module
    • use -m to the Ansible that you want to use a module
    • NAME is the module name, in our case "shell"
    • more info
  • -a COMMAND
    • this is the command we want to execute using the shell module
    • more info

[options] playbook.yml [playbook2 ...]

  • used to run complex commands (recipes)
  • every recipe contains:
  • a target group
  • a role to apply to the target group or host

Example: restart a worker from the production gearman servers
The folder structure for this command is:

  • a folder for inventories
    • at this level you'll have to put your group vars (variables which you want accesible for all hosts; in my case credentials)
  • a folder for playbooks: my collection of tasks grouped per scope (now I have only a gearman tasks group)
  • a folder for roles
    • when we use a role ansible will look for a tasks folder and a main.yml file

Ansible project folder structure

// run a defined ansible gearman playbook with a gearman inventory file using the ansible-playbook command
user@104IT56:/home/user/ansible_jobs/$ ansible-playbook -i inventories/gearman.yml -e "worker=update_job" -e "worker_action=restart" playbooks/gearman.yml
 
// CLI response
PLAY [gearman] ******************************************************************************************************************************************************************************************************************************
TASK [Restarting worker] ********************************************************************************************************************************************************************************************************************
changed: [gearman1-prod.network]
changed: [gearman2-prod.network]
changed: [gearman3-prod.network]
changed: [gearman4-prod.network]
changed: [gearman5-prod.network]
changed: [gearman6-prod.network]
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
gearman1-prod.network : ok=1    changed=1    unreachable=0    failed=0
gearman2-prod.network : ok=1    changed=1    unreachable=0    failed=0
gearman3-prod.network : ok=1    changed=1    unreachable=0    failed=0
gearman4-prod.network : ok=1    changed=1    unreachable=0    failed=0
gearman5-prod.network : ok=1    changed=1    unreachable=0    failed=0
gearman6-prod.network : ok=1    changed=1    unreachable=0    failed=0

Explanation:
[options]

  • -i INVENTORY_FILE to use (in this case the file with all the Gearman servers)

  • -e "NAME=VALUE": external variabile to be used

playbook.yml

  • playbooks/gearman.yml: the playbook I want to execute to get the job done (the job: iterate over each of the gearman servers and restart a worker)

Gearman playbook example without a role:

---
- hosts: gearman
  gather_facts: no  #i dont't want ansible to scan the remote host for setup info
  tasks:
    - name: "Restarting worker"
      command: supervisorctl -s "http://127.0.0.1:8989" -u admin -p parola "{{ worker_action }}" "{{ worker }}":*
 

Gearman playbook example with a role:

---
- hosts: gearman
  gather_facts: no
  roles:
    - ../roles/gearman

Gearman playbook example with an imported role:

---
- hosts: gearman
  gather_facts: no
  tasks:
    - import_role:
        name: ../roles/gearman

Or:

---
- hosts: gearman
  gather_facts: no
  tasks:
    - name: "Import gearman role"
      import_role:
        name: ../roles/gearman

My recipe

As I told you early I have to work with the latest servers versions that's why I needed a method to run one command and that command to apply on these servers all at once. This is where Ansible came into play and helped me to resolve all these steps.
The command:

user@104IT56:/home/user/ansible_jobs/$ ansible-playbook -i inventories/deploy.yml -e "service_type=gearman" -e "worker=update_job" -e "worker_action=restart" playbooks/manage_supervisor.yml
 
// CLI response
PLAY [deploy] **************************************************************************************************************************************************************************************************************************
TASK [/home/user/ansible_jobs/roles/gearman : Run get_gearman_servers] *****************************************************************************************************************************************************************
 [WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo
changed: [deploy.network.ro]
TASK [/home/user/ansible_jobs/roles/gearman : Prepare result to forward] ***************************************************************************************************************************************************************
changed: [deploy.network.ro]
TASK [/home/user/ansible_jobs/roles/gearman : Set servers list into facts] *************************************************************************************************************************************************************
ok: [deploy.network.ro]
TASK [/home/user/ansible_jobs/roles/gearman : Create an inventory template locally for gearman servers] ********************************************************************************************************************************
changed: [deploy.network.ro -> localhost]
TASK [/home/user/ansible_jobs/roles/gearman : Rename the inventory] ********************************************************************************************************************************************************************
changed: [deploy.network.ro -> localhost]
TASK [/home/user/ansible_jobs/roles/gearman : Run supervisor for sql worker and restart action] **********************************************************************************************************************
changed: [deploy.network.ro -> localhost]
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
deploy.network.ro : ok=6    changed=5    unreachable=0    failed=0

The steps:
#1: define project structure

Ansible recipe folder structure

#2: define the deploy inventory file (this is the place from which I can get the latest servers versions)
/inventories/deploy.yml
 
#3: define the playbook which will take care of my wishes
/playbooks/manage_supervisor.yml
 
#4: define the role for this job (I organised my Ansible code into roles for easy management)
/roles/gearman
 
#5: define the tasks for this gearman role

  • get servers list from the deploy machine
  • create locally (on the control machine) the gearman inventory file that I'll need for gearman tasks
    • by default Ansible works with remote machine
    • for situations where we need to work on the control machine we have to explicitly tell him this using the local_action command

Example:

---
- name: "Create an inventory template locally for gearman servers"
  local_action:
    module: template
    src: ../../gearman/templates/gearman_inventory.yml
    dest: ../../../inventories
- name: "Rename the inventory"
  local_action: command mv ../../../inventories/gearman_inventory.yml ../../../inventories/gearman.yml

Here I used a Jinja2 template file because I needed to generate a inventory file in the right format. Jinja2 is the templating framework used by Ansible. More info here.
 
In this template a used some information saved in facts. Facts are the place where Ansible gathers information about the remote machines and we can use this for custom variables needed all over the project. The facts are discovered automatically by the setup module whenever we use a ansible command.

  • run the restart|start|stop worker action on all the gearman servers for the desired country

 
#6: that's all!
 
You can download the source code from here.

Any questions or ideas of improvements are welcomed.

 


Leave a Comment:

User
Email
Website

Blog Search

Popular Blog Categories

Newsletter

Want to be informed about latest posts? Subscribe to our newsletter