Merge pull request #2 from ibm-messaging/enhancements-r1

Playbook and documentation enhancements.

README.md

@@ -3,7 +3,7 @@
 | :memo:        | Interested in contributing to this project? Please read our [IBM Contributor License Agreement](CLA.md) and our [Contributing Guide](CONTRIBUTING.md).       |
 |---------------|:------------------------|
 
-A collection for automating the installation and configuration of IBM MQ using Ansible on Ubuntu machines. Our aim is to make MQ-Ansible extensible for further and more detailed IBM MQ configuration.
+A collection for automating the installation and configuration of IBM MQ using Ansible on Ubuntu machines. Our aim is to make MQ-Ansible extensible for other platforms and more detailed IBM MQ configuration.
 
 This directory contains:
 - ansible [`roles`](https://github.com/ibm-messaging/mq-ansible/tree/main/ansible_collections/ibm/ibmmq/roles) for the installation and configuration of IBM MQ.
@@ -14,46 +14,81 @@ For a detailed explanation and documentation on how MQ-Ansible works, click [her
 
 ## Requirements
 
-- `ansible` and `ansible-lint` are required on your local machine to run playbooks implementing this collection.
+- `ansible`, `passlib` and `ansible-lint` are required on your local machine to run playbooks implementing this collection.
 - An Ubuntu target machine is required to run MQ.
 
-## Roles for IBM MQ installation
+ ##### *Ansible* installation ([Installation guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html))
 
-The roles in this collection carry out an installation of IBM MQ Advanced on an Ubuntu target machine with ansible roles as yaml files. The roles have been implemented to set up the required users on the machine, download the software, install and configure IBM MQ, copy over a configurable `dev-config.mqsc` file ready to be run on the target machine, and set and start the web console. Developers can change this file to allow better configuration of their queue managers.
+## Playbooks and Roles for IBM MQ installation
 
+The playbooks and roles in this collection carryout an installation of IBM MQ Advanced on an Ubuntu target machine. The roles have been implemented to set up the required users on the machine, download the software, install and configure IBM MQ, copy over a configurable `dev-config.mqsc` file ready to be run on the target machine, and setup and start the web console. Developers can change this file to customise the configuration of their queue managers. Here we use a playbook that calls other playbooks but you can run the roles in playbooks to suit your requirements.
 
-### Example
+### Example Playbooks
+
+ibmmq.yml - this playbook calls the mq-install and mq-setup playbooks, host names are passed into the imported playbook variable as {{ ansible_play_batch }}
+
+```yaml
+- name: Install and setup IBM MQ
+  hosts: ['servers']
+
+- name: Run the install playbook
+  import_playbook: mq-install.yml
+
+- name: Run the setup playbook
+  import_playbook: mq-setup.yml
+```
+
+mq-install.yml - this playbook installs IBM MQ with the SSH user specified in the inventory
 
 ```yaml
-- hosts: [YOUR_TARGET_MACHINES]
+- hosts: "{{ ansible_play_batch }}"
+  serial: 1
   become: false
   environment:
     PATH: /opt/mqm/bin:{{ ansible_env.PATH }}
 
-  roles: 
+  roles:
     - role: setupusers
       vars:
-        gid: 989
+        gid: 909
     - downloadmq
     - installmq
+```
+mq-setup.yml - this playbook sets up IBM MQ using the 'mqm' user
+
+```yaml
+- hosts: "{{ ansible_play_hosts }}"
+  serial: 1
+  become: yes
+  become_user: mqm
+  environment:
+    PATH: /opt/mqm/bin:{{ ansible_env.PATH }}
+
+  roles:
     - getconfig
     - setupconsole
     - startconsole
-```
 
+  tasks:
+
+    - name: Create a queue manager
+      queue_manager:
+        qmname:
+        - 'QM1'
+        - 'QM2'
+        state: 'present'
+```
 ## Modules for IBM MQ resources' configuration
 
 - `queue_manager.py`- Creates, starts, deletes an IBM MQ queue manager and runs an MQSC file. See the documentation [here.](QUEUE_MANAGER.md)
 
 # Run our sample playbook
 
-### Setup (inventory.ini)
+##### *Note*: *Ansible* must be installed on the local machine ([Installation guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html))
 
-##### Note: *Ansible* must be installed on the local machine. ([Installation guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html))
+Before running the playbook and implementing our modules and roles for IBM MQ:
 
-Before running the playbook implementing our modules and roles for IBM MQ:
-
-1. Check if you have an *ssh* key pair in order to access the target machines via Ansible. Go to the `~/.ssh` directory in your machine and look for the `id_rsa` and `id_rsa.pub` files.
+1. Check if you have an *ssh* key pair in order to access the target machines via SSH. Go to the `~/.ssh` directory in your machine and look for the public and private key files e.g. `id_rsa` and `id_rsa.pub`.
 
     ```shell
      cd ~/.ssh
@@ -65,68 +100,70 @@ Before running the playbook implementing our modules and roles for IBM MQ:
      ssh-keygen
     ```
 
-3. Once the keys have been generated, these need to be copied to the target machine's user `ssh` directory. 
+3. Once the keys have been generated, you need to copy the public key to the target machine's user `ssh` directory.
 
     ```shell
-     ssh-copy-id -i id_rsa.pub [USER]@[YOUR_TARGET_MACHINE_IP]
+     ssh-copy-id -i id_rsa.pub [USER]@[YOUR_TARGET_HOST]
     ```
 
 4. To confirm the keys have been copied successfully, connect to your target machine by:
 
     ```shell
-     ssh [USER]@[YOUR_TARGET_MACHINE_IP]
+     ssh [USER]@[YOUR_TARGET_HOST]
     ```
     This should connect to your target machine without asking for a password.
 
-5. Go to the `ansible_collections/ibm/ibmmq/` directory.
+5. On your local machine clone this repository. 
+
+6. Go to the `ansible_collections/ibm/ibmmq/` directory.
 
     ```shell
-     cd ..
-     cd ansible_collections/ibm/ibmmq/
+     cd mq-ansible/ansible_collections/ibm/ibmmq/
     ```
 
 
-6. Create a file `inventory.ini` inside the directory with the following content:
+7. Create a file `inventory.ini` inside the directory with the following content:
 
     ```ini
-    [YOUR_TARGET_MACHINES]
-    [YOUR_MACHINE_IP] ansible_ssh_user=[YOUR_USER]
-    ```
+    
+    [servers]
+    YOUR_HOST_ALIAS ansible_host=YOUR_HOSTNAME ansible_ssh_user=YOUR_SSH_USER
+    YOUR_HOST_ALIAS ansible_host=YOUR_HOSTNAME ansible_ssh_user=YOUR_SSH_USER
 
-   - Change `YOUR_TARGET_MACHINES` to your machines' group name, for example `fyre`.
-   - Change `YOUR_MACHINE_IP` to your target machine's public IP
-   - Change `YOUR_USER` to your target machine's user.
-   ##### *NOTE* : user on the target machine MUST NOT be root but MUST have `sudo` privileges.
+    ```
+   ##### *Note*: you can specify one or more hosts
+   - Change `YOUR_HOST_ALIAS` to an alias name that you wish to use e.g. `mq-host-1` , you can omit aliases if you prefer
+   - Change `YOUR_HOSTNAME` to your server/hostname, e.g. `myserver-1.fyre.com`
+   - Change `YOUR_SSH_USER` to your target machine's SSH user
+   ##### *Note*: the user on the target machine MUST have `root` or `sudo` privileges
 
 ### ibmmq.yml
 
 The sample playbook [`ibmmq.yml`](ansible_collections/ibm/ibmmq/ibmmq.yml) installs IBM MQ Advanced with our roles and configures a queue manager with the `queue_manager.py` module.
 
-1. Before running the playbook, ensure that you have added the directory path to the PATH environment variable.
+1. Before running the playbook, ensure that you have added the following directory path to the ANSIBLE_LIBRARY environment variable.
 
-    ##### *NOTE* : change `<PATH-TO>` to your local directory path:
+    ##### *Note*: change `<PATH-TO>` to your local directory path:
 
     - On Mac:
 
-          ```shell
-           export ANSIBLE_LIBRARY=<PATH-TO>/ansible_mq/ansible_collections/ibm/ibmmq/library
-          ```
-
-    - On Windows: 
+       ```shell
+          export ANSIBLE_LIBRARY=${ANSIBLE_LIBRARY}:<PATH-TO>/ansible_mq/ansible_collections/ibm/ibmmq/library
+       ```
 
-          ```shell
-           set ANSIBLE_LIBRARY=<PATH-TO>/ansible_mq/ansible_collections/ibm/ibmmq/library
-          ```
-
-2. Make sure you update the hosts in `ibmmq.yml` name to `YOUR_TARGET_MACHINES` group from your inventory file.
+    - On Windows:
+
+      ```shell
+          set ANSIBLE_LIBRARY=%ANSIBLE_LIBRARY%;<PATH-TO>/ansible_mq/ansible_collections/ibm/ibmmq/library
+       ```
 
-3. Run the following command to execute the tasks within the playbook:
+2. Run the following command to execute the tasks within the playbook:
       ```shell
-       ansible-playbook ./ibmmq.yml -i inventory.ini -K
+       ansible-playbook ./ibmmq.yml -i inventory.ini
       ```
-      - ##### *NOTE* : `-K` will prompt the user to enter the sudo password for [YOUR_USER] on the target machine.
+      - ##### *Note*: you can optionally add `-K` (uppercase) to the command, this will prompt the user to enter the sudo password for [YOUR_SSH_USER] on the target machine, you can omit if you have setup SSH keys
 
-4. The playbook should return the result of `dspmq` with the queue manager created listed. Log into your target machine and check it manually:
+3. The playbook should return the result of `dspmq` with the queue manager created listed. Log into your target machine and check it manually:
 
     ```shell
      dspmq
@@ -139,9 +176,9 @@ If one of the following errors appears during the run of the playbook, run the f
 - `Please add this host's fingerprint to your known_hosts file to manage this host.` - Indicates that an SSH password cannot be used instead of a key. 
 
   Fix:
-    ##### *NOTE* : change `[YOUR_MACHINE_IP]` to the target machine's public IP address
+  ##### *Note*: change `[YOUR_HOST]` to the target machine's network address
   ```shell
-  ssh-keyscan -H [YOUR_MACHINE_IP] >> ~/.ssh/known_hosts
+  ssh-keyscan -H [YOUR_HOST] >> ~/.ssh/known_hosts
   ```
 - `zsh: command not found: dspmq` - Appears that MQ environment variables have not been set.
 
@@ -170,10 +207,10 @@ To run the test playbooks first:
     ```
 2. export the modules to your Ansible library
     ```shell
-     export ANSIBLE_LIBRARY=<PATH-TO>/ansible_mq/ansible_collections/ibm/ibmmq/library
+     export ANSIBLE_LIBRARY=${ANSIBLE_LIBRARY}:<PATH-TO>/ansible_mq/ansible_collections/ibm/ibmmq/library
     ```
-   - ##### *NOTE* : change `<PATH-TO>` to your local directory path:
-3. run all test playbooks with `main.py`
+   - ##### *Note*: change `<PATH-TO>` to your local directory path:
+3. run all test playbooks with `python3 main.py`
 
 ## License
 

ansible_collections/ibm/ibmmq/ibmmq.yml

@@ -1,38 +1,8 @@
-- hosts: [YOUR_TARGET_MACHINES]
-  become: false
-  environment:
-    PATH: /opt/mqm/bin:{{ ansible_env.PATH }}
+- name: Install and setup IBM MQ
+  hosts: ['servers']
 
-  roles: 
-    - role: setupusers
-      vars:
-        gid: 999
-    - downloadmq
-    - installmq
-    - getconfig
-    - setupconsole
-    - startconsole
-
-  tasks:
+- name: Run the install playbook
+  import_playbook: mq-install.yml
 
-    - name: Create a queue manager
-      queue_manager:
-        qmname: 
-        - 'QM1'
-        - 'QM2'
-        state: 'present'
-
-    - name: Start a queue manager
-      queue_manager:
-        qmname:
-        - 'QM1'
-        - 'QM2'
-        state: 'running'
-
-    - name: Run MQSC File
-      queue_manager:
-        qmname:
-        - 'QM1'
-        - 'QM2'
-        state: 'running'
-        mqsc_file: '/home/{{ ansible_ssh_user }}/dev-config.mqsc'
+- name: Run the setup playbook
+  import_playbook: mq-setup.yml

ansible_collections/ibm/ibmmq/mq-install.yml

@@ -0,0 +1,13 @@
+- hosts: "{{ ansible_play_batch }}" 
+  serial: 1
+  become: false
+  environment:
+    PATH: /opt/mqm/bin:{{ ansible_env.PATH }}
+
+  roles: 
+    - role: setupusers
+      vars:
+        appUid: 909
+        appGid: 909
+    - downloadmq
+    - installmq

ansible_collections/ibm/ibmmq/mq-setup.yml

@@ -0,0 +1,35 @@
+- hosts: "{{ ansible_play_hosts }}"
+  serial: 1
+  become: yes
+  become_user: mqm 
+  environment:
+    PATH: /opt/mqm/bin:{{ ansible_env.PATH }}
+
+  roles: 
+    - getconfig
+    - setupconsole
+    - startconsole
+
+  tasks:
+
+    - name: Create a queue manager
+      queue_manager:
+        qmname: 
+        - 'QM1'
+        - 'QM2'
+        state: 'present'
+
+    - name: Start a queue manager
+      queue_manager:
+        qmname:
+        - 'QM1'
+        - 'QM2'
+        state: 'running'
+
+    - name: Run MQSC File
+      queue_manager:
+        qmname:
+        - 'QM1'
+        - 'QM2'
+        state: 'running'
+        mqsc_file: '/var/mqm/dev-config.mqsc'

ansible_collections/ibm/ibmmq/roles/downloadmq/tasks/main.yml

@@ -2,12 +2,14 @@
 
 - name: Download MQ Advanced for Developers
   get_url:
-    url: https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev920_ubuntu_x86-64.tar.gz
+    url: https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev931_ubuntu_x86-64.tar.gz
     dest: /tmp/mq.tar.gz
     force: no
+  tags: download
 
 - name: Extract MQ fom TAR
   unarchive:
     src: /tmp/mq.tar.gz
     remote_src: yes
-    dest: /tmp
+    dest: /tmp
+  tags: download

ansible_collections/ibm/ibmmq/roles/getconfig/tasks/main.yml

@@ -3,4 +3,4 @@
 - name: Copy developer config file to target
   copy:
     src: ../../../dev-config.mqsc
-    dest: "/home/{{ ansible_ssh_user }}"
+    dest: "/var/mqm"

ansible_collections/ibm/ibmmq/roles/installmq/tasks/main.yml

@@ -44,17 +44,6 @@
     apt-get update
   changed_when: 'installed_mq_packages.stdout_lines | string is not search("ibmmq")'
 
-- name: Add the user to group mqm
-  become: true
-  shell: adduser ${SUDO_USER:-${USER}} mqm
-
-- name: Add the user to group mqm
-  become: true
-  user:
-    name: "{{ ansible_ssh_user }}"
-    groups: mqm
-    append: yes
-
 - name: reset ssh connection
   meta: reset_connection