-
Notifications
You must be signed in to change notification settings - Fork 69
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
bitcoinz
committed
Sep 20, 2017
1 parent
5ad4b26
commit 2a440ef
Showing
124 changed files
with
1,507 additions
and
776 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -112,3 +112,6 @@ libzcashconsensus.pc | |
|
|
||
| contrib/debian/files | ||
| contrib/debian/substvars | ||
|
|
||
| .idea/ | ||
| *.iml | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,8 +28,7 @@ open-source licenses. For further details see 'contrib/debian/copyright'. | |
|
|
||
| This product includes software developed by the OpenSSL Project for use in the | ||
| OpenSSL Toolkit (https://www.openssl.org/). This product includes cryptographic | ||
| software written by Eric Young ([email protected]), | ||
| and UPnP software written by Thomas Bernard. | ||
| software written by Eric Young ([email protected]). | ||
|
|
||
|
|
||
| Although almost all of the Zcash code is licensed under "permissive" open source | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # Zcash CI workers | ||
|
|
||
| This folder contains the Ansible playbooks for configuring a fresh OS | ||
| installation for use as a Buildbot worker in Zcash's CI. | ||
|
|
||
| # Criteria for Adding Workers | ||
|
|
||
| a. Don't add workers until users complain about a problem on a platform | ||
| that doesn't yet have workers or if we anticipate many users will use | ||
| a platform, we may pre-emptively add an unsupported worker for it. | ||
|
|
||
| b. Prioritize the platforms that seem to have the most users. | ||
|
|
||
| c. When adding workers start by adding workers for the "most common" | ||
| variant of any distro, then if users later encounter problems with a | ||
| sub-variant, we can consider adding new workers at that point. | ||
| Example: add Ubuntu Desktop before Xubuntu, on the assumption the | ||
| former has a larger population base. | ||
|
|
||
| # Setting up a latent worker on Amazon EC2 | ||
|
|
||
| - Add a regular (non-latent) worker to the master.cfg for dev-ci.z.cash, and | ||
| deploy the changes. | ||
| - This enables the Ansible playbook to run to completion, ending in the worker | ||
| connecting to the master. | ||
|
|
||
| - Start a basic EC2 instance using the template AMI for the target OS. | ||
| - Choose the smallest instance size, it won't be used for building Zcash. | ||
|
|
||
| - Figure out which user to log into the instance with. | ||
| - E.g. for the Ubuntu template, use "ubuntu" instead of "root" | ||
| - If you get an Ansible error later with a message like "Failed to connect to | ||
| the host via ssh: Received message too long 1349281121\r\n", that means the | ||
| instance is sending a text string in response to the SSH connection, and the | ||
| Ansible protocol is balking. Try manually logging in with the same | ||
| credentials to diagnose. | ||
|
|
||
| - Create `inventory/hosts` containing the following: | ||
|
|
||
| [zcash-ci-worker-unix] | ||
| some-name ansible_host=<INSTANCE_IP> ansible_ssh_user=<USERNAME> | ||
|
|
||
| - Run `ansible-playbook -e buildbot_worker_host_template=templates/host.ec2.j2 -i inventory/hosts unix.yml`, | ||
| passing in the worker's Buildbot name and password. | ||
| - After a successful run, the worker should be connected to dev-ci.z.cash and | ||
| visible in its worker list. | ||
|
|
||
| - Create an AMI from the instance. This is the worker AMI to put into the | ||
| master.cfg for dev-ci.z.cash. | ||
| - 16 GB of storage should be sufficient. | ||
|
|
||
| - SSH into the instance, and edit the worker config to connect to ci.z.cash. | ||
|
|
||
| - Create an AMI from the instance. This is the worker AMI to put into the | ||
| master.cfg for ci.z.cash. | ||
| - 16 GB of storage should be sufficient. | ||
|
|
||
| - Delete the instance (it is no longer needed). | ||
|
|
||
| - Edit the master.cfg to turn the new worker into a latent (using the new AMI | ||
| IDs), add it to the appropriate worker groups, set up new builders etc. | ||
| - Deploy this via the normal PR review process. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| [ssh_connection] | ||
| pipelining = True |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| --- | ||
| - name: Fetch pip installer | ||
| get_url: | ||
| url: https://bootstrap.pypa.io/get-pip.py | ||
| dest: /tmp/get-pip.py | ||
|
|
||
| - name: Install pip | ||
| command: "{{ ansible_python.executable }} /tmp/get-pip.py" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| [Unit] | ||
| Description=Buildbot worker | ||
| Wants=network.target | ||
| After=network.target | ||
|
|
||
| [Service] | ||
| Type=forking | ||
| PIDFile=/home/{{ buildbot_worker_user }}/{{ buildbot_worker_name }}/twistd.pid | ||
| WorkingDirectory=/home/{{ buildbot_worker_user }} | ||
| ExecStart={{ pip_bin_dir }}/buildbot-worker start {{ buildbot_worker_name }} | ||
| ExecReload={{ pip_bin_dir }}/buildbot-worker restart {{ buildbot_worker_name }} | ||
| ExecStop={{ pip_bin_dir }}/buildbot-worker stop {{ buildbot_worker_name }} | ||
| Restart=always | ||
| User={{ buildbot_worker_user }} | ||
|
|
||
| [Install] | ||
| WantedBy=multi-user.target |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| OS: {{ ansible_distribution }} {{ ansible_distribution_version }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| OS: {{ ansible_distribution }} {{ ansible_distribution_version }} | ||
| Memory: {{ ansible_memtotal_mb }} MB | ||
| CPU: {{ ansible_processor[1] }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| --- | ||
| - name: Configure a Buildbot worker for Zcash CI | ||
| hosts: zcash-ci-worker-unix | ||
| become: true | ||
| gather_facts: False | ||
|
|
||
| vars_files: | ||
| - vars/default.yml | ||
| - vars/buildbot.yml | ||
|
|
||
| vars_prompt: | ||
| - name: "buildbot_worker_admin" | ||
| prompt: "Admin details" | ||
| default: "Zcash <[email protected]>" | ||
| - name: "buildbot_worker_name" | ||
| prompt: "Buildbot worker name (provided by ZECC)" | ||
| private: no | ||
| - name: "buildbot_worker_password" | ||
| prompt: "Buildbot worker password (provided by ZECC)" | ||
|
|
||
| pre_tasks: | ||
| - name: Install Python 2.7 for Ansible and Buildbot | ||
| raw: test -e /usr/bin/python || test -e /usr/bin/python2 || test -e /usr/bin/python2.7 || test -e /usr/local/bin/python2.7 || (test -e /usr/bin/apt && apt -qqy update && apt install -qqy python) || (test -e /usr/bin/dnf && dnf install -qqy python2) || (test -e /usr/sbin/pkg && pkg install -qqy python2) | ||
| register: output | ||
| changed_when: | ||
| - output.stdout != "" | ||
| - output.stdout != "\r\n" | ||
|
|
||
| - name: Check if Python is in the configured location | ||
| raw: test -e {{ ansible_python_interpreter }} | ||
| ignore_errors: true | ||
| register: python_check | ||
| when: ansible_python_interpreter is defined | ||
|
|
||
| - name: Fail if configured Python is unavailable | ||
| fail: | ||
| msg: "Python is not accessible at {{ ansible_python_interpreter }} on this host! Please set the inventory variable 'ansible_python_interpreter' to the location of the Python 2.7 binary." | ||
| when: ansible_python_interpreter is defined and python_check.rc == 1 | ||
|
|
||
| - name: Check if Python is in the default location | ||
| raw: test -e /usr/bin/python | ||
| ignore_errors: true | ||
| register: python_check | ||
| when: ansible_python_interpreter is undefined | ||
|
|
||
| - name: Fail if default Python is unavailable | ||
| fail: | ||
| msg: Python is not accessible at /usr/bin/python on this host! Please set the inventory variable 'ansible_python_interpreter' to the location of the Python 2.7 binary. | ||
| when: ansible_python_interpreter is undefined and python_check.rc == 1 | ||
|
|
||
| - name: Gathering Facts | ||
| setup: | ||
|
|
||
| - name: Fail if Python is the wrong version | ||
| fail: | ||
| msg: "The Python binary at {{ ansible_python.executable }} is version {{ ansible_python_version }}! Please set the inventory variable 'ansible_python_interpreter' to the location of the Python 2.7 binary." | ||
| when: ansible_python.version.major != 2 or ansible_python.version.minor != 7 | ||
|
|
||
| tasks: | ||
| - name: Get dependencies for distribution | ||
| include_vars: "{{ item }}" | ||
| with_first_found: | ||
| - files: | ||
| - "vars/{{ ansible_distribution }}-{{ ansible_distribution_version }}.yml" | ||
| - "vars/{{ ansible_distribution }}-{{ ansible_distribution_major_version | int }}.yml" | ||
| - "vars/{{ ansible_distribution }}.yml" | ||
| - "vars/{{ ansible_os_family }}.yml" | ||
| skip: true | ||
|
|
||
| - name: Collate dependencies | ||
| set_fact: | ||
| package_deps: "{{ buildbot_deps + fetch_deps + conf_deps + build_deps + link_deps + dist_deps }}" | ||
| python_modules: "{{ buildbot_modules + rpc_test_modules }}" | ||
|
|
||
| - name: Update rolling release [Arch Linux] | ||
| pacman: | ||
| update_cache: yes | ||
| upgrade: yes | ||
| when: ansible_distribution == 'Archlinux' | ||
|
|
||
| - name: Install required packages | ||
| package: | ||
| name: "{{ item }}" | ||
| state: present | ||
| with_items: "{{ package_deps }}" | ||
|
|
||
| - name: Install pip [CentOS] | ||
| include: tasks/install-pip.yml | ||
| when: ansible_distribution == 'CentOS' | ||
|
|
||
| - name: Install required Python modules | ||
| pip: | ||
| name: "{{ item }}" | ||
| state: latest | ||
| with_items: "{{ python_modules }}" | ||
| notify: restart buildbot-worker | ||
|
|
||
| - name: Set up the Buildbot worker user | ||
| user: | ||
| name: "{{ buildbot_worker_user }}" | ||
| comment: Buildbot worker | ||
| shell: /bin/bash | ||
| state: present | ||
|
|
||
| - name: Create Buildbot worker | ||
| command: > | ||
| buildbot-worker create-worker ~/{{ buildbot_worker_name }} | ||
| {{ buildbot_master_host }}:{{ buildbot_master_port }} | ||
| {{ buildbot_worker_name|quote }} {{ buildbot_worker_password|quote }} | ||
| args: | ||
| creates: "~/{{ buildbot_worker_name }}/buildbot.tac" | ||
| become_user: "{{ buildbot_worker_user }}" | ||
|
|
||
| - name: Set admin details for Buildbot worker | ||
| copy: | ||
| content: "{{ buildbot_worker_admin }}" | ||
| dest: "~{{ buildbot_worker_user }}/{{ buildbot_worker_name }}/info/admin" | ||
| owner: "{{ buildbot_worker_user }}" | ||
| group: "{{ buildbot_worker_user }}" | ||
| mode: "0644" | ||
|
|
||
| - name: Set host details for Buildbot worker | ||
| template: | ||
| src: "{{ buildbot_worker_host_template }}" | ||
| dest: "~{{ buildbot_worker_user }}/{{ buildbot_worker_name }}/info/host" | ||
| owner: "{{ buildbot_worker_user }}" | ||
| group: "{{ buildbot_worker_user }}" | ||
| mode: "0644" | ||
|
|
||
| - name: Copy Buildbot worker systemd service unit | ||
| template: | ||
| src: templates/buildbot-worker.service.j2 | ||
| dest: "/etc/systemd/system/buildbot-worker.service" | ||
| owner: root | ||
| group: root | ||
| mode: "0644" | ||
| notify: reload systemd | ||
|
|
||
| - name: Start Buildbot worker. | ||
| service: | ||
| name: buildbot-worker | ||
| state: started | ||
| enabled: yes | ||
|
|
||
| handlers: | ||
| - name: restart buildbot-worker | ||
| service: | ||
| name: buildbot-worker | ||
| state: restarted | ||
|
|
||
| - name: reload systemd | ||
| command: /bin/systemctl daemon-reload |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| buildbot_deps: | ||
| - python2-pip | ||
| build_deps: | ||
| - multilib/gcc | ||
| - make | ||
| pip_bin_dir: /usr/bin |
Oops, something went wrong.