Skip to content

Getting started

Jump to bottom Edit New page
Daniel Perrett edited this page Sep 15, 2023 · 66 revisions

This is a guide for setting up your local Species+ website. Assumes Ubuntu OS

!!! WARNING: on Ubuntu, rbenv seems to be causing segmentation fault errors with postgres. RVM works with no problems !!!

For Mac Users

Minimal setup

Prerequisite

  • You can obtain the .env file contents required for this project on the Shared-Informatics Development Environments LastPass searchable by SAPI Development Environment
  • You will also need an up-to-date SQL dump to restore the database. You can obtain a database dump from S3. One way is with Cyberduck. Credentials can be found for the S3 in the shared LastPass account. There should be a access_key_id and a secret_access_key under the name Species+ S3 access keys. Using duck through command line you could download e.g. duck -vd s3:/speciesplus.bkp/Daily/db/new_db_server/sapi_website_db_daily/2020.05.25.00.00.02/sapi_website_db_daily.tar ~/Downloads/ -- it will prompt for you for credentials that should be stored for future use. You can then extract the file in two steps, first untar e.g. tar -vxf sapi_website_db_daily.tar then unzip the SQL within e.g. bzip2 -d sapi_website_db_daily/databases/PostgreSQL.sql.bz2

Step 1: Install dependencies

Before you can install the Species+ application, you will need some tools and libraries installed in your system. Depending on the operating system you're using, these instructions may differ.

This guide is based on a Ubuntu installation. For MacOs jump to - MacOS users

  1. GitYou can obtain a database dump from S3. One way is with Cyberduck.

Credentials can be found for the S3 in the shared LastPass account. There should be a access_key_id and a secret_access_key. Using duck through command line you could download e.g. duck -vd s3:/speciesplus.bkp/PostgreSQL.sql.bz2 ~/Downloads/ -- it will prompt for you for credentials that will be stored for future use.

We use git (GitHub) for source version control.

sudo apt-get install git-core

Check for version to make sure git is installed correctly:

git --version

  1. PostgreSQL v. 9.5

PostgreSQL is the relational store that backs the Species+ database.

sudo apt-get install libpq-dev postgresql-9.5 postgresql-contrib-9.5

Check for version to make sure PostgreSQL is installed correctly:

psql --version

More information on PostgreSQL usage in Species+

  1. Ruby v. 2.0.0

We recommend using rvm to install ruby. We recommend NOT installing system ruby.

Update 04/2020

Please note that there may be an issue when performing a local rvm install of older versions of Ruby (older than ~v2.3) where the OpenSSL version required is no longer hosted resulting in a 404, in which case it's worth trying to install v2.4

For Ubuntu you will need the libssl1.0-dev library: sudo apt install libssl1.0-dev If you have Rbenv installed, you can setup Ruby v2.2.3 with: rbenv install 2.2.3. Based on the .ruby-version file in the root of the project, it should then be using v2.2.3 within the project.

Update 11/2021

Ruby v. 2.3.1

  1. other dependencies
`sudo apt-get install libgmp3-dev`

Step 2: Install the application

  1. Create a local copy of the code repository on your machine:

git clone [email protected]:unepwcmc/SAPI.git

Now change the branch to "develop":

git checkout develop

Please refer to Branching strategy for more information about using branches in Species+.

  1. Install bundler (version 1.17.3, not Bundler 2)

gem install bundler -v 1.17.3

In fact, please check the version specified hopefully within the Gemfile.lock right at the bottom: BUNDLED WITH, and use that (if you haven't already accidentally modified the lock).

  1. Install all the gems

Inside the SAPI directory:

bundle install

If bundle --version isn't 1.17.3, but it is installed - then you can specify the version to use like so: bundle _1.17.3_ install

(this might take a while)

  1. Configure GeoIP

We use a GeoIP mapping to track some user actions. Download the GeoLite Country and GeoLite City free databases from http://dev.maxmind.com/geoip/legacy/geolite/

Create a config file which points to the locations where you saved the databases. A sample config file is provided in config/max_mind.yml

Note you don't have a copy of the 'org_db', which is an optional subscription data set. Just leave the name out, e.g.

  city_db: '/usr/share/max_mind_lite/GeoLiteCity.dat'
  country_db: '/usr/share/max_mind_lite/GeoIP.dat'
  org_db:

This product includes GeoLite2 data created by MaxMind, available from http://www.maxmind.com.

It does appear that .dat files are no longer available. See also here. The impact this has on the project is unknown to this author - mmdb files may need to be converted to dat before use.

  1. Configure secrets

cp config/secrets.yml.example config/secrets.yml

  1. Create the database
cp config/database.yml.sample config/database.yml
rake db:create

This will copy the database configuration example file and create a sapi_development database in your PostgreSQL instance.

  1. Populate the database from a dump:

You may need to extract the SQL before import e.g. bunzip2 PostgreSQL.sql.bz2. Be prepared for a long wait both to extract and to import. Sorry!

Review and follow the following steps, omitting any you may already have done for other projects.

# Create a random password for the WCMC user:
# echo 'localhost:5432:*:wcmc:'$(head -c36 /dev/urandom | base64 | tr -dc '0-9a-zA-Z') >> ~/.pgpass
WCMC_PASSWORD=$(grep localhost:5432:.:wcmc: ~/.pgpass | cut -d: -f5 | tr -d '\n')
SQL_DUMP="../sapi_website_db_weekly/databases/PostgreSQL.sql"

# Create the WCMC user, unless you already have it from another project
psql -h localhost -U postgres -c "CREATE ROLE wcmc SUPERUSER LOGIN PASSWORD '$WCMC_PASSWORD'"

# Drop and create the sapi_development database
psql -h localhost -U postgres -c 'DROP DATABASE IF EXISTS sapi_development'
psql -h localhost -U postgres -c 'CREATE DATABASE sapi_development OWNER wcmc'

# Apply the dump
psql -X -v ON_ERROR_STOP=1 -f $SQL_DUMP -d sapi_development -h localhost -U wcmc

# Note that the REFRESH MATERIALIZED VIEW commands might fail either:
#
# - with error `function squish(text) does not exist`
# - with error `could not write to file "...": No space left on device`
#
# Because they happen at the end and the dump is not transactional, they can be
# done separately afterwards.

# In case any of the refresh materialised views failed:
#
# - the matview query will be more efficiently planned if statistics are correct
#   so run `VACUUM ANALYSE`
# - setting `work_mem` to a higher value than the default 4MB for the session
#   might reduce the amount of disk writes needed. Maybe.
psql -h localhost -U postgres -c 'VACUUM ANALYSE;'
{ echo "SET work_mem TO '64MB';"; grep REFRESH $SQL_DUMP; } | psql -d sapi_development -h localhost -U wcmc

If you have any issues, you can drop the DB with the dropdb variation of the above command.

  1. Apply any pending migrations

rake db:migrate

Note: don't worry about the verbose output of this command. The standard rails db:migrate task has been extended in SAPI to handle the installation of stored procedures and views. The output is just debug with a list of scripts that get run. As long as there's no error / exception it's all fine.

  1. Start the application server:

rails s

  1. Done! Try the following urls:

Additional setup

Enable background jobs

Background jobs are used for:

  • bulk operations in the Species Admin
  • bulk operations in the Trade Admin
  • generation of CITES Checklist outputs

Please proceed if you need any of the above enabled.

  1. Install Redis

See this very helpful guide.

We use redis as a key-value store for the background jobs runner (sidekiq).

sudo apt-get install redis-server

You can set it up as a background service by modifying the config sudo nano /etc/redis/redis.conf and change supervised no to supervised systemd then save & exit and sudo systemctl restart redis.service

Check for version to make sure redis is installed correctly:

redis-server -v

  1. start redis server:

redis-server

  1. start sidekiq

bundle exec sidekiq

Enable Checklist PDF Outputs

The CITES Checklist is a small Ember.js application that interfaces SAPI and needs to be set up separately. Follow the instructions in https://github.com/unepwcmc/cites-checklist.

To enable PDF outputs generation for the Checklist, it is necessary to install LaTeX.

sudo apt-get install texlive texlive-full texlive-latex-recommended texlive-fonts-recommended

The pdfs are generated by background jobs, so you need sidekiq running as well.

More information on LaTeX usage in Species+

MacOS users

(Updated on 16/Mar/2023)

The documendation is written based on Intel Chip Mac + OS 13.2.1

Having Arm (M1,M2) chip Mac?

Tools Needed:

  • Homebrew (Can be any version)
  • rbenv (Can be any version)
  • Ruby 2.3.1
  • Bundle 1.17.3
  • Postgres 11
  • Nodejs (v16 was used but it can be any version)

Instructions:

  1. Set Up Ruby 2.3.1
  2. Set Up Bundler && Install Dependencies
  3. Set Up Postgres 11 & Create Database
  4. Run DB Migration
  5. Run Server

Set Up Ruby 2.3.1:

Back To Instruction List

**The instructions is based on the solution here.

  1. Follow here to install rbenv

  2. Install OpenSSL v1.0

    You have OpenSSL v1.0 installed alreay

    You must change the --with-openssl-dir path to your openssl installation directory to install ruby 2.3.1

     RUBY_CFLAGS='-DUSE_FFI_CLOSURE_ALLOC -Wno-error=implicit-function-declaration' RUBY_CONFIGURE_OPTS='--enable-shared --with-openssl-dir=CHANGE ME TO YOUR OPENSSL DIRECTORY' rbenv install 2.3.1

    You have not installed OpenSSL v1.0

     brew install rbenv/tap/[email protected]

    ** (Optional But Recommended) Run the command below to unlink openssl because v1.0 is not secure any more.

     brew unlink [email protected]

    Run the command below to install ruby 2.3.1

     RUBY_CFLAGS='-DUSE_FFI_CLOSURE_ALLOC -Wno-error=implicit-function-declaration' RUBY_CONFIGURE_OPTS='--enable-shared --with-openssl-dir=/usr/local/opt/[email protected]' rbenv install 2.3.1

  3. After installing sucessfully, you will get a message telling you 'Installed ruby-2.3.1 to /Users/xxx/.rbenv/versions/2.3.1'

  4. Set Ruby Versions

    Use ruby 2.3.1 globally in your Mac

     rbenv global 2.3.1
 ruby -v

    Make sure version number is 2.3.1

    Use ruby 2.3.1 locally only for this project cd to your app root directory

     cd /root-folder (i.e cd /Document/SAPI/)
 rbenv local 2.3.1
 ruby -v

    Make sure version number is 2.3.1

Set Up Bundler && Install Dependencies

Back To Instruction List

  1. Install bundler

     gem install bundler -v 1.17.3

  2. Change pg version to 0.19.0 in Gemfile.lock file

  3. Install dependencies

     bundler install

Set Up Postgres 11 & Create Database:

Back To Instruction List

  1. install postgres@11 using brew by following the link

    **Remember to put version numbers.

     brew install postgres@11

  2. Create database using - pgAdmin or any other GUI DB Management Tools

    pgAdmin Tutorial

    Database name, username, password can be found in /SAPI/config/database.yml

    **Make sure username does exist

  3. Create database using - command line

     bundle exec rake db:create

     If there are errors on `bundle exec rake db:create` saying ***FATAL:  database "sapi_development" does not exist (PG::ConnectionBad)***

 Need to login to postgres:

     psql

 if error - database “someusername” (btw. someusername is your mac username) does not exist create it:

     createdb someusername
     sudo -i -u someusername
 In postgres cli the following will show list the someusername db (here my user name - dagmarakukla):

     \du


 ![image](https://user-images.githubusercontent.com/38047481/188676881-97c1a3dd-6e93-46fd-b112-baf2e3441b61.png)

    Create database

    CREATE DATABASE sapi_development WITH OWNER postgres;

    and exit:

     \q

Run DB Migration

Back To Instruction List

bundle exec rake db:migrate

Run Server

Back To Instruction List

You are all good to start running the server

bundle exec rails s
Add a custom footer
Add a custom sidebar
Clone this wiki locally