Skip to content

Latest commit

 

History

History
138 lines (95 loc) · 5.78 KB

Hydra Cheat Sheet.md

File metadata and controls

138 lines (95 loc) · 5.78 KB
Raw

Hydra Cheat Sheet

  1. Login to hydra Choose one of the login nodes:

    ssh [email protected] or ssh [email protected]

    You will be prompted to enter your password.

  2. If I forgot my password or locked my account?
    Use the password reset webpage: https://hydra-7.si.edu/ssp/?action=sendtoken

  3. How do I change my password?

You need to change your password every 180 days.
You can also use the password change website: https://hydra-7.si.edu/ssp/index.php

a. Login to hydra-login01.si.edu: `ssh [email protected]`  
b. Type in the password command: `passwd`  
c. You'll be prompted to enter your **current** password (once)  
d. Then  you enter your **new** password (twice)

  1. Where am I when I log in?
    In your home directory: /home/myusername
    (you can always type pwd to find where you are)

  2. Where are my analysis files?
    Your files should be in the directory: /scratch/genomics/myusername
    You might also have files in /pools/genomics/myusername, /data/genomics/myusername

  3. Which programs are available on hydra?
    module avail
    It will output a list of software/modules available on hydra

  4. How do I load a module? module load module/name

    Example: to load bwa v. 0.7.17, you would type: module load bioinformatics/bwa/0.7.17

    You can find out more about the module (like the name of the executable) with: module help module/name

  5. What if the program I need is not listed, can I install it on Hydra?
    Yes. Try compiling any new software you need in your own space and feel free to reach out with questions. You can also install miniconda and use that to install software.

  6. How do I start an analysis?

    You can create a job file either using nano or the Qsub Generator, which is a web tool to create job files.

    You can submit a job using the command
    qsub jobfile.job

    OR

    You can use the interactive node for light-weight jobs. To access the interactive node, just type:
    qrsh
    When you type qrsh, you go back to your home directory.

    The memory available without any additional parameters is 8GB. To increase this values use:
    qrsh -pe mthread N
    where N is between 2 and 10, to use 2 to 10 slots or 16 to 80GB of memory.

    In both cases, you need to load the necessary modules, either in the job file, or by typing module load [...] in the interactive node.

  7. What are the available queues?
    Queues are divided by how long a job can run, and also the amount of RAM available per CPU.

    • Short (s): up to 7 hours
    • Medium (m): up to 6 days
    • Long (l): up to 30 days
    • Unlimited (u): well... unlimited

    The queues are also divided into high CPU (up to 8GB per CPU) or high memory (above 8 GB). Here's an example:

    # ------------Parameters------------------ #
#$ -S /bin/sh
#$ -pe mthread 4 #(multithread w/ 4 CPUs)
#$ -q mThC.q #(medium Time, high CPU)
#$ -l mres=32G,h_data=8G,h_vmem=8G
    Important: more memory is not necessarily better if the software can't use it efficiently. Jobs are prioritized based on queue, so low-RAM jobs will move through the queue faster. Inefficient jobs are flagged by the Hydra admin.

  8. How do I check my job status?

    To check the status of all of your jobs:
    qstat

    To check the status of an specific job:
    qstat -j job_id_number

    You can find the job number in the log files, or when you type qstat.

  9. How do I kill a job?
    qdel job_id_number

  10. After my job is done, how to check its parameters, memory used, etc?
    qacct+ -j job_id_number

  11. My job failed… Why?
    a. Check the log file. Many programs will output the errors, which help understand the issue. Common issues are: misspelled filenames, wrong paths, etc.
    b. Check your job using the qacct command (previous question). If maxvmem is the same as the memory you requested, your job probably used too much memory and was killed
    c. Check which queue was used (s, m, l, u) and if the job ended right at the time limit of the queue.
    d. If none of those help answer your question, ask for help :)

  12. How do I check my disk quota on Hydra?
    quota+

    Important: Files older than 180 days on /scratch/genomics and /pool/genomics are scrubbed. Files in your home directory and /data/genomics are never scrubbed but disk quotas are smaller.

  13. How do I copy my files from Hydra to a local computer?
    Run this command from a local (aka not connected to Hydra) terminal window:

    Copying FROM hydra TO your computer scp [email protected]:/hydra-path/file destination-on-my-computer

    Copying TO hydra FROM your computer scp /path-on-my-computer/file [email protected]:/hydra-path

    (for directories, use the flag -r after scp)

  14. My files were scrubbed. What do I do?
    Please refer to this page for more information on how to request files to be restored (requests must be received by the Friday following scrubbing).

  15. How do I use Dropbox to backup my files?
    Please refer to this page for more information.

  16. How do I find help?