Skip to content

Commit

Permalink
import_tomograms and import_particles documented
Browse files Browse the repository at this point in the history
  • Loading branch information
Jasenko Zivanov committed Nov 19, 2020
1 parent 8e03aee commit 0a29244
Show file tree
Hide file tree
Showing 2 changed files with 59 additions and 18 deletions.
29 changes: 29 additions & 0 deletions documentation/tomography/programs/import_particles.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="utf-8">
<title>Import Particles</title>
<link rel="stylesheet" href="../style.css">
</head>

<body>
<h1><tt>relion_tomo_import_particles</tt></h1>

<p>This program imports a set of 3D coordinates and (optionally) Euler angles to construct a new <a href="../data_types/particle_set.html">particle set</a>. The 3D coordinates can be provided either in one big .star file, or as a set of shorter files, one for each tomogram.</p>

<p>If all the coordinates are provided as one .star file, it has to contain at least the following columns:</p>

<ul>
<li><tt><b>rlnTomoName</b></tt>: name of the tomogram to which a particle belongs.</li>
<li><tt><b>rlnCoordinate&lt;X/Y/Z&gt;</b></tt>: the 3D coordinates within that tomogram.</li>
</ul>

<p>Optionally, the file may also contain the particle angles: <tt><b>rlnAngle&lt;Rot/Tilt/Psi&gt;</b></tt></p>

<p>In case the other format is chosen, then the input file only needs to contain the following columns: <tt><b>rlnTomoName</b></tt> and <tt><b>rlnTomoImportParticleFile</b></tt>. The latter column points to a set of files (one for each tomogram) that contain the coordinates (<tt><b>rlnCoordinate&lt;X/Y/Z&gt;</b></tt>) and (optionally) angles (<tt><b>rlnAngle&lt;Rot/Tilt/Psi&gt;</b></tt>)</p>


</body>

</html>
48 changes: 30 additions & 18 deletions documentation/tomography/programs/import_tomograms.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,37 +3,51 @@

<head>
<meta charset="utf-8">
<title>Add Tomo</title>
<title>Import Tomograms</title>
<link rel="stylesheet" href="../style.css">
</head>

<body>
<h1><tt>relion_tomo_add_tomo</tt></h1>
<h1><tt>relion_tomo_import_tomograms</tt></h1>

<p>This program adds a new tilt series to a <a href="../data_types/tomogram_set.html">tomogram set</a>. It is typically the first program to be used to create a Relion tomography data set. It imports an existing alignment from <a href="https://bio3d.colorado.edu/imod/">IMOD</a> and the initial CTF parameters from either <a href="https://grigoriefflab.umassmed.edu/ctffind4">CTFFind</a> or <a href="https://bio3d.colorado.edu/imod/doc/man/ctfplotter.html">CtfPlotter</a>. Additional parameters, such as the fractional electron dose and the chronological sequence of frames have to be supplied through the command line. The default parameters correspond to default IMOD output file names, so they can be omitted if the program is pointed to an IMOD project directory (<tt>--id</tt> argument).</p>
<p>This program imports a set of tilt series to form a <a href="../data_types/tomogram_set.html">tomogram set</a>. It is typically the first program to be used when creating a Relion tomography data set. It imports an existing alignment from <a href="https://bio3d.colorado.edu/imod/">IMOD</a> and the initial CTF parameters from either <a href="https://grigoriefflab.umassmed.edu/ctffind4">CTFFind</a> or <a href="https://bio3d.colorado.edu/imod/doc/man/ctfplotter.html">CtfPlotter</a>. Additional parameters, such as the fractional electron dose and the chronological sequence of frames have to be supplied by the user.</p>

<p>The tilt series alignment effectively defines an arbitrary 3D coordinate system, as well as a set of projections that map those 3D coordinates to the individual 2D images of the tilt series. The aim of this program is to translate that alignment into the Relion nomenclature. If used successfully, this will produce an equivalent 3D coordinate system inside Relion that will allow particle positions already defined in an IMOD tomogram to be used unchanged.</p>
<p>The tilt series alignment effectively defines an arbitrary 3D coordinate system, as well as a set of projections that map those 3D coordinates to the individual 2D images of the tilt series. The aim of this program is to translate that alignment into a format readable by Relion. If used successfully, this will produce an equivalent 3D coordinate system inside Relion that will allow particle positions already defined in an IMOD tomogram to be used unchanged.</p>

<p>Optimally, the tilt series would be imported into Relion right after alignment, and the particles would be picked in a low-magnification tomogram generated by <a href="reconstruct_tomogram.html"><tt>relion_tomo_reconstruct_tomogram</tt></a>. That way, the particle positions would be guaranteed to be correct.</p>

<p>Note: the user is advised to call this program from a script, inside a loop running over all tilt series.</p>
<p>The primary input to <tt>relion_tomo_import_tomograms</tt> is a <tt>.star</tt> file containing one row for each tilt series and at least the following columns:</p>

<ul>
<li><tt><b>rlnTomoImportImodDir</b></tt>: path to the IMOD directory. That directory has to contain the command files used to run IMOD's <tt>tilt</tt> and <tt>newst</tt> programs. Typically, those are named <tt>tilt.com</tt> and <tt>newst.com</tt>. If those are not their names, then the actual names can be specified using the <b>--tc</b> and <b>--nc</b> arguments, respectively. Additional IMOD files that are referenced inside those <tt>.com</tt> files also need to be present inside the IMOD directory.</li>
<li><tt><b>rlnTomoImportCtfFindFile</b></tt> or <tt><b>rlnTomoImportCtfPlotterFile</b></tt>: path to the initial CTF estimate from either CTFFind or CtfPlotter, respectively.</li>
<li><tt><b>rlnTomoTiltSeriesName</b></tt>: path to the actual tilt series. Note: if the tilt series ends with <tt>.st</tt>, this needs to be specified with an <tt>.st:mrc</tt> ending to tell Relion to interpret it as an mrc file.</li>
</ul>

<p>The following additional columns may be present. If they are not, then the corresponding command line arguments will be used. In that case, they will be identical for all tilt series.</p>

<ul>
<li><tt><b>rlnTomoName</b></tt>: the tomogram name. The user is advised to specify a short and meaningful name here, since it will make the data more readable and simplify the resulting directory structure. Names like e.g. <tt>TS_001</tt> are recommended. If not specified, the entire path to the tilt series (i.e. the value of <tt>rlnTomoTiltSeriesName</tt>) will be used instead.</li>
<li><tt><b>rlnTomoImportFractionalDose</b></tt>: the electron dose corresponding to one tilt image. If omitted, the value of the <tt>--fd</tt> argument will be used.</li>
<li><tt><b>rlnTomoImportOrderList</b></tt>: path to a two-column text file specifying the <i>chronological</i> order in which the images were acquired. That file has to contain two numbers per line separated by a comma (and no spaces), where the first number counts up from 1, while the second describes the sequence of tilt angles. To determine the sequence of tilt images, the program will look for the closest tilt angle for each image (as stored in the <tt>.tlt</tt> file). This approach can deal with frames missing from the tilt series. If not specified in the .star file, the <tt>--ol</tt> input argument will be used.</li>
<li><tt><b>rlnOpticsGroupName</b></tt>: an arbitrary name for an optics group. This allows the set of tilt series to be separated into subsets that share the same optical aberrations. This is useful if the data have been collected in multiple sessions that might exhibit different aberrations. If omitted, all tilt series will be assigned to the same default optics group.</li>
<li><tt><b>rlnTomoImportOffset&lt;X/Y/Z&gt;</b></tt>: an arbitrary offset to the 3D coordinate system. This is useful if particles have already been picked in tomograms that have been cropped after reconstruction by IMOD. If the IMOD-internal <tt>SHIFT</tt> command has been used to apply offsets, then this will be handled internally and does <i>not</i> need to be specified here. If omitted, then the values of the <tt>--off&lt;x/y/z&gt;</tt> command line arguments will be used instead (which default to 0).</li>
<li><tt><b>rlnTomoImportCulledFile</b></tt>: output file name for a new tilt series with the excluded frames missing. This is only needed if tilt images have been excluded using IMOD's <tt>EXCLUDE</tt>, <tt>EXCLUDELIST</tt> or <tt>EXCLUDELIST2</tt> commands. In that case, this becomes a mandatory parameter.</li>

</ul>

<h3>General program arguments:</h3>

<ul>
<li><b>--ts</b>: tilt-series file name, typically an <tt>.st</tt> file. Note: since <tt>.st</tt> files are really <tt>.mrc</tt> files, these should be specified as <tt>&lt;name&gt;.st:mrc</tt> to tell Relion to read them as such.</li>
<li><b>--name</b>: the <tt>tomo_name</tt> chosen for this tilt series (see <a href="../data_types/tomogram_set.html">tomogram set</a>). If absent, it will be named <tt>ts_&lt;N&gt;</tt>, where <tt>&lt;N&gt;</tt> is the next available integral number. The user is strongly advised to name tomograms the same as the original IMOD directory to prevent confusion.</li>
<li><b>--hand</b>: the handedness of the tilt geometry. The value of this parameter is either +1 or -1, and it describes whether the focus increases or decreases as a function of Z distance. It has to be determined experimentally. In our experiments, it has always been -1.</li>
<li><b>--io</b>: the input <i>and</i> output <a href="../data_types/tomogram_set.html">tomogram set</a>. The current tilt series will be added to the existing set, and the existing set will be overwritten.</li>
<li><b>--oc</b>: output file name for a culled stack. In case images have been excluded inside IMOD (using the <tt>EXCLUDE</tt>, <tt>EXCLUDELIST</tt> or <tt>EXCLUDELIST2</tt> commands), a new tilt series with those images missing will be written to that file.</li>
<ul>
<li><b>--i</b>: path to the input .star file.</li>
<li><b>--t</b>: path to an optional input <a href="../data_types/tomogram_set.html">tomogram set</a>. If specified, the imported tomograms will be added to that set.</li>
<li><b>--hand</b>: the handedness of the tilt geometry. The value of this parameter is either +1 or -1, and it describes whether the focus increases or decreases as a function of Z distance. It has to be determined experimentally. In our experiments, it has always been -1.</li>
</ul>


<h3>Optics arguments:</h3>

<ul>
<li><b>--ctffind</b> or <b>--ctfplotter</b>: the output file from either CTFFind or CtfPlotter, specifying the initial defocus and astigmatism values of each frame.</li>
<li><b>--angpix</b>: the pixel size in &#8491;.</li>
<li><b>--voltage</b>: the voltage in kV.</li>
<li><b>--Cs</b>: the spherical aberration in mm.</li>
Expand All @@ -44,24 +58,22 @@ <h3>Optics arguments:</h3>
<h3>Electron dose arguments:</h3>

<ul>
<li><b>--ol</b>: the file name of a frame-order list file. This is a file containing two numbers per line separated by a comma (and no spaces), where the first number counts up from 1, while the second describes the <i>chronological</i> sequence of tilt angles. This is needed to associate the frames of the tilt series with the order in which they were collected. Specifically, each frame is associated with the line in the frame-order file that shows the closest tilt angle to the tilt angle of that frame (as stored in the <tt>.tlt</tt> file).</li>
<li><b>--tlt</b>: a list of tilt angles of each frame. If omitted, the corresponding <tt>.tlt</tt> file from the IMOD directory will be used.</li>
<li><b>--fd</b>: The fractional (i.e. per tilt-image) electron dose.</li>
<li><b>--ol</b>: the file name of a frame-order list file. Corresponds to the <tt>rlnTomoImportOrderList</tt> column in the input .star file.</li>
<li><b>--fd</b>: The fractional (i.e. per tilt-image) electron dose. Corresponds to the <tt>rlnTomoImportFractionalDose</tt> column in the input .star file.</li>
</ul>


<h3>IMOD import arguments:</h3>

<ul>
<li><b>--id</b>: the path to the IMOD directory corresponding to this tilt series. That directory has to contain the command files used to run IMOD's <tt>tilt</tt> and <tt>newst</tt> programs. Typically, those are named <tt>tilt.com</tt> and <tt>newst.com</tt>. If those are not their names, then the actual names can be specified using the <b>--tc</b> and <b>--nc</b> arguments, respectively. Additional IMOD files that are referenced inside those <tt>.com</tt> files also need to be present inside the IMOD directory.</li>
<li><b>--flipYZ</b>: if the IMOD tomogram has been flipped along Y and Z (i.e. rotated around X) after an IMOD reconstruction and before the particles have been picked, this will apply the same transformation to the Relion coordinate system. This will allow Relion to use particle positions defined in the X-rotated tomogram unchanged.</li>
<li><b>--flipZ</b>: same as above, in case the Z axis has been flipped. This can be used together with the <tt>--flipYZ</tt> option.</li>
<li><b>--flipAng</b>: indicates that all angles in the IMOD <tt>.tlt</tt> file have also been flipped. This is rare.</li>
<li><b>--off&lt;x/y/z&gt;</b>: applies an offset to the 3D coordinate system, in case the IMOD tomogram has been cropped after reconstruction and before particle picking. Note: if the coordinates have been shifted inside IMOD's <tt>tilt.com</tt> using the <tt>SHIFT</tt> command, then that shift will be handled internally and does not need to be specified here.</li>
<li><b>--off&lt;x/y/z&gt;</b>: applies an offset to the 3D coordinate system. Equivalent to the <tt>rlnTomoImportOffset&lt;X/Y/Z&gt;</tt> column in the input .star file.</li>
</ul>


<p>Final note: not all IMOD options are supported by this program. For example, the X-axis tilt option is not supported. In case it has been used, we recommend ignoring it and refining the resulting alignment using the program <a href="align_frames.html"><tt>relion_tomo_align_frames</tt></a>.</p>
<p>Final note: not all IMOD options are supported by this program. For example, the X-axis tilt option is not supported. In case it has been used, we recommend ignoring it and refining the resulting alignment using the program <a href="align.html"><tt>relion_tomo_align</tt></a>.</p>


</body>
Expand Down

0 comments on commit 0a29244

Please sign in to comment.