-
Notifications
You must be signed in to change notification settings - Fork 5
Keywords
John Hoffer edited this page Mar 17, 2017
·
7 revisions
Three keyword structures from UtilityLayer import INPUT, RUNTIME, OUTPUT help butterfly layers share the same keywords when modifying INPUT requests, loading tiles at RUNTIME and serving text OUTPUT.
The RequestParser.py and Query.py subclasses have each have their own set of these INPUT,RUNTIME, and OUTPUT structures. You'll get and set them from a DataQuery like this:
from QueryLayer import Utility
query = DataQuery(**terms)
img_format = query.INPUT.IMAGE.FORMAT
img_view = query.INPUT.IMAGE.VIEW
img_type = query.OUTPUT.INFO.TYPE
# For printing in a single line such as any of the four examples in block quote below
def show(c):
print c, img_view.NAME,'the', img_type.VALUE, img_format.VALUE, 'with', img_view.VALUE
# Only if grayscale view is set
if img_view.VALUE == img_view.GRAY.NAME:
# set the view based on the format
is_big_int = img_type.VALUE in img_type.ID_LIST
no_big_int_gray = img_format.VALUE in img_format.COLOR_LIST
# If big integers must not be grayscale, try colormap
if is_big_int and no_big_int_gray:
img_view.VALUE = img_view.COLOR.NAME
show('Changed:')
else:
show('')
else:
show('')
'''
view the uint8 png with grayscale
view the uint32 tif with grayscale
Changed: view the uint16 jpg with colormap
Changed: view the uint32 png with colormap
'''- For the needed
termsforDataQuery, please see the DataQuery page - You can always get static
NAMEandLISTkeywords fromDataQuery- You should only set or get
VALUEkeywords fromquery = DataQuery(**terms) - The
VALUEkeywords fromDataQuerywill always be a global defaultVALUE
- You should only set or get
-
InfoQueryandTileQuerywork in much the same way
- Here are all three keywords structures with all default
VALUE:
INPUT: | RUNTIME: | OUTPUT:
FEATURES: | CACHE: | FEATURES:
BOOL_LIST: | LIST: | LINKS:
- is_synapse | - max-cache-size | ID:
- is_neuron | - meta-size | NAME: synapse_id
LABEL_LIST: | MAX: | LIST:
- synapse_ids | NAME: max-cache-size | - synapse_parent_pre
- neuron_ids | VALUE: 1073741824 | - synapse_parent_post
LINK_LIST: | META: | - synapse_id
- synapse_parent | NAME: meta-size | POST:
- neuron_children | VALUE: 567 | NAME: synapse_parent_post
LIST: | DB: | PRE:
- neuron_children | FILE: | NAME: synapse_parent_pre
- synapse_ids | CONFIG: | INFO:
- neuron_ids | GROUP_LIST: | CHANNEL:
- synapse_keypoint | - experiments | NAME: name
- neuron_keypoint | - samples | LIST:
- voxel_list | - datasets | - path
- is_synapse | - channels | - data-type
- is_neuron | LIST: | - list
- synapse_parent | - path | - name
NAME: feature | - experiments | - dimensions
NEURON_CHILDREN: | - samples | NAMES:
NAME: neuron_children | - datasets | NAME: list
POINT_LIST: | - channels | PATH:
- synapse_keypoint | NAME: /home/john/.rh-config.yaml | NAME: path
- neuron_keypoint | PATH: | SIZE:
SYNAPSE_LINKS: | NAME: path | LIST:
NAME: synapse_parent | LIST: | - y
TABLES: | - synapse-connections.json | - x
LIST: | - /home/john/.rh-config.yaml | - z
- synapse | SYNAPSE: | NAME: dimensions
- neuron | LIST: | X:
NEURON: | - neuron_1 | NAME: x
LIST: | - neuron_2 | Y:
- neuron_keypoint | - synapse_center | NAME: y
- neuron_ids | NAME: synapse-connections.json | Z:
- is_neuron | NEURON_LIST: | NAME: z
NAME: neuron | - neuron_1 | TYPE:
SYNAPSE: | - neuron_2 | ID_LIST:
LIST: | POINT: | - uint16
- synapse_keypoint | LIST: | - uint32
- neuron_children | - z | LIST:
- synapse_parent | - y | - uint16
- synapse_ids | - x | - uint32
- is_synapse | NAME: synapse_center | - uint8
NAME: synapse | JOIN: | - float32
VOXEL_LIST: | NAME: '{}://{}' | NAME: data-type
- voxel_list | LIST: | RAW_LIST:
GROUP: | - '{}://{}' | - uint8
LIST: | TABLE: | - float32
- experiment | ALL: | VALUE: uint8
- sample | LIST: |
- dataset | - z |
- channel | - y |
NAME: name | - x |
IMAGE: | POINT_LIST: |
FORMAT: | - z |
COLOR_LIST: | - y |
- png | - x |
- jpg | LIST: |
- bmp | - neuron |
LIST: | - synapse |
- zip | NEURON: |
- png | KEY: |
- jpg | NAME: neuron |
- bmp | KEY_LIST: |
- tif | - neuron |
- tiff | LIST: |
NAME: format | - neuron |
TIF_LIST: | NAME: neuron |
- tif | SYNAPSE: |
- tiff | KEY: |
VALUE: png | NAME: __id |
ZIP_LIST: | KEY_LIST: |
- zip | - n1 |
LIST: | - n2 |
- view | LIST: |
- format | - n1 |
VIEW: | - n2 |
COLOR: | - __id |
NAME: colormap | NAME: synapse |
GRAY: | NEURON_LIST: |
NAME: grayscale | - n1 |
LIST: | - n2 |
- colormap | ERROR: |
- grayscale | CACHE: |
- rgb | ADD: |
NAME: view | ACT: 'Add {value} to cache. |
RGB: | Cache now {size} bytes. ' |
NAME: rgb | LOG: info |
VALUE: grayscale | NAME: add_query |
INFO: | LIST: |
FORMAT: | - over_max |
LIST: | - add_query |
- json | MAX: |
- yaml | ACT: 'Cannot cache {value}. |
NAME: format | {size} bytes is over max. ' |
VALUE: json | LOG: warning |
ID: | NAME: over_max |
NAME: id | CHECK: |
LIST: | NAME: check |
- id | LIST: |
- format | - term |
METHODS: | - value |
FEAT: | - check |
NAME: entity_feature | - size |
GROUP_LIST: | OUT: |
- experiments | NAME: value |
- samples | REQUEST: |
- datasets | CHECK: |
- channels | ACT: 'The {term} {value} is |
IMAGE_LIST: | not {check}. ' |
- data | LOG: info |
- mask | NAME: bad_check |
INFO_LIST: | LIST: |
- channel_metadata | - bad_check |
- entity_feature | SERVER: |
LIST: | LIST: |
- entity_feature | - start |
- channel_metadata | - stop |
- data | START: |
- mask | ACT: "\n*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*\n\ |
- experiments | \ Start server on port {value}.\n\ |
- samples | _______________________________\n\ |
- datasets | \ " |
- channels | LOG: info |
META: | NAME: start |
NAME: channel_metadata | STOP: |
NAME: method | ACT: "\n|||||||||||||||||||||||||||||||\n\ |
POSITION: | \ Stop server on port {value}.\n\ |
DEPTH: | *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*\n\ |
NAME: depth | \ " |
VALUE: 1 | LOG: info |
HEIGHT: | NAME: stop |
NAME: height | SIZE: |
LIST: | NAME: size |
- height | TERM: |
- width | NAME: term |
- depth | IMAGE: |
- y | BLOCK: |
- x | LIST: |
- z | - 1 |
WIDTH: | - 512 |
NAME: width | NAME: block-size |
X: | VALUE: |
NAME: x | - 1 |
Y: | - 512 |
NAME: y | - 512 |
Z: | LIST: |
NAME: z | - source-type |
RESOLUTION: | - block-size |
LIST: | SOURCE: |
- y-res | HDF5: |
- x-res | INNER: |
- z-res | NAME: dataset-path |
- resolution | VALUE: main |
X: | LIST: |
NAME: x-res | - filename |
XY: | - dataset-path |
NAME: resolution | NAME: hdf5 |
VALUE: 0 | OUTER: |
Y: | NAME: filename |
NAME: y-res | LIST: |
Z: | - hdf5 |
NAME: z-res | - tilespecs |
| - mojo |
| - regularimagestack |
| NAME: source-type |
| VALUE: regularimagestack |
| TILE: |
| KJI: |
| NAME: kji |
| LIST: |
| - kji |
| - zyx |
| - scales |
| SCALES: |
| NAME: scales |
| ZYX: |
| NAME: zyx |
| |
On x, y, and z
- The
NAMESofx,y, andzare overloaded- In
INPUT.POSITION, they define the origin of the full request in source voxels. - In
RUNTIME.TILE.OUTSIDE, they countblock-sizeoffsets of a tile from the origin. - In
OUTPUT.SIZE, they give the full-size source dimensions in source voxels.
- In
On block-size and resolution
- The
block-sizedefines the maximum output voxels for a given tile.- For all in
RUNTIME.SOURCEexceptmojo, block size is arbitrary- The
block-sizefor all butmojois the shape of one image or slice.
- The
- For all in
-
The concept of
resolution, scale, or zoom-level takes two forms here.-
INPUT.RESOLUTION.XYshould apply to x and y, but not to z- Other members of
INPUT.RESOLUTIONoverruleINPUT.RESOLUTION.XY - These give a number of times to halve the image resolution along an axis.
- Other members of
-
RUNTIME.TILEhasSCALESsuch that:RUNTIME.TILE.SCALES.VALUE == map(lambda r: 2**getattr(INPUT.RESOLUTION,r).VALUE,'ZYX')- Or, in pseudocode
RUNTIME.TILE.SCALES = 2**INPUT.RESOLUTION
-
-
Home
- API
- OCP
- Data Formats
- Developers
- Harvard RC Cluster
- Keywords
- Testing
- VM
- AccessLayer
- API
- OCP
- QueryLayer
- ImageLayer
- HDF5
- New Formats
- DatabaseLayer
- Mongo
- CoreLayer
- Core
- CacheSource
- CacheTile