diff --git a/doc/dev/css.rst b/doc/dev/css.rst deleted file mode 100644 index c8bf1a19c0..0000000000 --- a/doc/dev/css.rst +++ /dev/null @@ -1,68 +0,0 @@ -################### -CSS metadata layout -################### - -The Central State System (CSS) maintains information about the data stored in -Qserv. Currently, this consists of the data schema: the names of databases and -what tables they contain, the partitioning parameters used by the database, etc. - -************** -Implementation -************** - -CSS content is stored in a hierarchical key-value structure, with keys -structured much like filename paths. In standard Qserv installations, CSS data -is kept in a MySQL database, although parts of qserv may be tested with CSS -content stored in a more feature-light form, e.g., a file. - -*********** -Packed keys -*********** - -In order to reduce the number of key-value updates when manipulating CSS -content, some portions of the content tree are stored packed in JSON format. -This is signified by presence of ".packed.json" key. The content of this key is -a structure (JSON object) which contains some of the keys appearing at the same -level as ".packed.json" key. For example, suppose CSS content includes -(specified in python dict syntax): - -.. code-block:: python - - { "/foo/bar" : "12345", - "/foo/bar/baz" : "regular data", - "/foo/bar/.packed.json" : '{"a":"aa", "b":"bb"}' } - -The key "/foo/bar/.packed.json" is unpacked, thus the above content will be -interpreted as: - -.. code-block:: python - - { "/foo/bar" : "12345", - "/foo/bar/baz" : "regular data", - "/foo/bar/a" : "aa", - "/foo/bar/b" : "bb" } - -Note that the presence of "/foo/bar/.packed.json" does not prevent the presence -of "/foo/bar/baz". It is not specified what happens if the same key appears in -both regular CSS structure and in packed key value like in this structure: - -.. code-block:: python - - # it is unspecified which value /foo/bar/a has when unpacked - { "/foo/bar" : "12345", - "/foo/bar/a" : "regular key data", - "/foo/bar/.packed.json" : '{"a":"aa", "b":"bb"}' } - -The values in JSON object can be simple types like strings or numbers, complex -objects are not supported there. All values in JSON object are transformed to -strings when unpacked, so the packed object ``"{"key": 1}"`` is treated -identically to ``"{"key": "1"}"``. - -************* -CSS interface -************* - -CSS key-value storage is hidden completely behind CSS interface -(``css/CssAccess`` class). This interface defines all logical operations on -databases, tables, nodes, etc. and translates those operations into key-value -structure manipulatons, including packing and upacking of the keys. diff --git a/doc/dev/index.rst b/doc/dev/index.rst index 1704c53fae..0fc6b0cc50 100644 --- a/doc/dev/index.rst +++ b/doc/dev/index.rst @@ -1,8 +1,3 @@ -.. warning:: - - **Information in this guide is known to be outdated.** A documentation sprint is underway which will - include updates and revisions to this guide. - .. highlight:: sql ################# @@ -14,5 +9,4 @@ Developer's Guide quick-start-devel doc - css - wmgr-api + scisql diff --git a/doc/dev/quick-start-devel.rst b/doc/dev/quick-start-devel.rst index 4b0cd7b345..1bfdaabbe6 100644 --- a/doc/dev/quick-start-devel.rst +++ b/doc/dev/quick-start-devel.rst @@ -1,3 +1,7 @@ +.. warning:: + + Information in this guide is known to be outdated. The updated version will be available in the near future. + .. _quick-start-devel: ################################ diff --git a/doc/dev/scisql.rst b/doc/dev/scisql.rst new file mode 100644 index 0000000000..d533bd2c43 --- /dev/null +++ b/doc/dev/scisql.rst @@ -0,0 +1,113 @@ +.. note:: + The oficial documentation for the ``sciSQL`` project can be found at https://smonkewitz.github.io/scisql/ + +====================== +sciSQL Developer Notes +====================== + +The document presents the development methodology for ``sciSQL`` in the context of the Qserv container environment. + +Making a build container +------------------------ + +The most straight-forward way to do this involves starting with a generic MariaDB distribution container, +then layering in the development toolchain. Build and test then take place within this customized container. +Here is an example of a small ``Dockerfile`` for this: + +.. code-block:: dockerfile + + FROM mariadb:10.6 + RUN apt-get update \ + && apt-get install -y g++ git make libmariadb-dev python3 python3-pip vim \ + && pip3 install future mako mysqlclient \ + && update-alternatives --install /usr/bin/python python /usr/bin/python3 0 + ENV MARIADB_ROOT_PASSWORD=CHANGEME + VOLUME /root + +With that file the desired image is built by: + +.. code-block:: bash + + docker build -t scisql-dev - < Dockerfile + +The ``ENV`` and ``VOLUME`` lines of the ``Dockerfile`` file are for convenience when running the resulting container. +The stock MariaDB container already has a ``VOLUME`` for ``/var/lib/sql``. Passing this and the additional ``VOLUME`` +for ``/root`` conveniently captures your development state in case the container crashes or you otherwise wish to restart it. + +Running the container +--------------------- + +To run the container built as above: + +.. code-block:: bash + + docker run --name scisql-dev --init -d \ + -v scisql-dev-data:/var/lib/mysql \ + -v scisql-dev-home:/root scisql-dev + +You need to provide names for volumes holding MariaDB state and the root user home directory. This exact same command can be +repeated to re-launch with preserved state (providing, e.g., you check out and build under ``/root`` within the container). + +Now the container will start. If it is a first run on a given data volume, it will take some tens of seconds for MariaDB +to initialize. You can monitor docker logs on the container. When it is ready to go you will see "ready for connections" +near the end of the log. There will be a running MariaDB server within this container, into which scisql can be installed +and tested. + +At this point, it's recommended using a tool like ``VSCode``'s ``"connect to running container"`` to attach +the IDE to the container. It can take ``VSCode`` a little while to download and install its server-side support within +the container (another nice reason to have this persisted in the ``/root`` volume). You may wish to install a few niceties +like your favorite ``.profile``, ssh keys for GitHub, etc. now in ``/root``. + +Building and testing sciSQL +--------------------------- + +Now, inside the container, clone out from ``github.com/smonkewitz/scisql``. Configure and build with: + +.. code-block:: bash + + git clone https://github.com/smonkewitz/scisql.git + cd scisql + ./configure --mysql-includes=/usr/include/mariadb + make + +From here, the somewhat non-obvious iterated incantation to rebuild, deploy into the local MariaDB, and run the test +suite is: + +.. code-block:: bash + + make install && echo $MARIADB_ROOT_PASSWORD | PYTHONPATH=/usr/local/python \ + scisql-deploy.py --mysql-dir=/usr \ + --mysql-socket=/run/mysqld/mysqld.sock \ + --mysql-plugin-dir=/lib/mysql/plugin + +If you don't want to undeploy/redeploy the UDFs and plugin, but are just iterating on the unit tests themselves, +the following shortcut version is also useful: + +.. code-block:: bash + + make install && echo $MARIADB_ROOT_PASSWORD | PYTHONPATH=/usr/local/python \ + scisql-deploy.py --mysql-dir=/usr \ + --mysql-socket=/run/mysqld/mysqld.sock \ + --mysql-plugin-dir=/lib/mysql/plugin \ + --test + +Updating the HTML documentation +------------------------------- + +The HTML documentation is rendered by the Mako template library for Python: https://www.makotemplates.org from comments +embedded in the sources and templates in ``tools/templates/``. Incantation to build is just: + +.. code-block:: bash + + make html_docs + +If you are using ``VSCode``, you can get a tunneled live view on the documentation in your working tree by popping +an additional terminal, and incanting: + +.. code-block:: bash + + cd doc + exec python -m http.server + +Don't forget to add and commit the re-rendered documentation (under ``doc/``) on your PR. After a PR is merged to master, +the documentation will automatically update on github pages at https://smonkewitz.github.io/scisql/ diff --git a/doc/dev/wmgr-api.rst b/doc/dev/wmgr-api.rst deleted file mode 100644 index 27e6cbab62..0000000000 --- a/doc/dev/wmgr-api.rst +++ /dev/null @@ -1,1092 +0,0 @@ -Worker Manager API -################## - -Overview -******** - -Worker manager is a service which runs alongside every qserv worker or czar and -manages many common operations: - -* database operations (table creating, data loading, etc.) -* certain xrootd (plug-in) operations -* management of other services' lifetime - -Primary motivation for implementing wmgr service was to facilitate data loading -in integration tests. It is likely that some of the wmgr functions may be -changed in the future once we implement production-level services for data -loading and distribution. For more details on wmgr design consult pages: - -* https://jira.lsstcorp.org/browse/DM-1900 -* https://dev.lsstcorp.org/trac/wiki/db/Qserv/WMGRDesign - -This document describes wmgr HTTP-basedAPI in details. Note that there is also a -Python client library implemented on top of HTTP API. - - -General API description -*********************** - -Wmgr API is implemented on top of HTTP protocol following RESTful principles -(where possible). All communication with wmgr is performed over HTTP without -encryption (implementing SSL/TLS is feasible but needs solution for certificate -management). - -Wmgr can be configured to require one of few authentication mechanisms - none, -basic, or digest. Lack of SSL/TLS dictates use digest authentication for -production services. - -Whole wmgr API is split into three groups - database, xrootd, and process API. -Below is description of individual API methods (actions). - -Return codes and content types -============================== - -Responses generated by wmgr service use regular HTTP status codes to indicate -success or error. Codes in range 200-299 are used for success, 400-499 for -errors (range 500-599 typically means service failure). - -Normally responses generated by wmgr service include data in JSON format and -have their ``Content-Type`` header set to ``application/json``. This applies to -both successful completion and error conditions. In some cases when error -condition is returned by the transport/framework layer it will have different -content type. - -See also document which defines general structure of JSON objects for returned -response data: -https://confluence.lsstcorp.org/display/DM/REST+API+General+Guidelines - - -Database API -************ - -This section contains description of actions related to database operations - -creating/deleting databases and tables, loading table data, etc. - - -``GET /dbs`` -============ - - Return the list of all existing database names. - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for successful return - * 500 - for database errors - - Response body (for successful completion): - JSON object, "results" property is a list of database objects with keys: - * name: database name - * uri: URL for database operations - - Request/response example:: - - GET /dbs HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "results": [ - { - "name": "qservMeta", - "uri": "/dbs/qservMeta" - }, - { - "name": "qservResult", - "uri": "/dbs/qservResult" - } - ] - } - -``POST /dbs`` -============= - - Create new database. In addition to creating database itself this method - also grants all privileges on this database to regular non-privileged - account. - - Request headers: - * Content-Type: required as ``multipart/form-data`` - - Form Parameters: - * db: Database name - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 201 - if database was successfully created - * 400 - if database name is missing - * 409 - if database already exists - * 500 - for other database errors - - Response body (for successful completion): - JSON object, "result" property is a database object with keys: - * name: database name - * uri: URL for database operations - - Request/response example:: - - POST /dbs HTTP/1.0 - Content-Type: multipart/form-data; boundary=------------------------bb306714c15713c2 - - --------------------------bb306714c15713c2 - Content-Disposition: form-data; name="db" - - newDB - --------------------------bb306714c15713c2-- - - :: - - HTTP/1.0 201 CREATED - Content-Type: application/json - - { - "result": { - "name": "newDB", - "uri": "/dbs/newDB" - } - } - -``DELETE /dbs/`` -======================== - - Deletes database. - - Parameters: - * dbName: database name - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - if database was successfully deleted - * 400 - if parameters have invalid format - * 404 - if database does not exist - * 500 - for other database errors - - Response body (for successful completion): - JSON object, "result" property is a database object with keys: - * name: database name - * uri: URL for database operations - - Request/response example:: - - DELETE /dbs/newDB HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "name": "newDB", - "uri": "/dbs/newDB" - } - } - -``GET /dbs//tables`` -============================ - - Returns the list of tables in a database. - - Parameters: - * dbName: database name - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for successful return - * 400 - if parameters have invalid format - * 404 - if database does not exist - * 500 - for database errors - - Response body (for successful completion): - JSON object, "results" property is a list of table objects with keys: - * name: table name - * uri: URL for database operations - - Request/response example:: - - GET /dbs/qservMeta/tables HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "results": [ - { - "name": "QCzar", - "uri": "/dbs/qservMeta/tables/QCzar" - }, - { - "name": "QInfo", - "uri": "/dbs/qservMeta/tables/QInfo" - }, - ... - ] - } - -``POST /dbs//tables`` -============================= - - Create new table. - - If ``schemaSource`` (see below) is "request" then request must include - ``schema`` parameter which is an SQL DDL statement starting with 'CREATE - TABLE TableName ...'. - - If ``schemaSource`` is "css" then ``table`` parameter must be specified. - Table schema will be extracted from CSS in this case, ``schemaSource`` must - not be given. - - Parameters: - * dbName: database name - - Request headers: - * Content-Type: required as ``multipart/form-data`` - - Form Parameters: - * table: Table name - * schemaSource: source for schema name, possible - values: "request", "css", (default: "request") - * schema: complete "CREATE TABLE ..." statement - (optional) - * chunkColumns: boolean flag, false by default, - accepted values: '0', '1', 'yes', 'no', 'false', 'true'. If set - to true then delete columns "_chunkId", "_subChunkId" from table - (if they exist) and add columns "chunkId", "subChunkId" (if they - don't exist) - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 201 - if table was successfully created - * 400 - if parameters have invalid format or if form - parameters are missing or conflicting - * 409 - if table already exists - * 500 - if table is not defined in CSS or other - database errors - - Response body (for successful completion): - JSON object, "result" property is a table object with keys: - * name: database name - * uri: URL for database operations - - Request/response example:: - - POST /dbs/newDB/tables HTTP/1.0 - Content-Type: multipart/form-data; boundary=------------------------c5c44964f0f9add0 - - --------------------------c5c44964f0f9add0 - Content-Disposition: form-data; name="schema" - - CREATE TABLE newTable (I INT) - --------------------------c5c44964f0f9add0 - Content-Disposition: form-data; name="table" - - newTable - --------------------------c5c44964f0f9add0-- - - :: - - HTTP/1.0 201 CREATED - Content-Type: application/json - - { - "result": { - "name": "newTable", - "uri": "/dbs/newDB/tables/newTable" - } - } - -``DELETE /dbs//tables/`` -========================================= - - Drop a table and optionally all chunk/overlap tables. - - Parameters: - * dbName: database name - * tblName: table name - - Query Parameters: - * dropChunks: boolean flag, false by default, accepted - values: '0', '1', 'yes', 'no', 'false', 'true' - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - if table was successfully deleted - * 400 - if parameters have invalid format - * 404 - if table does not exist - * 500 - for other database errors - - Response body (for successful completion): - JSON object, "result" property is a table object with keys: - * name: database name - * uri: URL for database operations - - Request/response example:: - - DELETE /dbs/newDB/tables/newTable HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "name": "newTable", - "uri": "/dbs/newDB/tables/newTable" - } - } - - -``GET /dbs//tables//schema`` -============================================= - - Return result of SHOW CREATE TABLE statement for given table. - - Parameters: - * dbName: database name - * tblName: table name - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for successful return - * 400 - if parameters have invalid format - * 404 - if table does not exist - * 500 - for database errors - - Response body (for successful completion): - JSON object, "result" property is a string with resulting schema. - * name: table name - * uri: URL for database operations - - Request/response example:: - - GET /dbs/newDB/tables/newTable/schema HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": "CREATE TABLE `newTable` (\n `I` int(11) DEFAULT NULL\n) ENGINE=MyISAM DEFAULT CHARSET=latin1" - } - -``GET /dbs//tables//columns`` -============================================== - - Return result of SHOW COLUMNS statement for given table. - - Parameters: - * dbName: database name - * tblName: table name - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for successful return - * 400 - if parameters have invalid format - * 404 - if table does not exist - * 500 - for database errors - - Response body (for successful completion): - JSON object, "results" property is a list of column - objects with keys: name, type, key, default, null - - Request/response example:: - - GET /dbs/newDB/tables/newTable/columns HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "results": [ - { - "default": null, - "key": "", - "name": "I", - "null": "YES", - "type": "int(11)" - } - ] - } - -``GET /dbs//tables//chunks`` -============================================= - - Return the list of chunks in a table. For non-chunked table empty list - is returned. - - Parameters: - * dbName: database name - * tblName: table name - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for successful return - * 400 - if parameters have invalid format - * 404 - if table does not exist - * 500 - for database errors - - Response body (for successful completion): - JSON object, "results" property is a list of chunk objects with keys: - * chunkId: chunk number (integer) - * chunkTable: true if chunk has regular chunk - table (boolean) - * overlapTable: true if chunk has overlap - table (boolean) - * uri: URL for chunk operations - - Request/response example:: - - GET /dbs/qservTest_case01_qserv/tables/Object/chunks HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "results": [ - { - "chunkId": 7648, - "chunkTable": true, - "overlapTable": true, - "uri": "/dbs/qservTest_case01_qserv/tables/Object/chunks/7648" - }, - ... - ] - } - -``POST /dbs//tables//chunks`` -============================================== - - Create new chunk. - - Parameters: - * dbName: database name - * tblName: table name - - Request headers: - * Content-Type: required as ``multipart/form-data`` - - Form Parameters: - * chunkId: chunk ID, non-negative integer - * overlapFlag: if true then create overlap table too - (default is true), accepted values: '0', '1', 'yes', 'no', - 'false', 'true' - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 201 - if chunk tables were successfully created - * 400 - if parameters have invalid format or if form - parameters are missing or conflicting - * 404 - if table is missing - * 409 - if chunk table already exists - * 500 - if table is not defined in CSS or other - database errors - - Response body (for successful completion): - JSON object, "result" property is a chunk object with keys: - * chunkId: chunk number (integer) - * chunkTable: true if chunk has regular chunk - table (boolean) - * overlapTable: true if chunk has overlap - table (boolean) - * uri: URL for chunk operations - - Request/response example:: - - POST /dbs/newDB/tables/newTable/chunks HTTP/1.0 - Content-Type: multipart/form-data; boundary=------------------------df029da2ec8387ce - - --------------------------df029da2ec8387ce - Content-Disposition: form-data; name="chunkId" - - 1000 - --------------------------df029da2ec8387ce-- - - :: - - HTTP/1.0 201 CREATED - Content-Type: application/json - - { - "result": { - "chunkId": 1000, - "chunkTable": true, - "overlapTable": true, - "uri": "/dbs/newDB/tables/newTable/chunks/1000" - } - } - -``DELETE /dbs//tables//chunks/`` -========================================================== - - Delete chunk from a table, both chunk data and overlap data is dropped. - - Parameters: - * dbName: database name - * tblName: table name - * chunkId: chunk number, non-negative integer - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - if table was successfully deleted - * 400 - if parameters have invalid format - * 404 - if table does not exist - * 500 - for other database errors - - Response body (for successful completion): - JSON object, "result" property is a chunk object with keys: - * chunkId: chunk number (integer) - * chunkTable: true if chunk has regular chunk - table (boolean) - * overlapTable: true if chunk has overlap - table (boolean) - * uri: URL for chunk operations - - Request/response example:: - - DELETE /dbs/newDB/tables/newTable/chunks/1000 HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "chunkId": 1000, - "chunkTable": true, - "overlapTable": true, - "uri": "/dbs/newDB/tables/newTable/chunks/1000" - } - } - -``POST //tables//data`` -======================================== - - Upload data into a table using file format supported by mysql command - LOAD DATA [LOCAL] INFILE. - - Parameters: - * dbName: database name - * tblName: table name - - Request headers: - * Content-Type: required as ``multipart/form-data`` - - Form Parameters: - * table-data: the data come in original LOAD DATA - format with ``binary/octet-stream`` content type and binary - encoding, and it may be compressed with gzip. - * load-options: set of options encoded with usual - ``application/x-www-form-urlencoded`` content type, options are: - - delimiter - defaults to TAB - - enclose - defaults to empty string (strings are not enclosed) - - escape - defaults to backslash - - terminate - defaults to newline - - compressed - "0" or "1", by default is guessed from file extension (.gz) - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 201 - if chunk tables were successfully created - * 400 - if parameters have invalid format or if form - parameters are missing or conflicting - * 404 - if table is missing - * 409 - if chunk table already exists - * 500 - if table is not defined in CSS or other - database errors - - Response body (for successful completion): - JSON object, "result" property is an object with keys: - * status: string "OK" - * count: count of rows added to a table - - Request/response example:: - - POST /dbs/newDB/tables/newTable/data HTTP/1.0 - Content-Type: multipart/form-data; boundary=------------------------345ad77805210ac6 - - --------------------------345ad77805210ac6 - Content-Disposition: form-data; name="table-data"; filename="table.dat.gz" - Content-Type: application/octet-stream - - .....<.U..table.dat.3.2400.2.bS..;....... - - --------------------------345ad77805210ac6 - Content-Disposition: form-data; name="load-options" - - compressed=1&delimiter=%2C - --------------------------345ad77805210ac6-- - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "count": 4, - "status": "OK" - } - } - -``POST //tables//chunks//data`` -========================================================= - - Upload data into a chunk table using file format supported by mysql - command LOAD DATA [LOCAL] INFILE. - - This method works exactly as previous one taking the same form parameter - but it loads data into a chunk and has additional URL parameter specifying - chunk number. - -``POST //tables//chunks//overlap`` -============================================================ - - Upload data into overlap table using file format supported by mysql - command LOAD DATA [LOCAL] INFILE. - - This method works exactly as previous one taking the same form parameter - but it loads data into an overlap table and has additional URL parameter - specifying chunk number. - -``GET /dbs//tables//index`` -============================================ - - Return index data (array of (objectId, chunkId, subChunkId) triplets). - - Parameters: - * dbName: database name - * tblName: table name - * chunkId: chunk number (non-negative integer) - - Query Parameters: - * columns: specifies comma-separated list of three - column names. Default column names are "objectId", "chunkId", - "subChunkId". Result returns columns in the same order as they - are specified in 'columns' argument. - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for successful return - * 400 - if parameters have invalid format - * 404 - if table does not exist - * 500 - for other database errors - - Response body (for successful completion): - JSON object, "result" property is an object with keys: - * description: array of three objects - describing columns, each with keys "name" (column name) and - "type" (MySQL type name) - * rows: array of arrays of integers - - Request/response example:: - - GET /dbs/qservTest_case01_qserv/tables/Object/index HTTP/1.1 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "description": [ - { - "name": "objectId", - "type": "LONGLONG" - }, - { - "name": "chunkId", - "type": "LONG" - }, - { - "name": "subChunkId", - "type": "LONG" - } - ], - "rows": [ - [ - 386937898687249, - 6630, - 897 - ], - [ - 386942193651348, - 6630, - 660 - ], - ... - ] - } - } - -``GET /dbs//tables//chunks//index`` -============================================================= - - Return index data (array of (objectId, chunkId, subChunkId) triplets) - for single chunk. - - Does the same as previous method but for one chunk from partitioned - table. Useful when index for whole table may be too big. - - Request/response example:: - - GET /dbs/qservTest_case01_qserv/tables/Object/chunks/7648/index HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "description": [ - { - "name": "objectId", - "type": "LONGLONG" - }, - { - "name": "chunkId", - "type": "LONG" - }, - { - "name": "subChunkId", - "type": "LONG" - } - ], - "rows": [ - [ - 433306365599363, - 7648, - 5 - ], - [ - 433314955527561, - 7648, - 10 - ], - ... - ] - } - } - - -Xrootd API -********** - -This section contains description of actions related to xrootd operations - e.g. -publishing database via xrootd. - - -``GET /xrootd/dbs`` -=================== - - Return the list of databases known to xrootd. - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for success - * 500 - for other database errors - - Response body (for successful completion): - JSON object, "results" property is a list of database objects with keys: - * name: database name - * uri: URL for *xrootd* database operations - - Request/response example:: - - GET /xrootd/dbs HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "results": [ - { - "name": "qservTest_case01_qserv", - "uri": "/xrootd/dbs/qservTest_case01_qserv" - } - ] - } - -``POST /xrootd/dbs`` -==================== - - Register new database in xrootd chunk inventory. - - Request headers: - * Content-Type: required as ``multipart/form-data`` - - Form Parameters: - * db: database name (required) - * xrootdRestart: if set to 'no' then do not restart - xrootd (defaults to yes) - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 201 - if database was successfully registered - * 400 - if parameters are missing or have invalid - format - * 409 - if database is already registered - * 500 - on other database errors - - Response body (for successful completion): - JSON object, "results" property is a database object with keys: - * name: database name - * uri: URL for *xrootd* database operations - - Request/response example:: - - POST /xrootd/dbs HTTP/1.0 - Content-Type: multipart/form-data; boundary=------------------------370e6e4d60b7499e - - --------------------------370e6e4d60b7499e - Content-Disposition: form-data; name="db" - - newDB - --------------------------370e6e4d60b7499e - Content-Disposition: form-data; name="xrootdRestart" - - no - --------------------------370e6e4d60b7499e-- - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "name": "newDB", - "uri": "/xrootd/dbs/newDB" - } - } - -``DELETE /xrootd/dbs/`` -=============================== - - Unregister database from xrootd chunk inventory. - - Parameters: - * dbName: database name - - Query Parameters: - * xrootdRestart: if set to 'no' then do not restart - xrootd (defaults to yes) - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for success - * 400 - if parameters have invalid format - * 409 - if database is not registered - * 500 - for other database errors - - Response body (for successful completion): - JSON object, "results" property is a database object with keys: - * name: database name - * uri: URL for *xrootd* database operations - - Request/response example:: - - DELETE /xrootd/dbs/newDB?xrootdRestart=no HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "name": "newDB", - "uri": "/xrootd/dbs/newDB" - } - } - -``GET /xrootd/dbs/`` -============================ - - Return the list of chunk IDs in a database as known to xrootd. - - .. note:: Not implemented yet. - - -Services API -************ - -This section contains description of actions related to operations on services - -e.g. stopping and starting processes. - - -``GET /services`` -================= - - Return the list of services. - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for success - - Response body (for successful completion): - JSON object, "results" property is a list of service objects with keys: - * name: service name - * uri: URL for service operations - - Request/response example:: - - GET /services HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "results": [ - { - "name": "xrootd", - "uri": "/services/xrootd" - }, - { - "name": "mysqld", - "uri": "/services/mysqld" - } - ] - } - -``GET /services/`` -=========================== - - Return service state. - - Parameters: - * service: service name - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for success - * 404 - for invalid service name - - Response body (for successful completion): - JSON object, "result" property is a service object with keys: - * name: service name - * state: one of "active" or "stopped" - * uri: URL for service operations - - Request/response example:: - - GET /services/mysqld HTTP/1.0 - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "name": "mysqld", - "state": "active", - "uri": "/services/mysqld" - } - } - -``PUT /services/`` -=========================== - - Execute some action on service, like "stop" or "restart". - - Parameters: - * service: service name - - Request headers: - * Content-Type: required as ``multipart/form-data`` - - Form Parameters: - * action: action: one of 'stop', 'start', 'restart' - (required) - - Response headers: - * Content-Type: ``application/json`` - - Status Codes: - * 200 - for success - * 400 - if parameters are missing or have invalid - format - * 409 - if action has failed - - Response body (for successful completion): - JSON object, "result" property is a service object with keys: - * name: service name - * state: state of service after action, one of - "active" or "stopped" - * uri: URL for service operations - - Request/response example:: - - PUT /services/mysqld HTTP/1.0 - Content-Type: multipart/form-data; boundary=------------------------48169e483bc7d12e - - --------------------------48169e483bc7d12e - Content-Disposition: form-data; name="action" - - restart - --------------------------48169e483bc7d12e-- - - :: - - HTTP/1.0 200 OK - Content-Type: application/json - - { - "result": { - "name": "mysqld", - "state": "active", - "uri": "/services/mysqld" - } - }