From da18711932b6dabcba41ca29f904dedba03ce5c1 Mon Sep 17 00:00:00 2001 From: aksm Date: Wed, 14 Dec 2022 10:00:03 -0500 Subject: [PATCH] Some stray spacing cleanup and broken admonitions fixes. --- docs/find_and_replace_action_json_patch.md | 370 ++++++++++----------- 1 file changed, 185 insertions(+), 185 deletions(-) diff --git a/docs/find_and_replace_action_json_patch.md b/docs/find_and_replace_action_json_patch.md index ddc8ccf0..86ab0ce4 100644 --- a/docs/find_and_replace_action_json_patch.md +++ b/docs/find_and_replace_action_json_patch.md @@ -29,7 +29,7 @@ Before we dive into the mechanics of doing JSON Patching Batch in Archipelago we A JSON Patch is a `JSON` Document containing precise operations that can heavily modify the structure and values of an existing JSON Document, in this case the RAW JSON found inside a strawberryfield of our ADOs. -The operations available for modifications of an JSON document are: +The operations available for modifications of an JSON document are: - “add” - “remove” @@ -78,7 +78,7 @@ These JSON Pointers will resolve in the following manner: } ``` -- `/subject_lcgft_terms/0/label` +- `/subject_lcgft_terms/0/label` ``` Photographs @@ -94,7 +94,7 @@ AS you can see a JSON Pointer is very precise , allowing you to target complete ] ``` -and you want to "test" for the existence of `"term3"` before applying a change, you would need to know exactly at what position inside the `terms Array` (Starting from 0) it will/should be found. And that might not be consistent across every ADO. +and you want to "test" for the existence of `"term3"` before applying a change, you would need to know exactly at what position inside the `terms Array` (Starting from 0) it will/should be found. And that might not be consistent across every ADO. So how do we use these pointers in an operation inside a JSON Patch document? Using the same fake "terms" JSON snippet the previously listed, operations are written like this: @@ -116,14 +116,14 @@ This will add before the first term (in this case "term1" ) "term_another" as a { "op": "remove", "path": "/terms/1"} ``` -This will remove the second term (in this case "term2" ) +This will remove the second term (in this case "term2" ) ## replace ```JSON { "op": "replace", "path": "/terms/1", "value": "term_again" } ``` -This will remove the second term (in this case "term2" ) and put in its place "term_again" as a value. Basically two operationes, “remove“ and “add “ in a single step +This will remove the second term (in this case "term2" ) and put in its place "term_again" as a value. Basically two operationes, “remove“ and “add “ in a single step ## move @@ -131,7 +131,7 @@ This will remove the second term (in this case "term2" ) and put in its place " { "op": "move", "from": "/terms", "path": "/terms_somewhere_else"} ``` -This will copy all values inside top JSON key `terms` into a new top JSON key named `terms_somewhere_else` and remove then the old `terms` key. +This will copy all values inside top JSON key `terms` into a new top JSON key named `terms_somewhere_else` and remove then the old `terms` key. ## copy @@ -157,7 +157,7 @@ So. When to use JSON Patch? There are a few general rules/suggestions: - When you need to use data that is already inside the JSON Document and move it around. Sometimes `fixing` a value, in the sense of putting a static replacement, is not what you need. Other `Actions` [documented](find_and_replace.md) will allow you to replace one value with another fixed one. But JSON Patch will allow you to use existing data inside your target JSON and move it/copy it. - When you operate on multiple JSON Keys and can target / match complete values. You can not replace a single word inside a value via a JSON Patch. But you can apply multiple precise operations in a single pass. - When you need to change data types. e.g. convert a single value into an array. -- When you need safety. The fact that a single failed “test“ will cancel a whole JSON Patch allows you to operate with safety and ensure the final destination will always be a JSON +- When you need safety. The fact that a single failed “test“ will cancel a whole JSON Patch allows you to operate with safety and ensure the final destination will always be a JSON Now that we know what it is and when we should do it, we can make a small exercise. The goal: @@ -166,11 +166,11 @@ Now that we know what it is and when we should do it, we can make a small exerci - add an extra LOD entry - Copy a Value into a new Key -## Step 1: Select the ADOs to be Modified +## Step 1: Select the ADOs to be Modified As with every other VBO action described in our documentation, start by selecting the ADOs you want to modify using the exposed Search Field(s) and/or Facets present in the Search and Replace `View` found at `/search-and-replace` . -Once you have filtered down the list to manageable size, containing at least the ADOs you plan on modifying (but for JSON Patch operation could be more too because you can “test“ and match ), press either on `Select / deselect all results in this view` to pass all the result (this includes all pages, not only the currently visible one) or go selectively one by one by checking on the `toggle` found beside each ADO's title. You will see how the number in `Selected 0 item in this view` increases. Now press `Apply to select Items`. We will use for this example `Laddie the dog running in the garden, Bronx, N.Y., undated [c. 1910-1918?]`, an ADO we provide in our DEMO sets. +Once you have filtered down the list to manageable size, containing at least the ADOs you plan on modifying (but for JSON Patch operation could be more too because you can “test“ and match ), press either on `Select / deselect all results in this view` to pass all the result (this includes all pages, not only the currently visible one) or go selectively one by one by checking on the `toggle` found beside each ADO's title. You will see how the number in `Selected 0 item in this view` increases. Now press `Apply to select Items`. We will use for this example `Laddie the dog running in the garden, Bronx, N.Y., undated [c. 1910-1918?]`, an ADO we provide in our DEMO sets. Redundant to say, Batch Actions are intended to be used when a modification needs to be applied to more than one item, implying there is a pattern. For single ADOs you can always do this faster directly via the **EDIT** tab. @@ -190,187 +190,187 @@ A bit of repetition but needed to explain the JSON Patch document. You can see o ??? info "RAW JSON of `Laddie the dog running in the garden, Bronx, N.Y., undated [c. 1910-1918?]`" -```JSON title="Laddie the dog running in the garden, Bronx, N.Y., undated [c. 1910-1918?] | before Patching" -{ - "note": "", - "type": "Photograph", - "viaf": "", - "label": "Laddie the dog running in the garden, Bronx, N.Y., undated [c. 1910-1918?]", - "model": [], - "owner": "New-York Historical Society, 170 Central Park West, New York, NY 10024, 212-873-3400.", - "audios": [], - "images": [ - 26 - ], - "models": [], - "rights": "This digital image may be used for educational or scholarly purposes without restriction. Commercial and other uses of the item are prohibited without prior written permission from the New-York Historical Society. For more information, please visit the New-York Historical Society's Rights and Reproductions Department web page at [http:\/\/www.nyhistory.org\/about\/rights-reproductions](http:\/\/www.nyhistory.org\/about\/rights-reproductions).", - "videos": [], - "creator": "", - "ap:tasks": { - "ap:sortfiles": "index" - }, - "duration": "", - "ispartof": [], - "language": "English", - "documents": [], - "edm_agent": "", - "publisher": "", - "ismemberof": [ - 16 - ], - "creator_lod": [ - { - "name_uri": "", - "role_uri": "http:\/\/id.loc.gov\/vocabulary\/relators\/pht", - "agent_type": "personal", - "name_label": "Stonebridge, George Ehler", - "role_label": "Photographer" - } - ], - "description": "George Ehler Stonebridge (d. 1941) was an amateur photographer who lived and worked in the Bronx, New York. He left little record of himself, but an invaluable one of his surroundings and interests. Stonebridge lived at several locations in the Bronx with his wife Bella, and their three children Grace, George, and William. He worked at the Northern Gaslight Company, although the position he held is unknown. In addition to taking photographs, Stonebridge wrote poetry and prose about his love of the Bronx, his children, and in honor of military victories. Some of Stonebridge's photographs appeared in local papers. In 1898, he was an authorized reporter and photographer for the North Side News; in 1905 he was an authorized reporter for the Bronx Borough Record and Times, and probably took photographs for that paper as well. Stonebridge was fascinated with the subject of military preparedness. Training rituals and staged battles were one of his favorite photographic subjects. His 1898 poem, \"Remember the Maine,\" celebrates the United States' victory in the Spanish-American War. He was especially proud of soldiers from the Bronx, and photographed historical tablets throughout the Borough commemorating previous military victories. Stonebridge also used his photographs to illustrate lectures. In 1907, he gave several lectures on \"The Training of War,\" using colored lantern slides.", - "interviewee": "", - "interviewer": "", - "pubmed_mesh": null, - "sequence_id": "", - "subject_loc": [ - { - "uri": "http:\/\/id.loc.gov\/authorities\/subjects\/sh85038796", - "label": "Dogs" - } - ], - "website_url": "", - "as:generator": { - "type": "Update", - "actor": { - "url": "http:\/\/localhost:8001\/form\/descriptive-metadata", - "name": "descriptive_metadata", - "type": "Service" + ```JSON title="Laddie the dog running in the garden, Bronx, N.Y., undated [c. 1910-1918?] | before Patching" + { + "note": "", + "type": "Photograph", + "viaf": "", + "label": "Laddie the dog running in the garden, Bronx, N.Y., undated [c. 1910-1918?]", + "model": [], + "owner": "New-York Historical Society, 170 Central Park West, New York, NY 10024, 212-873-3400.", + "audios": [], + "images": [ + 26 + ], + "models": [], + "rights": "This digital image may be used for educational or scholarly purposes without restriction. Commercial and other uses of the item are prohibited without prior written permission from the New-York Historical Society. For more information, please visit the New-York Historical Society's Rights and Reproductions Department web page at [http:\/\/www.nyhistory.org\/about\/rights-reproductions](http:\/\/www.nyhistory.org\/about\/rights-reproductions).", + "videos": [], + "creator": "", + "ap:tasks": { + "ap:sortfiles": "index" }, - "endTime": "2022-12-05T09:19:37-05:00", - "summary": "Generator", - "@context": "https:\/\/www.w3.org\/ns\/activitystreams" - }, - "date_created": "1910-01-01", - "issue_number": null, - "date_published": "", - "subjects_local": null, - "term_aat_getty": "", - "ap:entitymapping": { - "entity:file": [ - "model", - "audios", - "images", - "videos", - "documents", - "upload_associated_warcs" + "duration": "", + "ispartof": [], + "language": "English", + "documents": [], + "edm_agent": "", + "publisher": "", + "ismemberof": [ + 16 ], - "entity:node": [ - "ispartof", - "ismemberof" - ] - }, - "europeana_agents": "", - "europeana_places": "", - "local_identifier": "nyhs_PR066_6136", - "subject_wikidata": [ - { - "uri": "http:\/\/www.wikidata.org\/entity\/Q3381576", - "label": "black-and-white photography" + "creator_lod": [ + { + "name_uri": "", + "role_uri": "http:\/\/id.loc.gov\/vocabulary\/relators\/pht", + "agent_type": "personal", + "name_label": "Stonebridge, George Ehler", + "role_label": "Photographer" + } + ], + "description": "George Ehler Stonebridge (d. 1941) was an amateur photographer who lived and worked in the Bronx, New York. He left little record of himself, but an invaluable one of his surroundings and interests. Stonebridge lived at several locations in the Bronx with his wife Bella, and their three children Grace, George, and William. He worked at the Northern Gaslight Company, although the position he held is unknown. In addition to taking photographs, Stonebridge wrote poetry and prose about his love of the Bronx, his children, and in honor of military victories. Some of Stonebridge's photographs appeared in local papers. In 1898, he was an authorized reporter and photographer for the North Side News; in 1905 he was an authorized reporter for the Bronx Borough Record and Times, and probably took photographs for that paper as well. Stonebridge was fascinated with the subject of military preparedness. Training rituals and staged battles were one of his favorite photographic subjects. His 1898 poem, \"Remember the Maine,\" celebrates the United States' victory in the Spanish-American War. He was especially proud of soldiers from the Bronx, and photographed historical tablets throughout the Borough commemorating previous military victories. Stonebridge also used his photographs to illustrate lectures. In 1907, he gave several lectures on \"The Training of War,\" using colored lantern slides.", + "interviewee": "", + "interviewer": "", + "pubmed_mesh": null, + "sequence_id": "", + "subject_loc": [ + { + "uri": "http:\/\/id.loc.gov\/authorities\/subjects\/sh85038796", + "label": "Dogs" + } + ], + "website_url": "", + "as:generator": { + "type": "Update", + "actor": { + "url": "http:\/\/localhost:8001\/form\/descriptive-metadata", + "name": "descriptive_metadata", + "type": "Service" + }, + "endTime": "2022-12-05T09:19:37-05:00", + "summary": "Generator", + "@context": "https:\/\/www.w3.org\/ns\/activitystreams" }, - { - "uri": "http:\/\/www.wikidata.org\/entity\/Q60", - "label": "New York City" - } - ], - "date_created_edtf": "", - "date_created_free": null, - "date_embargo_lift": null, - "physical_location": null, - "related_item_note": null, - "rights_statements": "In Copyright - Educational Use Permitted", - "europeana_concepts": "", - "geographic_location": { - "lat": "40.8466508", - "lng": "-73.8785937", - "city": "New York", - "state": "New York", - "value": "The Bronx, Bronx County, New York, United States", - "county": "", - "osm_id": "9691916", - "country": "United States", - "category": "boundary", - "locality": "", - "osm_type": "relation", - "postcode": "", - "country_code": "us", - "display_name": "The Bronx, Bronx County, New York, United States", - "neighbourhood": "Bronx County", - "state_district": "" - }, - "note_publishinginfo": null, - "subject_lcgft_terms": [ - { - "uri": "http:\/\/id.loc.gov\/authorities\/genreForms\/gf2017027249", - "label": "Photographs" - } - ], - "upload_associated_warcs": [], - "physical_description_extent": "", - "subject_lcnaf_personal_names": "", - "subject_lcnaf_corporate_names": "", - "subjects_local_personal_names": "", - "related_item_host_location_url": null, - "subject_lcnaf_geographic_names": [ - { - "uri": "http:\/\/id.loc.gov\/authorities\/names\/n81059724", - "label": "Bronx (New York, N.Y.)" + "date_created": "1910-01-01", + "issue_number": null, + "date_published": "", + "subjects_local": null, + "term_aat_getty": "", + "ap:entitymapping": { + "entity:file": [ + "model", + "audios", + "images", + "videos", + "documents", + "upload_associated_warcs" + ], + "entity:node": [ + "ispartof", + "ismemberof" + ] }, - { - "uri": "http:\/\/id.loc.gov\/authorities\/names\/n79007751", - "label": "New York (N.Y.)" + "europeana_agents": "", + "europeana_places": "", + "local_identifier": "nyhs_PR066_6136", + "subject_wikidata": [ + { + "uri": "http:\/\/www.wikidata.org\/entity\/Q3381576", + "label": "black-and-white photography" + }, + { + "uri": "http:\/\/www.wikidata.org\/entity\/Q60", + "label": "New York City" + } + ], + "date_created_edtf": "", + "date_created_free": null, + "date_embargo_lift": null, + "physical_location": null, + "related_item_note": null, + "rights_statements": "In Copyright - Educational Use Permitted", + "europeana_concepts": "", + "geographic_location": { + "lat": "40.8466508", + "lng": "-73.8785937", + "city": "New York", + "state": "New York", + "value": "The Bronx, Bronx County, New York, United States", + "county": "", + "osm_id": "9691916", + "country": "United States", + "category": "boundary", + "locality": "", + "osm_type": "relation", + "postcode": "", + "country_code": "us", + "display_name": "The Bronx, Bronx County, New York, United States", + "neighbourhood": "Bronx County", + "state_district": "" + }, + "note_publishinginfo": null, + "subject_lcgft_terms": [ + { + "uri": "http:\/\/id.loc.gov\/authorities\/genreForms\/gf2017027249", + "label": "Photographs" + } + ], + "upload_associated_warcs": [], + "physical_description_extent": "", + "subject_lcnaf_personal_names": "", + "subject_lcnaf_corporate_names": "", + "subjects_local_personal_names": "", + "related_item_host_location_url": null, + "subject_lcnaf_geographic_names": [ + { + "uri": "http:\/\/id.loc.gov\/authorities\/names\/n81059724", + "label": "Bronx (New York, N.Y.)" + }, + { + "uri": "http:\/\/id.loc.gov\/authorities\/names\/n79007751", + "label": "New York (N.Y.)" + } + ], + "related_item_host_display_label": null, + "related_item_host_local_identifier": null, + "related_item_host_title_info_title": "", + "related_item_host_type_of_resource": null, + "physical_description_note_condition": null + } + ``` + + **Based on our own plan:** + + 1. Check first if of type `photograph` and `memberof` Node ID `16` + + ```JSON + { "op": "test", "path": "/type", "value": "Photograph"}, + { "op": "test", "path": "/ismemberof", "value": [16]} + ``` + + Notice that to be really sure we also match the data type, an `array` with a single value `16` of type integer (makes sense since the Operation is also a JSON and will be evaluated in the same way as the source RAW JSON). This is a precise match. if the ADO belongs to multiple Collections it will fail of course. + + 2. Now we are going to convert `description` key from a single value into an array. + + ```JSON + {"op": "add","path": "/temp_description_array","value": []}, + {"op": "move","from": "/description","path": "/temp_description_array/-"}, + {"op": "move","from": "/temp_description_array","path": "/description"}, + ``` + + This is a multi step operation. Given that JSON Patch can not "cast" types and depends on a given datatype to be present before, e.g. adding a new value to it, we use here a temporary key. + Notice that you can not “add“ or “move“ to e.g. the position `0` because the destination array is indeed empty (that will fail). But by using the `dash` you can command it to put it at the end, which on an empty list is also at the beginning (we are starting to understand this!). + + 3. And the extra LOD entry for wikidata, in this case the Q Item for [collie](https://www.wikidata.org/wiki/Q1196071) (type of herding dog) + + ```JSON + { "op": "add", "path": "/subject_wikidata/-", "value": { + "uri": "https://www.wikidata.org/wiki/Q1196071", + "label": "collie" } - ], - "related_item_host_display_label": null, - "related_item_host_local_identifier": null, - "related_item_host_title_info_title": "", - "related_item_host_type_of_resource": null, - "physical_description_note_condition": null - } - ``` - -**Based on our own plan:** - -1. Check first if of type `photograph` and `memberof` Node ID `16` - -```JSON -{ "op": "test", "path": "/type", "value": "Photograph"}, -{ "op": "test", "path": "/ismemberof", "value": [16]} -``` - -Notice that to be really sure we also match the data type, an `array` with a single value `16` of type integer (makes sense since the Operation is also a JSON and will be evaluated in the same way as the source RAW JSON). This is a precise match. if the ADO belongs to multiple Collections it will fail of course. - -2. Now we are going to convert `description` key from a single value into an array. - -```JSON -{"op": "add","path": "/temp_description_array","value": []}, -{"op": "move","from": "/description","path": "/temp_description_array/-"}, -{"op": "move","from": "/temp_description_array","path": "/description"}, -``` - -This is a multi step operation. Given that JSON Patch can not "cast" types and depends on a given datatype to be present before, e.g. adding a new value to it, we use here a temporary key. -Notice that you can not “add“ or “move“ to e.g. the position `0` because the destination array is indeed empty (that will fail). But by using the `dash` you can command it to put it at the end, which on an empty list is also at the beginning (we are starting to understand this!). - -3. And the extra LOD entry for wikidata, in this case the Q Item for [collie](https://www.wikidata.org/wiki/Q1196071) (type of herding dog) - -```JSON -{ "op": "add", "path": "/subject_wikidata/-", "value": { - "uri": "https://www.wikidata.org/wiki/Q1196071", - "label": "collie" } -} -``` + ``` -4. Finally we are going to copy the `geographic_location.state` value and put it into `subjects_local`: +4. Finally we are going to copy the `geographic_location.state` value and put it into `subjects_local`: ```JSON {"op": "remove", "path": "/subjects_local"}, @@ -378,9 +378,9 @@ Notice that you can not “add“ or “move“ to e.g. the position `0` because {"op": "copy", "from": "/geographic_location/state", "path": "/subjects_local/-"} ``` -Why so many operations? Because initially ` "subjects_local"` had a value of `null` and it is not suited to generate/add to it as a multivalued key because of that. So we need to remove it first, recreate it as an empty list and then we can copy. **Pro Note:** you could partially rewrite this as `replace` operation! +Why so many operations? Because initially ` "subjects_local"` had a value of `null` and it is not suited to generate/add to it as a multivalued key because of that. So we need to remove it first, recreate it as an empty list and then we can copy. **Pro Note:** you could partially rewrite this as `replace` operation! -??? info "what if it subjects_local has data? +??? info "what if subjects_local has data?" You will lose it! you can add a “test“ or move data instead of recreating. Many choices.