Integrate with SOS by using Iceland

[nuest]

Things to consider/steps to be taken:

Switch to Iceland

• create a maintenance branch of OSS > tag https://github.com/52North/OpenSensorSearch/releases/tag/pre-iceland
• add new 52N parent pom
• create branch oss-iceland
  • switch from Guice modules to iceland providers, might have to reorganize the source code so that functionality is more closely modelled after the interfaces and operations
  • change license of OSS (back to) GPL (because of SOS dependencies)
  • re-check licenses of depedencies
  • switch to latest Solr and fix compile time errors
  • move concrete logging dependencies and configurations to webapp module
  • breakthroughs to be done by @nuest
    • GetCapabilities works in standalone version of the OSS based on iceland
    • GetCapabilities in bundle with the SOS, which contains the bundled SOS as harvested service (see http://52north.org/schema/sir/0.3.2/sirGetCapabilities.xsd) read from iceland configuration, maybe to the webapp-bundle, see comment below by @autermann
    • OpenSearch works using the SOS DAOs (sensor count) and database model, i.e. reading from a database created by the SOS
    • transactional operations from the SIR result in "not implemented" OWS Exception
  • prioritized list of features (remove implementations of not required features, http://52north.org/communities/sensorweb/discovery/index.html); in the OSS becomes read-only
    • SensorSearch operation of SIR (high), if SIR then also do DescribeSensor
    • JSON binding (high)
    • Autocomplete endpoint with document index database (high)
    • UI (medium)
    • Harvesting (medium)
      • re-add document database (becomes relevant only with external harvested SOS, or with millions of sensors)
      • concept of "internal" and "external" sensors to the document database (e.g. listen to SOS event bus and delete internal sensors on DeleteSensor@SOS)
    • Validation endpoints (low)
    • deactivate transactional interface of SIR (throw exceptions)
    • deactivate status operations of SIR (throw exceptions)
    • optional: JSON binding and autocomplete works with Angular UI
• switch from XMLBeans to JAXB

Notes about modules

• arsearch > not needed anymore, remove
• common > only home dir config mechanism > replaced by iceland configuration, remove??
• it > fix first! remove JS-harvesting stuff, ignore tests
• misc > keep, of course, will use SOS code so change the license to GPL
• service > actual business logic, keep all classes but focus on OpenSearch + SOS DAOs
• sir-common >
• sir-it > SIR integration tests
• sir-json > keep as needed for JSON encoding
• sor-common > Sensor Observable Registry > drop it.
• sor > Sensor Observable Registry > drop it.
• website > keep as independent search client / landing page, remove some unneeded code (SOR, all the Java classes, JSPs, and login-related stuff, remove all "under development", ..), test update Angular
• remove Swagger

Adjustments for SOS-integration

• extend data model to include "internal" and "external" sensor descriptions
  • internal = in SOS database using SOS DAOs, need some internal harvesting for search DB
  • external = in Solr using OSS DAOs, or just use Lucene internally without Solr?
  • fromt the outside (to a client) there is no distinguishment between the
• check Esri Geoportal Server how they can automatically create a Solr instance at installation time - or do they just use Lucene?
• switch configuration to use iceland's Settings API
• add internal harvesting (e.g. add internal sensors to the search database)
• remove SOR
• What about the validation endpoints?
• And the conversion endpoints?

[nuest]

@autermann @CarstenHollmann Can you confirm that, if we switch OpenSensorSearch to be based on iceland, it should be very straightforward to add it to the (existing or a new one) webapp-bundle? Then configuration and endpoints etc. should "just work", correct?

[nuest]

@autermann @CarstenHollmann Would it be possible to "add" example requests to the test client from a different file? I don't want to overwrite the example requests in the bundle, but include an additional file with example requests to be shown (given the respective endpoints are activated) and so forth.

[autermann]

Currently that's not possible. We're loading the example requests from a single JSON file called client-config.json via JavaScript, so we can't just load a set of file e.g. via a file pattern.

But, as that file is already 5k lines long, it would make sense to allow splitting it up... Something like this should be viable:

{
  "examples": [
    "static/conf/examples/sos_v100.json",
    "static/conf/examples/sos_v20.json",
    "static/conf/examples/aqd_v10.json",
  ]
}

Then you still have to replace the file, but you have to provide far less content...

[nuest]

Or why not reading all files from a directory instead of having this "index" file? Then an overlay could just add a file, or also replace the index file if they do not want other examples to be loaded.

{
  "examples": [
    "static/conf/examples/",
    "static/conf/examples/*.json",
  ]
}

But once the examples are loaded, then the filtering within the client works, right?

I also assume that iceland does not include the test client - correct?

[autermann]

Am 15.06.2015 18:29 schrieb "Daniel Nüst" [email protected]:

Or why not reading all files from a directory instead of having this
"index" file? Then an overlay could just add a file, or also replace the
index file if they do not want other examples to be loaded.

{
"examples": [
"static/conf/examples/",
"static/conf/examples/*.json",
]
}

And how do I load all files in a directory from a browser using JavaScript?

But once the examples are loaded, then the filtering within the client
works, right?

It should :-)

I also assume that iceland does not include the test client - correct?

That's right...

[nuest]

Good point about the javascript loading :-) Regexes also won't work. Ok - I updated the first entry to reflect the agreement on example requests.

[nuest]

Will work on this within https://github.com/nuest/OpenSensorSearch/tree/feature/iceland