Messages to inbox are not forwarded to followers #40

acka47 · 2020-10-14T13:31:09Z

Whether I use the editor or curl, I can post to the inbox but it won't be forwarded to the followers' inboxes.

Curl example:

$ curl -vH "content-type: application/json" 'https://skohub.io/inbox?actor=dini-ag-kim%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271' --data-binary '{"@context":["https://w3id.org/kim/lrmi-profile/draft/context.jsonld",{"@language":"de"}],"name":"222Test-Deutsch als Zweitsprache","id":"https://unterrichten.zum.de/wiki/Deutsch_als_Zweitsprache","description":"Hier ein Versuch, einige markante Unterschiede in wenigen Worten darzustellen: Der entscheidende Unterschied ist die Spracherwerbssituation:","about":[{"id":"https://w3id.org/kim/hochschulfaechersystematik/n271","prefLabel":{"de":"Deutsch als Fremdsprache oder als Zweitsprache"},"type":"Concept","inScheme":{"id":"https://w3id.org/kim/hochschulfaechersystematik/scheme"}}]}'
*   Trying 193.30.112.187...
* TCP_NODELAY set
* Connected to skohub.io (193.30.112.187) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=DE; ST=Nordrhein-Westfalen; L=Koeln; O=Hochschulbibliothekszentrum des Landes Nordrhein-Westfalen; OU=Rechenzentrum; CN=www.skohub.io
*  start date: May 13 06:41:09 2020 GMT
*  expire date: Aug 15 06:41:09 2022 GMT
*  subjectAltName: host "skohub.io" matched cert's "skohub.io"
*  issuer: C=DE; O=Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.; OU=DFN-PKI; CN=DFN-Verein Global Issuing CA
*  SSL certificate verify ok.
> POST /inbox?actor=dini-ag-kim%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271 HTTP/1.1
> Host: skohub.io
> User-Agent: curl/7.58.0
> Accept: */*
> content-type: application/json
> Content-Length: 586
> 
* upload completely sent off: 586 out of 586 bytes

The command will stop at this point for quite some time and after a while this is added:

< HTTP/1.1 404 Not Found
< Date: Wed, 14 Oct 2020 13:15:31 GMT
< Server: Apache/2.4.38 (Debian)
< Content-Length: 275
< Content-Type: text/html; charset=iso-8859-1
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<hr>
<address>Apache/2.4.38 (Debian) Server at 193.30.112.82 Port 80</address>
</body></html>
* Connection #0 to host skohub.io left intact

Result: the message is in the inbox but I do not get it via Mastodon although I am a follower of the respective concept.

Using the editor or the extension it is very similar:

I describe a resource and press "Publish".
The payload will be delivered to the inbox.
It takes quite some time and then an error message pops up:

@dr0i, there must be something wrong on the server side, I guess. It did function before, though. Maybe @literarymachine has an idea...

The text was updated successfully, but these errors were encountered:

literarymachine · 2020-10-14T19:18:43Z

I believe this is the relevant message:

ALPN, server did not agree to a protocol

This could well be something in the proxy config.

acka47 · 2020-10-15T08:42:14Z

On the test server, it doesn't work as well:

$ curl -vH "content-type: application/json" 'https://test.skohub.io/inbox?actor=hbz%2Fvocabs-edu%2Fheads%2Fmaster%2Fw3id.org%2Fclass%2Fesc%2Fn0322' --data-binary '{"@context":["https://w3id.org/kim/lrmi-profile/draft/context.jsonld",{"@language":"de"}],"name":"222Test-Bibliaries","id":"https://unterrichten.zum.de/wiki/Deutsch_als_Zweitsprache","description":"Hier ein Versuch, einige markante Unterschiede in wenigen Worten darzustellen: Der entscheidende Unterschied ist die Spracherwerbssituation:","about":[{"id":"https://test.skohub.io/hbz/vocabs-edu/heads/master/w3id.org/class/esc/n0322","prefLabel":{"de":"Deutsch als Fremdsprache oder als Zweitsprache"},"type":"Concept","inScheme":{"id":"https://w3id.org/kim/hochschulfaechersystematik/scheme"}}]}'
*   Trying 193.30.112.187...
* Connected to test.skohub.io (193.30.112.187) port 443 (#0)
* found 127 certificates in /etc/ssl/certs/ca-certificates.crt
* found 520 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.skohub.io (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: C=DE,ST=Nordrhein-Westfalen,L=Koeln,O=Hochschulbibliothekszentrum des Landes Nordrhein-Westfalen,OU=Rechenzentrum,CN=www.skohub.io
* 	 start date: Wed, 13 May 2020 06:41:09 GMT
* 	 expire date: Mon, 15 Aug 2022 06:41:09 GMT
* 	 issuer: C=DE,O=Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.,OU=DFN-PKI,CN=DFN-Verein Global Issuing CA
* 	 compression: NULL
* ALPN, server did not agree to a protocol
> POST /inbox?actor=hbz%2Fvocabs-edu%2Fheads%2Fmaster%2Fw3id.org%2Fclass%2Fesc%2Fn0322 HTTP/1.1
> Host: test.skohub.io
> User-Agent: curl/7.47.0
> Accept: */*
> content-type: application/json
> Content-Length: 595
> 
* upload completely sent off: 595 out of 595 bytes
< HTTP/1.1 404 Not Found
< Date: Thu, 15 Oct 2020 08:39:24 GMT
< Server: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.1
< Content-Length: 196
< Content-Type: text/html; charset=iso-8859-1
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
</body></html>
* Connection #0 to host test.skohub.io left intact

dr0i · 2020-10-15T08:48:24Z

If you use without SSL, it seems to work, right?

acka47 · 2020-10-15T09:47:12Z

If you use without SSL, it seems to work, right?

Nope, when using http-only, the server sends a 301 redirect to the https version:

$ curl -vH "content-type: application/json" 'http://skohub.io/inbox?actor=dini-ag-kim%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271' --data-binary '{"@context":["https://w3id.org/kim/lrmi-profile/draft/context.jsonld",{"@language":"de"}],"name":"222Test-Deutsch als Zweitsprache","id":"https://unterrichten.zum.de/wiki/Deutsch_als_Zweitsprache","description":"Hier ein Versuch, einige markante Unterschiede in wenigen Worten darzustellen: Der entscheidende Unterschied ist die Spracherwerbssituation:","about":[{"id":"https://w3id.org/kim/hochschulfaechersystematik/n271","prefLabel":{"de":"Deutsch als Fremdsprache oder als Zweitsprache"},"type":"Concept","inScheme":{"id":"https://w3id.org/kim/hochschulfaechersystematik/scheme"}}]}'
*   Trying 193.30.112.187...
* Connected to skohub.io (193.30.112.187) port 80 (#0)
> POST /inbox?actor=dini-ag-kim%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271 HTTP/1.1
> Host: skohub.io
> User-Agent: curl/7.47.0
> Accept: */*
> content-type: application/json
> Content-Length: 586
> 
* upload completely sent off: 586 out of 586 bytes
< HTTP/1.1 301 Moved Permanently
< Date: Thu, 15 Oct 2020 09:45:19 GMT
< Server: Apache/2.4.10 (Linux/SUSE)
< Location: https://skohub.io/inbox?actor=dini-ag-kim%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271
< Content-Length: 427
< Content-Type: text/html; charset=iso-8859-1
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://skohub.io/inbox?actor=dini-ag-kim%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271">here</a>.</p>
<hr>
<address>Apache/2.4.10 (Linux/SUSE) Server at skohub.io Port 80</address>
</body></html>
* Connection #0 to host skohub.io left intact

dr0i · 2020-10-15T10:04:36Z

Interesting. Use the "-L" with curl and it seems to work, no?

dr0i · 2020-10-15T10:20:47Z

Uh, right: "POST has been converted to GET and no more data is forwarded." , so this is a different thing.

dr0i · 2020-10-15T13:00:47Z

Ok. Got it:

HD space was low (<5 GB), elasticsearch complains. This was passed to the skohub-pubsub.log:

(node:12036) UnhandledPromiseRejectionWarning: ResponseError: cluster_block_exception
at IncomingMessage. (/home/lod/git/skohub-pubsub/node_modules/@elastic/elasticsearch/lib/Transport.js:294:25)

Fixed this by freeing disk space. Restarted ES.

After fixing 1. and POSTing the data ES responds with:

MapperParsingException: object mapping for [attachment.@context] tried to parse field [null] as object, but found a concrete value

Manipulating the data to POST to:

  "@context" :{ 
     "id" : "https://w3id.org/kim/lrmi-profile/draft/context.jsonld"
},
...

the document is created.

To not happen 1. again I will put free space observated by Nagios which warns us by email.
To fix 2. : you may want to have a language attribute for @context even if this does not make sense (or does it?) but we have to tweak the ES settings to allow this. (would this be compatible with the already indexed data?)

dr0i · 2020-10-15T13:05:15Z

fyi, index was set to "read only" because of the low watermark of disk space since 2020-10-03.

dr0i · 2020-10-15T13:57:50Z

As discussed offline with @acka47 we delete the ES index. A new indexed document would then determine the ES settings field structure to be ok with what is used atm.

dr0i · 2020-10-15T14:09:08Z

Not working. After deleting and recreation with the json in question, ES logs:

java.lang.IllegalArgumentException: Can't merge a non object mapping [attachment.@context] with an object mapping [attachment.@context]

dr0i · 2020-10-15T14:17:31Z

@acka47 this would work :

{"@context":[ {"@id" : "https://w3id.org/kim/lrmi-profile/draft/context.jsonld"},{"@language":"de"}],

we are ok with that?

dr0i · 2020-10-15T14:39:02Z

@acka47 confirmed to change the schema at https://github.com/dini-ag-kim/lrmi-profile to comply to elasticsearch's "Arrays with a mixture of data types are not supported".

acka47 · 2020-10-15T15:12:47Z

@acka47 confirmed to change the schema at https://github.com/dini-ag-kim/lrmi-profile to comply to elasticsearch's "Arrays with a mixture of data types are not supported".

See dini-ag-kim/amb#26

@context

In JSON-LD an '@context' is used to define semantics of JSON fields. These definitions can consist of an array with mixed data types. This is invalid when tried to be indexed into elasticsearch. As a mapping of the index can be defined to ignore the '@context' field for indexing such an JSON-LD document can then be loaded into elasticsearch. The mapping must be set when initializing the index. This is done in the start script provided by skohub-pubsub. See #40.

@context

In JSON-LD an '@context' is used to define semantics of JSON fields. These definitions can consist of an array with mixed data types. This is invalid when tried to be indexed into elasticsearch. As a mapping of the index can be defined to ignore the '@context' field for indexing such an JSON-LD document can then be loaded into elasticsearch. The mapping must be set when initializing the index. This is done in the start script provided by skohub-pubsub. See #40.

dr0i · 2020-10-27T08:22:18Z

Deployed to test. Please test it @acka47, use the 'test.skohub.io' URL to do so.

acka47 · 2020-10-29T09:53:31Z

It does work better than before but the messages won't get trough to Mastodon anyway. I created a new build at https://test.skohub.io/build/?id=e06c1fe4-a93f-4a92-bed7-61e70a7eabd1.

I followed n271, see the followers list. I posted with curl like this:

$ curl -vH "content-type: application/json" 'https://test.skohub.io/inbox?actor=acka47%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271' --data-binary '{"@context":["https://w3id.org/kim/lrmi-profile/draft/context.jsonld",{"@language":"de"}],"name":"222Test-Deutsch als Zweitsprache","id":"https://unterrichten.zum.de/wiki/Deutsch_als_Zweitsprache","description":"Hier ein Versuch, einige markante Unterschiede in wenigen Worten darzustellen: Der entscheidende Unterschied ist die Spracherwerbssituation:","about":[{"id":"https://w3id.org/kim/hochschulfaechersystematik/n271","prefLabel":{"de":"Deutsch als Fremdsprache oder als Zweitsprache"},"type":"Concept","inScheme":{"id":"https://w3id.org/kim/hochschulfaechersystematik/scheme"}}]}'
*   Trying 193.30.112.187...
* Connected to test.skohub.io (193.30.112.187) port 443 (#0)
* found 127 certificates in /etc/ssl/certs/ca-certificates.crt
* found 520 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.skohub.io (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: C=DE,ST=Nordrhein-Westfalen,L=Koeln,O=Hochschulbibliothekszentrum des Landes Nordrhein-Westfalen,OU=Rechenzentrum,CN=www.skohub.io
* 	 start date: Wed, 13 May 2020 06:41:09 GMT
* 	 expire date: Mon, 15 Aug 2022 06:41:09 GMT
* 	 issuer: C=DE,O=Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.,OU=DFN-PKI,CN=DFN-Verein Global Issuing CA
* 	 compression: NULL
* ALPN, server did not agree to a protocol
> POST /inbox?actor=acka47%2Fhochschulfaechersystematik%2Fheads%2Fmaster%2Fw3id.org%2Fkim%2Fhochschulfaechersystematik%2Fn271 HTTP/1.1
> Host: test.skohub.io
> User-Agent: curl/7.47.0
> Accept: */*
> content-type: application/json
> Content-Length: 586
> 
* upload completely sent off: 586 out of 586 bytes
< HTTP/1.1 201 Created
< Date: Thu, 29 Oct 2020 09:45:23 GMT
< Server: Apache/2.4.10 (Linux/SUSE)
< X-Powered-By: Express
< Access-Control-Allow-Origin: *
< Location: https://test.skohub.io/m/5a0776c9-9856-4ccf-9ecd-9c89435f673e
< Content-Length: 0
< 
* Connection #0 to host test.skohub.io left intact

However, the message is not forwarded to the followers it seems as I don't get a message in Mastodon. One strange thing in this context: the concept is not listed in my "following" list on Mastodon (but others are):

acka47 · 2020-10-29T10:02:17Z

However, as the original problem seems to be fixed, we can already deploy the changes to production and then look into the remaining issue.

See #40.

dr0i · 2020-10-29T12:56:17Z

Deployed to production. Please test it @acka47 .

acka47 · 2020-10-29T14:27:40Z

I am sorry to say that but it works on production:

dr0i · 2020-10-29T14:44:12Z

Comparing the logs of production (it's just empty) and the one I got on test.skohub:

(node:24850) UnhandledPromiseRejectionWarning: Error: Unauthorized
at Request.callback (/home/lod/git/skohub-pubsub/node_modules/superagent/lib/node/index.js:804:15)
at IncomingMessage. (/home/lod/git/skohub-pubsub/node_modules/superagent/lib/node/index.js:1036:18)
at Stream.emit (events.js:311:20)
at Unzip. (/home/lod/git/skohub-pubsub/node_modules/superagent/lib/node/unzip.js:55:12)
at Unzip.emit (events.js:323:22)
at endReadableNT (_stream_readable.js:1204:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
(node:24850) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 14)

acka47 · 2020-10-29T14:49:20Z

@literarymachine, does this maybe ring a bell? If not, we can leave this for now nbecause it works on production where it is most important...

literarymachine · 2020-10-30T12:29:17Z

UnhandledPromiseRejectionWarning: Error: Unauthorized

It could be that the message's signature cannot be verified, you might want to compare the configurations on test and prod starting here:

https://github.com/skohub-io/skohub-vocabs/blob/master/.env.example#L6
https://github.com/skohub-io/skohub-pubsub/blob/master/src/activitypub.js#L19

Unfortunately I don't really have the time right now to wrap my head back around those authentication issues...

dr0i · 2020-11-23T08:36:10Z

PUBLIC_KEY on both machines are exactly the same.
npm test on both machines (shutting down the running daemon before because the test is using the same port as the inbuild-webHookServer test) return succesfully.
Is there a private key involved and are these not the same for test and production?

acka47 assigned literarymachine and dr0i Oct 14, 2020

acka47 added the bug Something isn't working label Oct 14, 2020

acka47 unassigned literarymachine Oct 15, 2020

acka47 mentioned this issue Oct 15, 2020

Make "type" array with mandatory default "LearningResource" dini-ag-kim/amb#22

Closed

acka47 mentioned this issue Oct 15, 2020

Context array mit verschiedenen Datentypen ist inkompatibel mit Elasticsearch dini-ag-kim/amb#26

Closed

dr0i assigned acka47 and unassigned dr0i Oct 15, 2020

dr0i mentioned this issue Oct 27, 2020

Set mappings of elastisearch to work with dini kim lrmi profile #41

Merged

dr0i added a commit that referenced this issue Oct 29, 2020

Merge branch '40-forwardMessageElasticsearchMapping'

93ba124

See #40.

acka47 removed their assignment Oct 29, 2020

dr0i assigned literarymachine Oct 30, 2020

literarymachine removed their assignment Oct 30, 2020

dr0i assigned literarymachine Nov 23, 2020

dr0i added question Further information is requested and removed bug Something isn't working labels Dec 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Messages to inbox are not forwarded to followers #40

Messages to inbox are not forwarded to followers #40

acka47 commented Oct 14, 2020

literarymachine commented Oct 14, 2020

acka47 commented Oct 15, 2020

dr0i commented Oct 15, 2020

acka47 commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020 •

edited

Loading

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020 •

edited

Loading

dr0i commented Oct 15, 2020

acka47 commented Oct 15, 2020

dr0i commented Oct 27, 2020

acka47 commented Oct 29, 2020

acka47 commented Oct 29, 2020

dr0i commented Oct 29, 2020

acka47 commented Oct 29, 2020

dr0i commented Oct 29, 2020

acka47 commented Oct 29, 2020 •

edited

Loading

literarymachine commented Oct 30, 2020

dr0i commented Nov 23, 2020

Messages to inbox are not forwarded to followers #40

Messages to inbox are not forwarded to followers #40

Comments

acka47 commented Oct 14, 2020

literarymachine commented Oct 14, 2020

acka47 commented Oct 15, 2020

dr0i commented Oct 15, 2020

acka47 commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020 • edited Loading

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020

dr0i commented Oct 15, 2020 • edited Loading

dr0i commented Oct 15, 2020

acka47 commented Oct 15, 2020

dr0i commented Oct 27, 2020

acka47 commented Oct 29, 2020

acka47 commented Oct 29, 2020

dr0i commented Oct 29, 2020

acka47 commented Oct 29, 2020

dr0i commented Oct 29, 2020

acka47 commented Oct 29, 2020 • edited Loading

literarymachine commented Oct 30, 2020

dr0i commented Nov 23, 2020

dr0i commented Oct 15, 2020 •

edited

Loading

dr0i commented Oct 15, 2020 •

edited

Loading

acka47 commented Oct 29, 2020 •

edited

Loading