Url preview endpoint

[adnull]

This PR introduces the missing URL Preview functionality ( Issue matrix-org/dendrite#1303 )

According to the specification that endpoint requests require authentication and are rate limited. There are both endpoints (v1 and v3) as some clients still use deprecated endpoints for url preview requests.

Key features:

• takes info from html pages or image links. Max image size downloadable is configurable. Timeout is configurable
• stores the url's image as a thumbnail in media storage if the thumbnail size for url previews is configured. Otherwise stores the original image (to make it sy-test compatible).
• uses the maxThumbnailGenerators variable to prevent overload. Waits 30s for free generator.
• keeps an in-memory cache of url preview responses. The cache entry expiration time is configurable
• stores the the url preview json responses in the media storage to prevent any extra load
• url preview blacklist is configurable as a list of regexps in the config

Missing:

• some tests
• synapse's ability to query/extract oEmbed resources
• ip blacklist checks

I'd like somebody to take a look at the code.

• I have added Go unit tests or Complement integration tests for this PR or I have justified why this PR doesn't need tests
• Pull request includes a sign off below using a legally identifiable name or I have already signed off privately

Signed-off-by: adnull <[email protected]>

[CLAassistant]

All committers have signed the CLA.