generalize hyperlink extractor

[marianneke]

Overview

Closes #XXXX.

What problem does this address?

• Hyperlink extractor does not return URLs in tags other than <a> tags
• Hyperlink extractor also does not return information inside the tags, which could contain the actual filename, e.g. <a href="...">filename.zip</a>
• In the case of the new data archiver Add USGS USWTDB data #539 the name would be needed for extracting properties such as dataset upload date.

What did you change in this PR?

• Hyperlink extractor returns all hyperlinks (based on some list of attributes rather than only the href attribute in the tag)
• Return value is dict instead of set, with values {hyperlink: name} where name contains the text between tags, if present, and defaults to hyperlink itself.
• Check for name matches in both the name and the hyperlink instead of just the hyperlink

Testing

WIP
Note that the change from datatype set -> dict should not be an issue downstream as long as the result of get_hyperlinks is looped over, since a simple for loop returns the keys, which are the hyperlinks, and ignores the values. This should be checked.
Errors might occur if there is text between the tags but it lacks information that is present in the hyperlink, resulting in hyperlinks not being downloaded. solved by checking both name and link for pattern

To-do list

Tasks

Preview Give feedback

1. How to handle cases where the text between tags does not contain the filename (so we may want to default to the hyperlink for the name in that case)
2. Use the name as the filename if downstream usecase does expect a valid filename, and the hyperlink does not contain the filename
3. Update relevant documentation - like comments, docstrings, README, release notes, etc.
4. Review the PR yourself and call out any questions or issues you have

[zschira]

This all makes sense at a high level, I'm mostly just concerned with getting the CI passing.

I'm not 100% sure what still needs to be done for handling the filename stuff, but happy to take another look when ready, or discuss further if you have any questions. I will say, if we need to enable some more complex html extraction, I think it could be reasonable to bring in beautiful-soup. Also, it's not really a big deal if we end up using different filenames from those coming from the upstream sources, so if we can construct our own meaningful files without extracting them from the html, that's a totally fine option.

[zschira]

This looks good to me, @jdangerx do you think we should we hold off on merging if you're testing beautiful soup, or go for it and we can replace with BS in the future?