scrapy-spider-metadata support. #75
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #75 +/- ##
==========================================
- Coverage 95.58% 93.94% -1.64%
==========================================
Files 15 15
Lines 747 760 +13
==========================================
Hits 714 714
- Misses 33 46 +13
☔ View full report in Codecov by Sentry. |
|
I haven't added tests because the project doesn't currently test the command output either. It also needs a Scrapy project, which is possible (we generate Scrapy projects in tests in other libraries). |
| # make sure it's serializable | ||
| json.dumps(metadata_dict) | ||
| except (TypeError, ValueError): | ||
| continue |
There was a problem hiding this comment.
The command prints a JSON text which some other components read, we could print warnings to stderr but I don't think anything will read them (but I'm not sure).
There was a problem hiding this comment.
Still, it seems currently it's the only place where the error could be show; without it there is no indication of issue at all. But maybe there is a better place for it?
There was a problem hiding this comment.
I tried to do that and found that the command explicitly disables logging:
There was a problem hiding this comment.
We might need to find another place to log these errorrs, or document it (in scrapy-spider-metadata?). But it seems this shouldn't block this PR.
|
OK, I guess we need a test because the initially pushed code didn't work correctly. |
| try: | ||
| from scrapy_spider_metadata import get_spider_metadata | ||
| except ImportError: | ||
| pass |
There was a problem hiding this comment.
It seems we lack tests on CI which check a case where the library is not installed
There was a problem hiding this comment.
Why isn't this line detected as covered?
There was a problem hiding this comment.
None of run() is detected as covered: https://app.codecov.io/github/scrapinghub/scrapinghub-entrypoint-scrapy/commit/ebf23f19ff3bc1853a35f001a8b2df7e1b1ec63b/blob/sh_scrapy/commands/shub_image_info.py
Not sure if codecov can detect coverage for code run as a separate process.
No description provided.