New Components - zenscrape #15467
New Components - zenscrape #15467
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 3 Skipped Deployments
|
WalkthroughThis pull request introduces comprehensive support for the Zenscrape service in Pipedream. The changes include a new Zenscrape application module with enhanced configuration options, two new actions for retrieving credit status and website content, and a new source for monitoring website content updates. The package version has been updated, and new dependencies have been added to support these functionalities. Changes
Assessment against linked issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (5)
components/zenscrape/zenscrape.app.mjs (3)
6-36: LGTM! Consider enhancing credit usage visibility.The prop definitions are well-structured with clear descriptions. The credit costs for premium (20 credits) and render (5 credits) features are properly documented.
Consider adding a method to calculate total credits that would be consumed based on the selected options, to help users better manage their credit usage.
38-53: Add error handling for API requests.The request implementation looks good, but it would benefit from proper error handling.
Add error handling to catch and transform API errors into user-friendly messages:
_makeRequest({ $ = this, path, ...opts }) { - return axios($, { + return axios($, { url: `${this._baseUrl()}${path}`, headers: { apikey: this.$auth.api_key, }, ...opts, - }); + }).catch((error) => { + const statusCode = error.response?.status; + const message = error.response?.data?.message || error.message; + throw new Error(`Zenscrape API request failed: ${statusCode} - ${message}`); + }); }
54-64: Add input validation for API methods.The methods are clean but would benefit from parameter validation.
Consider adding parameter validation:
getContent(opts = {}) { + const { params = {} } = opts; + if (!params.url) { + throw new Error("URL parameter is required"); + } return this._makeRequest({ path: "/get", ...opts, }); }components/zenscrape/actions/get-credit-status/get-credit-status.mjs (1)
1-19: Enhance the success summary message.The implementation looks good, but the summary message could be more informative.
Include the actual credit status in the summary message:
- $.export("$summary", "Successfully retrieved credit status."); + $.export("$summary", `Successfully retrieved credit status. Available credits: ${response.remaining_credits}`);components/zenscrape/actions/get-website-content/get-website-content.mjs (1)
1-56: Add rate limit warning for premium features.The implementation looks good, but users should be warned about credit consumption.
Add a warning when premium features are enabled:
async run({ $ }) { + const creditCost = (this.premium ? 20 : 1) + (this.render ? 5 : 0); + if (creditCost > 1) { + $.export("$summary", `Warning: This request will consume ${creditCost} credits`); + } const response = await this.zenscrape.getContent({ $, params: {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (5)
components/zenscrape/actions/get-credit-status/get-credit-status.mjs(1 hunks)components/zenscrape/actions/get-website-content/get-website-content.mjs(1 hunks)components/zenscrape/package.json(2 hunks)components/zenscrape/sources/website-content-updated/website-content-updated.mjs(1 hunks)components/zenscrape/zenscrape.app.mjs(1 hunks)
🧰 Additional context used
🪛 GitHub Check: Lint Code Base
components/zenscrape/sources/website-content-updated/website-content-updated.mjs
[warning] 7-7:
Source names should start with "New". See https://pipedream.com/docs/components/guidelines/#source-name
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: Verify TypeScript components
- GitHub Check: pnpm publish
- GitHub Check: Publish TypeScript components
🔇 Additional comments (2)
components/zenscrape/package.json (2)
3-3: LGTM! Version bump is appropriate.The minor version bump (0.0.1 → 0.1.0) aligns with semver for the addition of new features (Zenscrape components) without breaking changes.
15-18: Consider security and version management improvements.
The
md5package:
- It's using an old version (2.3.0 from 2016)
- MD5 is cryptographically broken, though it may be acceptable for content change detection
- Consider using more secure alternatives like SHA-256 for content hashing
For
@pipedream/platform:
- Verify if 3.0.3 is the latest compatible version
Let's check the latest versions and any known vulnerabilities:
components/zenscrape/sources/website-content-updated/website-content-updated.mjs
Outdated
Show resolved
Hide resolved
components/zenscrape/sources/website-content-updated/website-content-updated.mjs
Outdated
Show resolved
Hide resolved
components/zenscrape/sources/website-content-updated/website-content-updated.mjs
Outdated
Show resolved
Hide resolved
|
/approve |
Resolves #9706
Summary by CodeRabbit
Release Notes
New Features
Improvements
Version Update
Dependencies
@pipedream/platformandmd5dependencies