Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Change evaluation workflow input data format (no-changelog) #13109

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

burivuhster
Copy link
Contributor

Summary

This PR makes the following changes to the format of data supplied to the evaluation workflow by test runner:

  • removed some internals, making the data structure simpler
  • added annotation data: vote, annotation tags and highlighted data
  • nodes are referenced by their IDs instead of names

Related Linear tickets, Github issues, and Community forum posts

Review / Merge checklist

  • PR title and summary are descriptive. (conventions)
  • Docs updated or follow-up ticket created.
  • Tests included.
  • PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

@burivuhster burivuhster marked this pull request as ready for review February 6, 2025 17:08
@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Feb 6, 2025
Copy link
Contributor

@jeanpaul jeanpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me, but should we cover this with some tests?

Copy link

codecov bot commented Feb 7, 2025

@burivuhster burivuhster requested a review from jeanpaul February 7, 2025 12:47
Copy link

cypress bot commented Feb 7, 2025

n8n    Run #9191

Run Properties:  status check passed Passed #9191  •  git commit 336c8e7cec: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 burivuhster 🗃️ e2e/*
Project n8n
Branch Review ai-635-add-tags-or-annotations-any-tags-or-thumbs-up-thumbs-down
Run status status check passed Passed #9191
Run duration 04m 36s
Commit git commit 336c8e7cec: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 burivuhster 🗃️ e2e/*
Committer Eugene Molodkin
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 5
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 434
View all changes introduced in this branch ↗︎

Copy link
Contributor

github-actions bot commented Feb 7, 2025

✅ All Cypress E2E specs passed

jeanpaul
jeanpaul previously approved these changes Feb 7, 2025
Copy link
Contributor

@jeanpaul jeanpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Copy link
Contributor

github-actions bot commented Feb 7, 2025

✅ All Cypress E2E specs passed

…-thumbs-up-thumbs-down

# Conflicts:
#	packages/cli/src/evaluation.ee/test-runner/__tests__/test-runner.service.ee.test.ts
#	packages/cli/src/evaluation.ee/test-runner/utils.ee.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants