Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Jest Integration Tests.x-pack/plugins/task_manager/server/integration_tests - unrecognized task types should be no workload aggregator errors when there are removed task types #194208

Closed
kibanamachine opened this issue Sep 26, 2024 · 18 comments · Fixed by #195496 or #196179
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Sep 26, 2024

A test failed on a tracked branch

Error: expect(jest.fn()).not.toHaveBeenCalled()

Expected number of calls: 0
Received number of calls: 1

1: "Task SLO:ORPHAN_SUMMARIES-CLEANUP-TASK \"SLO:ORPHAN_SUMMARIES-CLEANUP-TASK:1.0.0\" failed: ResponseError: search_phase_execution_exception
	Root causes:
		no_shard_available_action_exception: null", {"error": {"stack_trace": "ResponseError: search_phase_execution_exception
	Root causes:
		no_shard_available_action_exception: null
    at KibanaTransport._request (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/node_modules/@elastic/transport/src/Transport.ts:601:17)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at /opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/node_modules/@elastic/transport/src/Transport.ts:704:22
    at KibanaTransport.request (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/node_modules/@elastic/transport/src/Transport.ts:701:14)
    at KibanaTransport.request (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/packages/core/elasticsearch/core-elasticsearch-client-server-internal/src/create_transport.ts:72:17)
    at Client.SearchApi [as search] (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/node_modules/@elastic/elasticsearch/src/api/api/search.ts:96:10)
    at SloOrphanSummaryCleanupTask.fetchSloSummariesIds (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/x-pack/plugins/observability_solution/slo/server/services/tasks/orphan_summary_cleanup_task.ts:137:20)
    at SloOrphanSummaryCleanupTask.runTask (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/x-pack/plugins/observability_solution/slo/server/services/tasks/orphan_summary_cleanup_task.ts:83:48)
    at TaskManagerRunner.run (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/x-pack/plugins/task_manager/server/task_running/task_runner.ts:389:22)"}, "service": {"node": {"roles": ["background_tasks", "ui"]}}, "tags": ["SLO:ORPHAN_SUMMARIES-CLEANUP-TASK", "SLO:ORPHAN_SUMMARIES-CLEANUP-TASK:1.0.0", "task-run-failed"]}
    at Object.toHaveBeenCalled (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727373500529827755/elastic/kibana-on-merge/kibana/x-pack/plugins/task_manager/server/integration_tests/removed_types.test.ts:124:29)
    at runNextTicks (node:internal/process/task_queues:60:5)
    at processTimers (node:internal/timers:511:9)

First failure: kibana-on-merge - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Sep 26, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Sep 26, 2024
@kibanamachine kibanamachine added the Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) label Sep 26, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Sep 26, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

ymao1 added a commit that referenced this issue Oct 9, 2024
…r/server/integration_tests - unrecognized task types should be no workload aggregator errors when there are removed task types (#195496)

Resolves #194208

## Summary

The original integration test was checking for the (non) existence of
any error logs on startup when there are removed task types, which was
not specific enough because there were occasionally error logs like

```
"Task SLO:ORPHAN_SUMMARIES-CLEANUP-TASK \"SLO:ORPHAN_SUMMARIES-CLEANUP-TASK:1.0.0\" failed: ResponseError: search_phase_execution_exception
```

so this PR updates the integration test to check specifically for
workload aggregator error logs

Co-authored-by: Elastic Machine <[email protected]>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 9, 2024
…r/server/integration_tests - unrecognized task types should be no workload aggregator errors when there are removed task types (elastic#195496)

Resolves elastic#194208

## Summary

The original integration test was checking for the (non) existence of
any error logs on startup when there are removed task types, which was
not specific enough because there were occasionally error logs like

```
"Task SLO:ORPHAN_SUMMARIES-CLEANUP-TASK \"SLO:ORPHAN_SUMMARIES-CLEANUP-TASK:1.0.0\" failed: ResponseError: search_phase_execution_exception
```

so this PR updates the integration test to check specifically for
workload aggregator error logs

Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 742cd13)
@kibanamachine kibanamachine reopened this Oct 10, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@ymao1
Copy link
Contributor

ymao1 commented Oct 10, 2024


FAIL  x-pack/plugins/task_manager/server/integration_tests/removed_types.test.ts (89.987 s)
--
  | 2024-10-10 09:47:40 UTC | ● unrecognized task types › should be no workload aggregator errors when there are removed task types
  | 2024-10-10 09:47:40 UTC |  
  | 2024-10-10 09:47:40 UTC | expect(received).not.toMatch(expected)
  | 2024-10-10 09:47:40 UTC |  
  | 2024-10-10 09:47:40 UTC | Matcher error: received value must be a string
  | 2024-10-10 09:47:40 UTC |  
  | 2024-10-10 09:47:40 UTC | Received has type:  object
  | 2024-10-10 09:47:40 UTC | Received has value: {"error": {"stack_trace": "ResponseError: search_phase_execution_exception
  | 2024-10-10 09:47:40 UTC | Root causes:
  | 2024-10-10 09:47:40 UTC | no_shard_available_action_exception: null
  | 2024-10-10 09:47:40 UTC | at KibanaTransport._request (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/node_modules/@elastic/transport/src/Transport.ts:601:17)
  | 2024-10-10 09:47:40 UTC | at processTicksAndRejections (node:internal/process/task_queues:95:5)
  | 2024-10-10 09:47:40 UTC | at /opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/node_modules/@elastic/transport/src/Transport.ts:704:22
  | 2024-10-10 09:47:40 UTC | at KibanaTransport.request (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/node_modules/@elastic/transport/src/Transport.ts:701:14)
  | 2024-10-10 09:47:40 UTC | at KibanaTransport.request (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/packages/core/elasticsearch/core-elasticsearch-client-server-internal/src/create_transport.ts:72:17)
  | 2024-10-10 09:47:40 UTC | at Client.SearchApi [as search] (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/node_modules/@elastic/elasticsearch/src/api/api/search.ts:96:10)
  | 2024-10-10 09:47:40 UTC | at SloOrphanSummaryCleanupTask.fetchSloSummariesIds (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/x-pack/plugins/observability_solution/slo/server/services/tasks/orphan_summary_cleanup_task.ts:137:20)
  | 2024-10-10 09:47:40 UTC | at SloOrphanSummaryCleanupTask.runTask (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/x-pack/plugins/observability_solution/slo/server/services/tasks/orphan_summary_cleanup_task.ts:83:48)
  | 2024-10-10 09:47:40 UTC | at TaskManagerRunner.run (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1728552484023113068/elastic/kibana-on-merge/kibana/x-pack/plugins/task_manager/server/task_running/task_runner.ts:389:22)"}, "service": {"node": {"roles": ["background_tasks", "ui"]}}, "tags": ["SLO:ORPHAN_SUMMARIES-CLEANUP-TASK", "SLO:ORPHAN_SUMMARIES-CLEANUP-TASK:1.0.0", "task-run-failed"]}
  | 2024-10-10 09:47:40 UTC

We added a regex matcher to only look for the (non) presence of WorkloadAggregator errors but in this instance this threw an error because the error logger was called with an object not a string 🤦‍♀

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@jbudz
Copy link
Member

jbudz commented Oct 16, 2024

For anyone looking to skip 8.x: the backport is in progress now. Holding off.

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@ymao1
Copy link
Contributor

ymao1 commented Oct 16, 2024

8.x backport PR just merged so closing this issue.

@ymao1 ymao1 closed this as completed Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment