Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jobs and resources charts in dashboard page #378

Merged
merged 23 commits into from
Nov 11, 2024
Merged

Jobs and resources charts in dashboard page #378

merged 23 commits into from
Nov 11, 2024

Conversation

rezib
Copy link
Contributor

@rezib rezib commented Nov 5, 2024

Tasks to finish this feature:

  • Disable pretty formatting in prometheus/metrics assets to avoid too many new lines
  • Python unit tests to cover new code on backend
  • Update documentation

@rezib rezib added the feature New feature or enhancement to develop. label Nov 5, 2024
@rezib rezib added this to the v4.0.0 milestone Nov 5, 2024
@rezib rezib self-assigned this Nov 5, 2024
- Increase RAM size and cores on emulated nodes
- Enable metrics on tiny and emulator
- Set select/cons_tres on emulator
- Define a partition with DefMemPerCPU on tiny
- Adopt new FireHPC syntax to define partition parameters
- Rename slurm_partitions > node→nodes to reflect latest FireHPC
  expectations.
Create SSH tunnel to request metrics on prometheus running on
emulated clusters in development environment, except on pocket cluster
to test a cluster without metrics.
@rezib rezib force-pushed the pr/feat-275 branch 6 times, most recently from 7aebd56 to f70729e Compare November 7, 2024 15:58
@rezib rezib linked an issue Nov 7, 2024 that may be closed by this pull request
@rezib rezib marked this pull request as ready for review November 7, 2024 15:58
Add a boolean to indicate if metrics feature is enabled in agent /info
endpoint, in complement of the cluster and infrastructure names.
Return boolean metrics feature flag of every clusters in gateway
/clusters endpoint.
Manage query parameters forwarded to agent more generically in
request_agent() instead of specifically in racksdb view.
This subdirectory will also hold all other upcoming metrics related
modules.
Add dependency on charts.js and luxon adapter and draw charts with
timeseries metrics.
Add possibility to query metrics from Prometheus in agent.

This commit introduces slurmweb.metrics.db module with SlurmwebMetricsDB
class designed to query nodes, cores and jobs metrics, with various
range and resolution in PromQL (Prometheus Query Language) from
a Prometheus database. This class is instanciated in agent application
when metrics feature is enabled, its method are called by metrics view,
accessible through /v<version>/metrics/<metric> API endpoint.

Note that SlurmwebAppAgent.metrics attribute is renamed
metrics_collector to avoid name conflict with new metrics_db attribute.
Proxy metrics requests to agent through
/api/agents/<cluster>/metrics/<metric> enpoint.
Display charts of resources (nodes/cores) status and jobs queue in
dashboard page based on metrics from Prometheus, when metrics feature is
enabled.

fix #275
Set jest-canvas-mock as a development dependency and use it to setup
vitest to mock canvas element in NodeJS when running unit tests.
@rezib rezib merged commit 2889297 into main Nov 11, 2024
9 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Nov 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature New feature or enhancement to develop.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Time-series charts
1 participant