Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Report broad failures in daily CI builds #12408

Merged
merged 18 commits into from
Jan 31, 2025
Merged

Report broad failures in daily CI builds #12408

merged 18 commits into from
Jan 31, 2025

Conversation

mrodm
Copy link
Contributor

@mrodm mrodm commented Jan 21, 2025 •
edited
Loading

Proposed commit message

Update ReportFailedTests to report daily CI builds with broad failures creating GitHub issues, or updating them in case there was an existing one.

Example of GitHub issue that would be created:

  • Subject: [Stack 9.0.0-SNAPSHOT] Too many packages failing in daily job
  • Labels:
    • flaky-test
    • automation
    • Team:Ecosystem
  • Description:
- Stack version: 9.0.0-SNAPSHOT
- Packages:
    - coredns
    - envoyproxy
    - gitlab
    - jamf_pro
    - juniper_junos
    - juniper_netscreen
    - microsoft_exchange_server
    - mongodb
    - ping_federate
    - ti_custom
    - tychon
- Owners:
    - @elastic/ecosystem



First build failed: https://buildkite.com/elastic/integrations/builds/20805

Author's Checklist

How to test this PR locally

  1. Create some XML files (jUnit failures) or download them from a build job into build/test-results folder.
  2. Ensure there are more than CI_MAX_TESTS_REPORTED failing tests.
  3. Run the tool with DRY_RUN mode enabled:
DRY_RUN=true CI_MAX_TESTS_REPORTED=2 BUILDKITE_BUILD_URL=https://buildkite.com/elastic/integrations/builds/1 SERVERLESS=false STACK_VERSION=9.0.0-SNAPSHOT mage -v ReportFailedTests build/test-result

Related issues

@mrodm mrodm self-assigned this Jan 21, 2025
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Jan 21, 2025 •
edited
Loading

🚀 Benchmarks report

Package 1password 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit_events 4484.3 3067.48 -1416.82 (-31.6%) 💔

Package activemq 👍(1) 💚(3) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
broker 62500 52631.58 -9868.42 (-15.79%) 💔

Package apache 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
access 4524.89 2652.52 -1872.37 (-41.38%) 💔
error 5494.51 4201.68 -1292.83 (-23.53%) 💔

Package apache_tomcat 👍(2) 💚(5) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
catalina 11235.96 3484.32 -7751.64 (-68.99%) 💔
memory 26315.79 22222.22 -4093.57 (-15.56%) 💔

Package arista_ngfw 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 1872.66 1490.31 -382.35 (-20.42%) 💔

Package atlassian_jira 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 3174.6 2564.1 -610.5 (-19.23%) 💔

Package authentik 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
group 6896.55 5524.86 -1371.69 (-19.89%) 💔

Package aws 👍(12) 💚(0) 💔(7)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
guardduty 973.71 766.87 -206.84 (-21.24%) 💔
rds 19230.77 16129.03 -3101.74 (-16.13%) 💔
s3access 4310.34 3484.32 -826.02 (-19.16%) 💔
vpcflow 6451.61 4716.98 -1734.63 (-26.89%) 💔
apigateway_logs 10000 8333.33 -1666.67 (-16.67%) 💔
elb_logs 4566.21 3773.58 -792.63 (-17.36%) 💔
firewall_logs 3597.12 2673.8 -923.32 (-25.67%) 💔

Package awsfirehose 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
metrics 12658.23 9708.74 -2949.49 (-23.3%) 💔

Package azure 👍(6) 💚(3) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
identity_protection 4237.29 3300.33 -936.96 (-22.11%) 💔
platformlogs 5128.21 4329 -799.21 (-15.58%) 💔

Package azure_metrics 👍(1) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
container_instance 333333.33 250000 -83333.33 (-25%) 💔

Package azure_network_watcher_vnet 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 1615.51 1190.48 -425.03 (-26.31%) 💔

Package azure_openai 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
logs 1615.51 1170.96 -444.55 (-27.52%) 💔
metrics 250000 100000 -150000 (-60%) 💔

Package barracuda 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
waf 701.26 548.55 -152.71 (-21.78%) 💔

Package barracuda_cloudgen_firewall 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 10638.3 9009.01 -1629.29 (-15.32%) 💔

Package bbot 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
asm_intel 3802.28 2061.86 -1740.42 (-45.77%) 💔

Package beyondinsight_password_safe 👍(2) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
managedaccount 2906.98 2314.81 -592.17 (-20.37%) 💔

Package canva 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 751.88 582.41 -169.47 (-22.54%) 💔

Package ceph 👍(2) 💚(3) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
cluster_health 33333.33 20000 -13333.33 (-40%) 💔
cluster_status 7042.25 5494.51 -1547.74 (-21.98%) 💔

Package checkpoint_email 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
event 3058.1 2092.05 -966.05 (-31.59%) 💔

Package cisco_meraki 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
events 500000 250000 -250000 (-50%) 💔

Package cisco_meraki_metrics 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
device_health 71428.57 43478.26 -27950.31 (-39.13%) 💔

Package cisco_secure_endpoint 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
event 3333.33 2242.15 -1091.18 (-32.74%) 💔

Package citrix_adc 👍(2) 💚(0) 💔(4)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
interface 5988.02 3289.47 -2698.55 (-45.07%) 💔
lbvserver 7812.5 6060.61 -1751.89 (-22.42%) 💔
log 874.89 720.46 -154.43 (-17.65%) 💔
system 6289.31 4975.12 -1314.19 (-20.9%) 💔

Package cloud_security_posture 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
vulnerabilities 200000 166666.67 -33333.33 (-16.67%) 💔

Package cloudflare_logpush 👍(12) 💚(3) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
network_analytics 3095.98 2375.3 -720.68 (-23.28%) 💔
audit 5586.59 4651.16 -935.43 (-16.74%) 💔
dns 9803.92 7352.94 -2450.98 (-25%) 💔

Package couchbase 👍(7) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
xdcr 27027.03 20833.33 -6193.7 (-22.92%) 💔

Package crowdstrike 👍(3) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
falcon 12195.12 9174.31 -3020.81 (-24.77%) 💔

Package cyberarkpas 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 1519.76 1267.43 -252.33 (-16.6%) 💔
monitor 11235.96 3968.25 -7267.71 (-64.68%) 💔

Package cybereason 👍(2) 💚(2) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
logon_session 1814.88 1524.39 -290.49 (-16.01%) 💔
suspicions_process 1497.01 1219.51 -277.5 (-18.54%) 💔

Package elasticsearch 👍(3) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
server 6756.76 3496.5 -3260.26 (-48.25%) 💔

Package entityanalytics_entra_id 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
entity 9615.38 6944.44 -2670.94 (-27.78%) 💔

Package fireeye 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
nx 2217.29 1626.02 -591.27 (-26.67%) 💔

Package forgerock 👍(3) 💚(4) 💔(4)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
am_activity 14705.88 9345.79 -5360.09 (-36.45%) 💔
am_core 13157.89 7194.24 -5963.65 (-45.32%) 💔
idm_access 9259.26 7042.25 -2217.01 (-23.94%) 💔
idm_activity 23809.52 16666.67 -7142.85 (-30%) 💔

Package fortinet_fortiedr 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 753.01 639.8 -113.21 (-15.03%) 💔

Package gcp 👍(2) 💚(3) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
vpcflow 4608.29 2923.98 -1684.31 (-36.55%) 💔

Package google_workspace 👍(7) 💚(6) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
saml 7936.51 4830.92 -3105.59 (-39.13%) 💔
context_aware_access 3861 2976.19 -884.81 (-22.92%) 💔

Package hadoop 👍(0) 💚(2) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
application 5952.38 4098.36 -1854.02 (-31.15%) 💔
cluster 6369.43 4329 -2040.43 (-32.03%) 💔
namenode 11627.91 8849.56 -2778.35 (-23.89%) 💔

Package haproxy 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 2283.11 1926.78 -356.33 (-15.61%) 💔

Package hashicorp_vault 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 4201.68 2932.55 -1269.13 (-30.21%) 💔
log 11235.96 5617.98 -5617.98 (-50%) 💔

Package iis 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
access 2304.15 1782.53 -521.62 (-22.64%) 💔

Package infoblox_bloxone_ddi 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
dns_config 1216.55 930.23 -286.32 (-23.54%) 💔

Package istio 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
access_logs 2061.86 1506.02 -555.84 (-26.96%) 💔

Package logstash 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 5714.29 4524.89 -1189.4 (-20.81%) 💔
slowlog 8620.69 5181.35 -3439.34 (-39.9%) 💔

Package m365_defender 👍(3) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 5208.33 4048.58 -1159.75 (-22.27%) 💔

Package menlo 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
dlp 3571.43 2277.9 -1293.53 (-36.22%) 💔

Package microsoft_dnsserver 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
analytical 3278.69 2762.43 -516.26 (-15.75%) 💔
audit 16666.67 13513.51 -3153.16 (-18.92%) 💔

Package microsoft_exchange_server 👍(3) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
messagetracking 10638.3 8771.93 -1866.37 (-17.54%) 💔

Package microsoft_sqlserver 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
performance 6211.18 3787.88 -2423.3 (-39.02%) 💔

Package mimecast 👍(3) 💚(1) 💔(7)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
ttp_ip_logs 12658.23 10638.3 -2019.93 (-15.96%) 💔
ttp_url_logs 12658.23 7575.76 -5082.47 (-40.15%) 💔
archive_search_logs 9174.31 6896.55 -2277.76 (-24.83%) 💔
dlp_logs 10638.3 7692.31 -2945.99 (-27.69%) 💔
siem_logs 3164.56 2604.17 -560.39 (-17.71%) 💔
threat_intel_malware_customer 6944.44 3058.1 -3886.34 (-55.96%) 💔
threat_intel_malware_grid 6134.97 3389.83 -2745.14 (-44.75%) 💔

Package mongodb_atlas 👍(3) 💚(0) 💔(5)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
disk 14705.88 8196.72 -6509.16 (-44.26%) 💔
mongod_audit 8130.08 5649.72 -2480.36 (-30.51%) 💔
mongod_database 20000 14492.75 -5507.25 (-27.54%) 💔
process 2227.17 1745.2 -481.97 (-21.64%) 💔
project 3289.47 2457 -832.47 (-25.31%) 💔

Package nagios_xi 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
service 2967.36 2403.85 -563.51 (-18.99%) 💔

Package netflow 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 11627.91 8620.69 -3007.22 (-25.86%) 💔

Package netscout 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
sightline 100000 22222.22 -77777.78 (-77.78%) 💔

Package o365 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 2150.54 1757.47 -393.07 (-18.28%) 💔

Package o365_metrics 👍(10) 💚(4) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
teams_user_activity_user_counts 10526.32 4444.44 -6081.88 (-57.78%) 💔

Package oracle 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
database_audit 23255.81 9615.38 -13640.43 (-58.65%) 💔

Package osquery 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
result 23255.81 17241.38 -6014.43 (-25.86%) 💔

Package panw_metrics 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
system 24390.24 18518.52 -5871.72 (-24.07%) 💔

Package ping_federate 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
admin 2028.4 1602.56 -425.84 (-20.99%) 💔
audit 3787.88 2358.49 -1429.39 (-37.74%) 💔

Package ping_one 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 2631.58 2127.66 -503.92 (-19.15%) 💔

Package platform_observability 👍(0) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
kibana_log 13333.33 8928.57 -4404.76 (-33.04%) 💔

Package prisma_access 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
event 592.07 429.18 -162.89 (-27.51%) 💔

Package proxysg 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
log 4366.81 2314.81 -2052 (-46.99%) 💔

Package qualys_vmdr 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
asset_host_detection 993.05 761.61 -231.44 (-23.31%) 💔

Package rapid7_insightvm 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
vulnerability 4484.3 3174.6 -1309.7 (-29.21%) 💔

Package rubrik 👍(2) 💚(1) 💔(5)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
filesets 12820.51 10869.57 -1950.94 (-15.22%) 💔
managed_volumes 20833.33 13513.51 -7319.82 (-35.14%) 💔
physical_hosts 41666.67 24390.24 -17276.43 (-41.46%) 💔
tasks 43478.26 31250 -12228.26 (-28.12%) 💔
virtualmachines 50000 40000 -10000 (-20%) 💔

Package salesforce 👍(2) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
setupaudittrail 13513.51 9433.96 -4079.55 (-30.19%) 💔

Package servicenow 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
event 4739.34 3533.57 -1205.77 (-25.44%) 💔

Package slack 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 2923.98 2252.25 -671.73 (-22.97%) 💔

Package sophos_central 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alert 1118.57 919.96 -198.61 (-17.76%) 💔

Package spring_boot 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
gc 7936.51 6711.41 -1225.1 (-15.44%) 💔

Package symantec_edr_cloud 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
incident 2415.46 1904.76 -510.7 (-21.14%) 💔

Package system 👍(0) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
security 2688.17 1949.32 -738.85 (-27.49%) 💔
syslog 333333.33 166666.67 -166666.66 (-50%) 💔

Package tanium 👍(1) 💚(2) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
endpoint_config 15873.02 8064.52 -7808.5 (-49.19%) 💔
reporting 15151.52 12500 -2651.52 (-17.5%) 💔
threat_response 1007.05 801.92 -205.13 (-20.37%) 💔

Package tenable_io 👍(2) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
plugin 2777.78 2277.9 -499.88 (-18%) 💔
scan 17857.14 12820.51 -5036.63 (-28.21%) 💔

Package tenable_sc 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
vulnerability 1831.5 1472.75 -358.75 (-19.59%) 💔

Package ti_anomali 👍(1) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
threatstream 4784.69 3816.79 -967.9 (-20.23%) 💔

Package ti_crowdstrike 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
intel 1721.17 1199.04 -522.13 (-30.34%) 💔
ioc 4566.21 3184.71 -1381.5 (-30.25%) 💔

Package ti_eset 👍(1) 💚(0) 💔(6)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
apt 1760.56 1019.37 -741.19 (-42.1%) 💔
botnet 8196.72 5813.95 -2382.77 (-29.07%) 💔
cc 10526.32 6097.56 -4428.76 (-42.07%) 💔
domains 11235.96 6535.95 -4700.01 (-41.83%) 💔
files 10416.67 7042.25 -3374.42 (-32.39%) 💔
ip 13333.33 6666.67 -6666.66 (-50%) 💔

Package ti_maltiverse 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
indicator 2793.3 2053.39 -739.91 (-26.49%) 💔

Package ti_rapid7_threat_command 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
vulnerability 3484.32 2444.99 -1039.33 (-29.83%) 💔

Package ti_threatq 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
threat 2793.3 2183.41 -609.89 (-21.83%) 💔

Package tines 👍(0) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit_logs 4065.04 3164.56 -900.48 (-22.15%) 💔

Package trellix_epo_cloud 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
device 2739.73 2314.81 -424.92 (-15.51%) 💔

Package trend_micro_vision_one 👍(1) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alert 1663.89 924.21 -739.68 (-44.45%) 💔
audit 6451.61 4273.5 -2178.11 (-33.76%) 💔

Package tychon 👍(12) 💚(6) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
cmrs 41666.67 29411.76 -12254.91 (-29.41%) 💔

Package vsphere 👍(3) 💚(3) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
cluster 28571.43 22222.22 -6349.21 (-22.22%) 💔
resourcepool 43478.26 20408.16 -23070.1 (-53.06%) 💔

Package wiz 👍(2) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
audit 3012.05 2544.53 -467.52 (-15.52%) 💔

Package zeek 👍(24) 💚(13) 💔(6)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
known_certs 66666.67 45454.55 -21212.12 (-31.82%) 💔
ntlm 22222.22 8771.93 -13450.29 (-60.53%) 💔
ntp 52631.58 40000 -12631.58 (-24%) 💔
dce_rpc 15873.02 11494.25 -4378.77 (-27.59%) 💔
dnp3 20408.16 14285.71 -6122.45 (-30%) 💔
dns 27777.78 20408.16 -7369.62 (-26.53%) 💔

Package zerofox 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alerts 3690.04 2169.2 -1520.84 (-41.21%) 💔

Package zscaler_zia 👍(5) 💚(1) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
alerts 4347.83 3154.57 -1193.26 (-27.44%) 💔
audit 3597.12 2941.18 -655.94 (-18.24%) 💔

Package zscaler_zpa 👍(4) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
app_connector_status 2380.95 1538.46 -842.49 (-35.38%) 💔

To see the full report comment with /test benchmark fullreport

mrodm
mrodm commented Jan 22, 2025
View reviewed changes
.buildkite/pipeline.yml Outdated
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes of this file must be reverted before merging the PR.

@mrodm
Copy link
Contributor Author

mrodm commented Jan 23, 2025

Updated description with an example of the issue (subject, labels and description) that would be created.

@mrodm mrodm marked this pull request as ready for review January 23, 2025 17:25
@mrodm mrodm requested a review from a team as a code owner January 23, 2025 17:25
jsoriano
jsoriano reviewed Jan 30, 2025
View reviewed changes
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

dev/testsreporter/builderror.go Outdated
buildReportingTeamLabel = "Team:Ecosystem"
)

type buildError struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might extract the common fields with packageError to some common struct. Not sure though if there would be big benefits.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added common struct dataError in 561a22e

dev/testsreporter/testsreporter.go Outdated Show resolved Hide resolved
dev/testsreporter/reporter.go Outdated Show resolved Hide resolved
@mrodm mrodm requested a review from jsoriano January 30, 2025 16:42
@elasticmachine
Copy link

elasticmachine commented Jan 30, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

cc @mrodm

@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@mrodm mrodm merged commit b9f2d0a into elastic:main Jan 31, 2025
4 of 5 checks passed
@mrodm mrodm deleted the report-broad-failures branch January 31, 2025 12:15
harnish-elastic pushed a commit to harnish-elastic/integrations that referenced this pull request Feb 4, 2025
@mrodm
Report broad failures in daily CI builds (elastic#12408)
d90c907 
Update ReportFailedTests to report daily CI builds with broad
failures creating GitHub issues, or updating them in case
there was an existing one. A broad failure depends on the value
of the environment variable CI_MAX_TESTS_REPORTED
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jsoriano jsoriano jsoriano approved these changes

Assignees

@mrodm mrodm

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mrodm @elasticmachine @jsoriano