[Reporting] New UI for migrating reporting indices ILM policy #103853

jloleysens · 2021-06-30T09:53:08Z

Summary

This contribution adds new UI to the reporting plugin for guiding users to migrate their reporting indices to the reporting-provided ILM policy. Introduced in #103850.

How to test

Start Kibana + ES on basic and activate your trial to access all features
In Kibana's start up logs you should see Creating ILM policy for managing reporting indices... if this is the first time starting with the new changes. Subsequent starts should contain a debug callout that this step is being skipped.
Navigate to the "Reporting" plugin, on a fresh start this page should be empty, no callout present.
Create a report by going to the "dashboard" app and using the "Share" menu to generate a PDF report
Navigate to the "Reporting" plugin, there should still be no callout present since all of your indices are up-to-date
Run the following request, unsetting the ILM policy managing reporting indices:

curl --request PUT \
  --url 'http://localhost:9200/.reporting-*/_settings' \
  --header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
  --header 'Content-Type: application/json' \
  --data '{
	"settings": {
		"index.lifecycle.name": null
	}
}'

Go to the reporting plugin, the callout to migrate indices should be present (as well as the link to the kibana-reporting ILM policy in the bottom right), clicking it should migrate all reporting indices to use the reporting ILM policy.
Re-run the request to unset the ILM policies and delete the kibana-reporting ILM policy, can be done through the ILM policy UI
Navigate to the Reporting UI, this time the banner should be present and the link to the ILM policy should be removed
Press the primary button in the call-to-action
The callout should be gone and the link to the ILM policy should be back (clicking this link takes you to the ILM UI)

Notes

There was jest snapshot that was captured against UI that was in "loading" state. The changes introduced here await the end of the loading state and so the snapshot is populated with a lot more UI than before. This is the primary source of new lines.

Screenshots

Call-to-action to migrate reporting indices

Successful migration of reporting indices

Note the success callout
Note the link to "Edit ILM policy" in the bottom right

ILM policy exists, but migration is still needed

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
We could overwrite an existing ILM policy	Low	Med	This requires that a user has a policy called `kibana-reporting` and that there is a bug in our code that will re-create the policy. Further mitigations will be achieved by adding a functional API test

* restructured files for the policy manager code * re-added the IlmPolicyStatusResponse interface

in one * added ilm policy context and context for api client * added es ui shared so we can use userequest

…ermining when to show ilm-policy link

…g component

jloleysens · 2021-06-30T10:13:10Z

@elasticmachine merge upstream

elasticmachine · 2021-06-30T13:25:21Z

Pinging @elastic/kibana-app-services (Team:AppServices)

elasticmachine · 2021-06-30T13:25:22Z

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

…-policy-2 * 'master' of github.com:elastic/kibana: [Reporting] Reintroduce "ILM policy for managing reporting indices" (elastic#103850) [Security Solution][Endpoint] Allow activity log scrolling on small screens (elastic#103852) Allow zero (0) to unset unenroll_timeout field (elastic#103790) [TSVB] Metric count is depicted as `-` instead of 0 (elastic#103717) [Query] Es query/field base (elastic#103177) Remove add data button from nav (elastic#103810) Fix telemetry advanced setting style (elastic#103838) [Transform] Fix default naming and sorting fields suggestion for `top_metrics` agg (elastic#103690) [APM] use conventional error rate color for correlations (elastic#103500) # Conflicts: # x-pack/plugins/reporting/server/lib/store/store.ts

jloleysens · 2021-07-01T08:15:51Z

@elasticmachine merge upstream

tsullivan · 2021-07-01T19:05:51Z

...porting/public/management/migrate_ilm_policy_callout/ilm_policy_migration_needed_callout.tsx

+      onMigrationDone();
+      toasts.addSuccess({ title: i18nTexts.migrateSuccessTitle });
+    } catch (e) {
+      toasts.addError(e, { title: i18nTexts.migrateErrorTitle });


Suggested change

toasts.addError(e, { title: i18nTexts.migrateErrorTitle });

toasts.addError(e, {

title: i18nTexts.migrateErrorTitle,

toastMessage: e.body?.message,

});

tsullivan

Just a small suggestion on showing more info about the error in the toast message

tsullivan

This is great - I'm really happy with how this turned out.

I tested the default behavior, and it works well.

I also tested what happens when there is a custom ILM policy for reporting in place before Kibana runs with this change.

(I had to do some fiddling with the code to prevent kibana-reporting policy from being set, which is already happening in master but not 7.x)

Create a custom ILM policy, link it to an index template, create a report. The list of policies should show 1 linked index.
Run Kibana with this change. The new kibana-reporting policy is created:
Go to Stack Management > Reporting. You see the "Reporting indices are not managed by the same ILM policy" notice.
Click the "Migrate Indices" button. A success toast message pops up.
Back in the ILM listing, kibana-reporting is linked to 1 index and the custom policy is linked to 0

This is what I expected should happen - there should only be 1 policy for the indices.

Then I tried:

Delete the current reporting index
Generate another report
In the ILM policy listing, the kibana-reporting policy is the only one that shows its linked to an index

Big 👍 !!

jloleysens · 2021-07-05T08:01:14Z

@elasticmachine merge upstream

jloleysens · 2021-07-05T12:20:52Z

Any follow up feedback will be handled in follow up PRs!

kibanamachine · 2021-07-05T12:23:52Z

💔 Backport failed

Status	Branch	Result
❌	7.x	Commit could not be cherrypicked due to conflicts

To backport manually run:
node scripts/backport --pr 103853

…c#103853) * revert revert * wip; first iteration of server-side functionality * wip; public side code started. using the new ilm policy status endpoint * * refactored ReportingIlmPolicyManager -> IlmReportingManager * restructured files for the policy manager code * re-added the IlmPolicyStatusResponse interface * added the ability to use useRequest * * removed extra server-side endpoint, we do both migration steps in one * added ilm policy context and context for api client * added es ui shared so we can use userequest * added working link to ILM policy * refactor of the migration route response (again), added logic for determining when to show ilm-policy link * fix type issues and refactor to use testbed pattern for report listing component * added tests for base cases of ilm policy banner * remove non-existent prop * added minor fixes (handling 404 from ES) and added API integration tests * resolve merge conflict and remove unused file * update toast error message Co-authored-by: Kibana Machine <[email protected]>

… (#104312) * revert revert * wip; first iteration of server-side functionality * wip; public side code started. using the new ilm policy status endpoint * * refactored ReportingIlmPolicyManager -> IlmReportingManager * restructured files for the policy manager code * re-added the IlmPolicyStatusResponse interface * added the ability to use useRequest * * removed extra server-side endpoint, we do both migration steps in one * added ilm policy context and context for api client * added es ui shared so we can use userequest * added working link to ILM policy * refactor of the migration route response (again), added logic for determining when to show ilm-policy link * fix type issues and refactor to use testbed pattern for report listing component * added tests for base cases of ilm policy banner * remove non-existent prop * added minor fixes (handling 404 from ES) and added API integration tests * resolve merge conflict and remove unused file * update toast error message Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Kibana Machine <[email protected]>

kibanamachine · 2021-07-06T16:04:19Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`reporting`	51	59	+8

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`reporting`	131	132	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`reporting`	64.9KB	70.4KB	+5.5KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`reporting`	54.4KB	56.6KB	+2.2KB

Unknown metric groups

API count

id	before	after	diff
`reporting`	132	133	+1

History

💚 Build #136289 succeeded 9c2773b
💚 Build #136265 succeeded 4b26074
💚 Build #135904 succeeded 3579254
💛 Build #135905 was flaky 9ab2f28
💔 Build #135580 failed 3797167

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

debadair · 2021-07-09T21:49:09Z

I think we want to avoid overloading the term migrate, since that action has a specific meaning within the context of ILM. Instead, I'd focus on applying the new policy:

Apply new lifecycle policy for reports

Not all reporting indices are using the recommended ILM policy.
To ensure your reports are managed consistently,
all reporting indices should use the kibana-reporting policy.

Apply kibana-reporting policy

For the success toast, I'd focus on the resolution of the problem:

Recommended policy active for all reporting indices

I'd specify "reporting" in the edit link--without the context of the migration prompt, it seems too generic:

Edit reporting ILM policy

jloleysens added 10 commits June 29, 2021 15:13

revert revert

3f56f07

wip; first iteration of server-side functionality

68b9d19

wip; public side code started. using the new ilm policy status endpoint

73d0972

* refactored ReportingIlmPolicyManager -> IlmReportingManager

3a8b218

* restructured files for the policy manager code * re-added the IlmPolicyStatusResponse interface

added the ability to use useRequest

ad145c8

* removed extra server-side endpoint, we do both migration steps

df25813

in one * added ilm policy context and context for api client * added es ui shared so we can use userequest

added working link to ILM policy

4abd092

refactor of the migration route response (again), added logic for det…

93c5c03

…ermining when to show ilm-policy link

fix type issues and refactor to use testbed pattern for report listin…

ee87c68

…g component

added tests for base cases of ilm policy banner

bbda9f9

jloleysens added release_note:enhancement v8.0.0 Team:AppServices auto-backport Deprecated - use backport:version if exact versions are needed v7.15.0 labels Jun 30, 2021

jloleysens requested a review from tsullivan June 30, 2021 09:53

jloleysens mentioned this pull request Jun 30, 2021

[Reporting] Reintroduce "ILM policy for managing reporting indices" #103850

Merged

kibanamachine and others added 3 commits June 30, 2021 06:13

Merge branch 'master' into reporting/ilm-policy-2

619f315

remove non-existent prop

7f4b518

added minor fixes (handling 404 from ES) and added API integration tests

508c947

jloleysens marked this pull request as ready for review June 30, 2021 13:25

jloleysens added 2 commits June 30, 2021 15:27

resolve merge conflict and remove unused file

3579254

Merge branch 'master' into reporting/ilm-policy-2

9ab2f28

tsullivan reviewed Jul 1, 2021

View reviewed changes

tsullivan approved these changes Jul 1, 2021

View reviewed changes

kibanamachine and others added 2 commits July 5, 2021 04:01

Merge branch 'master' into reporting/ilm-policy-2

4b26074

update toast error message

9c2773b

jloleysens merged commit 4c448c9 into elastic:master Jul 5, 2021

jloleysens deleted the reporting/ilm-policy-2 branch July 5, 2021 12:21

jloleysens mentioned this pull request Jul 5, 2021

[7.x] [Reporting] New UI for migrating reporting indices ILM policy (#103853) #104312

Merged

jloleysens mentioned this pull request Jul 6, 2021

[Reporting] ILM migration callout should only be visible to users who can action it #104472

Closed

jloleysens mentioned this pull request Jul 12, 2021

[Reporting] Updated copy for reporting ILM policy callout #105184

Merged

sophiec20 added the (Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead label Aug 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Reporting] New UI for migrating reporting indices ILM policy #103853

[Reporting] New UI for migrating reporting indices ILM policy #103853

jloleysens commented Jun 30, 2021 •

edited

Loading

jloleysens commented Jun 30, 2021

elasticmachine commented Jun 30, 2021

elasticmachine commented Jun 30, 2021

jloleysens commented Jul 1, 2021

tsullivan Jul 1, 2021

tsullivan left a comment

tsullivan left a comment •

edited

Loading

jloleysens commented Jul 5, 2021

jloleysens commented Jul 5, 2021

kibanamachine commented Jul 5, 2021

kibanamachine commented Jul 6, 2021

API count

debadair commented Jul 9, 2021

[Reporting] New UI for migrating reporting indices ILM policy #103853

[Reporting] New UI for migrating reporting indices ILM policy #103853

Conversation

jloleysens commented Jun 30, 2021 • edited Loading

Summary

How to test

Notes

Screenshots

Call-to-action to migrate reporting indices

Successful migration of reporting indices

ILM policy exists, but migration is still needed

Checklist

Risk Matrix

jloleysens commented Jun 30, 2021

elasticmachine commented Jun 30, 2021

elasticmachine commented Jun 30, 2021

jloleysens commented Jul 1, 2021

tsullivan Jul 1, 2021

Choose a reason for hiding this comment

tsullivan left a comment

Choose a reason for hiding this comment

tsullivan left a comment • edited Loading

Choose a reason for hiding this comment

jloleysens commented Jul 5, 2021

jloleysens commented Jul 5, 2021

kibanamachine commented Jul 5, 2021

💔 Backport failed

kibanamachine commented Jul 6, 2021

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

History

debadair commented Jul 9, 2021

jloleysens commented Jun 30, 2021 •

edited

Loading

tsullivan left a comment •

edited

Loading