Configure OpenSearch Dashboard

[bofstein]

Create filters, views in OpenSearch
Talk to Terry and Jing and Scott who set dashboards up before

• Not sure what exactly we need until we talk to them and start to use them

Ticket is for initial configuration; will update more over time

• 500 and 400 error codes
• Usage rates (can possibly pull from Matomo too)
• Other key metrics the other people suggest that did this already
• API usage vs website usage

[jupiter007]

I think that based on my research into creating a dashboard in OpenSearch in this ticket (687), creating a dashboard shouldn't be too difficult as long as the data is available. I think making sure we have the data that we need in our logs, and indexing them, will be the more time consuming part.

As part of the research in #687, I saved some data off of the ezid index patterns, and then created visualizations for them and added them to my test dashboard juliet-viz-test.

We will want to make sure that our logs contain:

• status_codes (500 errors, 400 errors, 300 codes, etc)
• A function field that will help us track what activities are happening on what parts our site
• error type field, like Forbidden, Bad User Input, Unauthorized, etc.
• graph showing which countries the requests are coming from

[briri]

thanks for this analysis @jupiter007. Terry sent me links to the changes he made for his Rails app.

I can confirm that we will have the status code by default. I'll look into what it would take to include the function field and error type.

I'm not sure we'll be able to show the country info. I think at most we would be able to include the IP of the caller, but that may be getting swallowed up by the load balancer and our apache/rails logs may only have the IP of the ELB

[jupiter007]

thanks for this analysis @jupiter007. Terry sent me links to the changes he made for his Rails app.

I can confirm that we will have the status code by default. I'll look into what it would take to include the function field and error type.

I'm not sure we'll be able to show the country info. I think at most we would be able to include the IP of the caller, but that may be getting swallowed up by the load balancer and our apache/rails logs may only have the IP of the ELB

Yea, I actually don't think we need the country info after all.

[jupiter007]

@briri I just noticed that I added slightly different notes at the OpenSearch Research ticket: #687

It would be great if we could get metrics like response times for pages

[briri]

Cool. The Lograge gem looks like it includes response times by default (full duration, view render duration and db query duration):
method=GET path=/ format=json controller=HomeController action=index status=200 duration=79.0 view=78.8 db=0.0

Beyond the above basics, I am going to have it include:

• current user id
• the params in the request (e.g. template id, plan id, form inputs, etc.
• the request timestamp
• the caller IP (still not sure if that will be the IP of the ELB or the actual caller's machine ... will need to wait until we deploy to a stg)