Hitl dashbaord model viz improve

[mialsy]

Description

• Update frontend/backend support for visualizing the model:
  - Based on @ethan-carlson 's suggestion, changed from showing loss and accuracy of validation and text_span to showing loss and accuracy separately; and show training loss/accuracy & validation loss/accuracy in the same graph.
  - Updated the graph visualization container to be responsive to browser window changes.
• Added new API to get batch ids for runs that has the models need to be visualized in a pipeline.

Type of change

Please check the options that are relevant.

• Bug fix (non-breaking change that fixes an issue)
• Proposes a change (non-breaking change that isn't necessarily a bug)
• Refactor
• New feature (non-breaking change that adds a new functionality)
• Breaking change (fix or feature that would break some existing functionality downstream)
• This is a unit test
• Documentation only change
• Datasets Release
• Models Release

Type of requested review

• I want a thorough review of the implementation.
• I want a high level review.
• I want a deep design review.

Before and After

API changes:

• get_best_model_loss_acc_by_id
  • Before: this API returns model loss and accuracy for validation and the text_span.
  • After: returns loss and accuracy for validation and training.

Added new API:

• get_best_model_bids_by_pipeline
  • get best models' batch ids for a specific pipeline
  • input:
    • pipeline name (can be nlu, tao or vision)
  • output:
    • a list of the batch ids of the best models

Demo:

• Demo:
  - Best models loss & accuracy for a pipeline:
  -
  - Best model in a run (showing loss and accuracy seperately, graph responsive to browser window changes):
  -

Testing

Manually tested.

Checklist:

• I have performed manual end-to-end testing of the feature in my environment.
• I have added Docstrings and comments to the code.
• I have made changes to existing documentation where needed.
• I have added tests that show that the PR is functional.
• New and existing unit tests pass locally with my changes.
• I have added relevant collaborators to review the PR before merge.
• [Polymetis only] I ran on hardware (1) all scripts in tests/scripts, (2) asv benchmarks.

[ethan-carlson]

This looks great, thank you!

One small nit - could we move the hard coded styling parameters to be constants listed at the top of the file? Or a stylesheet if used multiple places.

[mialsy]

Updated model visualization based on @snyxan 's comment, can I get review for this again?

[facebook-github-bot]

Hi @mialsy!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!