v1.0.3: Last version, but ParlAI-stable

Overview

v1.0.2 shipped with a bug that broke ParlAI-chat related tasks. This release only includes the fix for that (#859), but is otherwise equivalent to v1.0.2.

v1.0.2: Key Bugfixes and QoL Upgrades

Overview

Some issues still remaining from v1.0.0 (and even sooner) have been ironed out, so this release aims to fix those as well as include some new QoL upgrades.

Highlights

• A status mismatch issue related to collection slowdowns on long-running tasks has been squashed (#830)
• Adding testing for our example files (#795, #800)
• Upgrading completely to Poetry as our package manager (#779)
• Core work for new worker addons package, to be included in coming release notes (#791, #785)

Additional Changes

• MTurk improvements and fixes:
  • #810: Functionality to specify the lifetime of a task
  • #780: Fix to HIT tags that were being improperly submitted as individual characters
  • #767: Fix to storing and restoring AWS credentials
• Bugfix for deploying to the HerokuArchitect via git (#824)
• Locking down the version of Node that our router uses to solve issues deploying on the EC2Architect (#762)
• Cleanup to the Operator.shutdown method to allow disconnected onboardings to be cleared (#769)
• Developer QoL:
  • #814: Consolidating the AgentState classes and introducing a metadata attribute
  • #770: Making Agent and OnboardingAgent pull from the same base class for code deduplication
• Bugfixes to the RemoteProcedureBlueprint:
  • #763: Fixing an issue that prevented reloading from restoring the state
  • #798, #782: Cleaning up on the RemoteProcedure examples
  • #784: Adding a new RemoteProcedure example using a text model-in-the-loop

1.0.1 Release: minor 1.0.0 Bugfixes

Overview

Following the 1.0.0 release, a few bugs were surfaced. The largest involved an update of mephisto-task already published, and this release syncs the underlying Mephisto library to refer to the same version.

Full Changelog

• #728 Fixes an issue preventing live tasks from working over https
• #749 Adds back some missing mephisto-task package versions that were preventing correct builds at runtime
• #756 Fixes a small bug in the HTMLStaticBlueprint
• #745 Clarifying correct typing for data passed via handleSubmit in our StaticBlueprint tasks
• #754 fixes docker imaging issue if mephisto metrics install has already been run locally
• #747 Set some defaults for max_num_concurrent_tasks to encourage people to use this option more consistently

Mephisto 1.0.0 Release

This PR captures all of the work the team has done for Mephisto up to now, setting our first stable API, prioritizing user-friendly features, and providing a massively updated documentation site. Read more about this release on the blog.

EC2 Architect, Documentation, and Various Bugfixes

Overall this update doesn't change any existing semantics for Mephisto, but includes some new functionality (including launches on EC2 instead of Heroku!) and some small bugfixes. In total, this release contains the following:

• EC2 architect and followup fixes (#603, #606, #609, #618, #622)
• Documentation overhaul (#611, #612, #613)
• Early implementation for using Gold labelling worker quality control (#585)
• Bugfixes for existing blueprints (#588, #594, #637)
• Various lint and dependency updates

Hydra and ParlAI fixes, `BlueprintMixin`s.

Minor version update

This release primarily serves to help with the transition from hydra 1.0 to 1.1, though also includes some additional fixes, especially for ParlAI tasks.
This release also includes BlueprintMixins, which serve as an easier way to inject first-class features into blueprints, and to provide shared functionality across otherwise disjoint tasks.
Key changes:

• Blueprint Mixin refactor, and the ScreenTaskRequired mixin (#582, #573, #566)
• Compensation HIT launch script (#586)
• PyPI Release prep (#548)
• ParlAI Onboarding conversion fix, and other bugfixes (#573, #554, #555)
• JS Package reorg (#565, #574, #570)
• Hydra migration fixes (#587)
• Other bugfixes (#535, #547, #560)

Hydra 1.1 upgrade

Notices

• The upgrade to Hydra 1.1 (#494) means that previous scripts may not function properly. Remediation steps for this are included in #529.
• The old method of directly initializing Mephisto data model elements (such as Units and Agents) is being changed to <Class>.get(db, db_id) rather than a direct constructor. Both are supported for now, but the old version will be deprecated in 1.0.

Contents

• This version release primarily focuses around our upgrade to hydra 1.1 (#494).
• Hydra 1.1 means we can now specify profiles for common launch patterns between jobs (#496).
• Deprecating direct data model initialization to move off of metaclasses in 1.0 (#437).
• Stability improvements - better state tracking for MTurk units (#527)
• UX improvements - easier local development of static react tasks (#509), centralizing review utilities for scripts (#526), better messaging for unexpected behaviors (#528, #530)

Dependency Updates, Bugfixes

This release is fairly minor, including fixes to Botocore and a number of other underlying dependencies.

Also included are #486 and #489, which aim to improve detection of disconnects with the HerokuArchitect and clarify messaging about heroku setup failures.

Speedup Improvements + More

Summary

This release focuses on Mephisto speedups, including fixes to reduce the runtime of many common operations. Most of these are further reduced by using the (BETA) MephistoSingletonDB. In combination, these reduce Mephisto's cpu usage by 90% or more for the same number of active tasks, allowing for increased concurrency.

It also includes a number of bugfixes and stability improvements since January

Included changes

• max_num_concurrent_units option for all Mephisto tasks #405
• Hook to register callbacks after the completion of a task #339
• Speedups for interactions requiring the MTurkProvider #435
• Speedups for overall Mephisto operations by no longer loading AgentState for every Agent, preventing unnecessary data reads. #341
• Significant speedups using the MephistoSingletonDB #432 #441
• Other speed-related fixes #440
• Flask option for Heroku router #398
• Various improvements to HerokuArchitect #403 #413 #411 #418
• MTurk related bugfixes #382 #385 #445
• Other bugfixes #400 #421 #410 #439 #443 #456

Deprecating ParlAI-MTurk

This version removes any references to the old ParlAI-MTurk implementation, as it is soon to be removed entirely from the ParlAI repo.