Skip to content

Commit

Permalink
Merge pull request #1102 from facebookresearch/form-builder-app
Browse files Browse the repository at this point in the history
Mephisto v1.3
  • Loading branch information
meta-paul authored Feb 27, 2024
2 parents 5176bf0 + 9bad7f3 commit 2dd08af
Show file tree
Hide file tree
Showing 240 changed files with 53,662 additions and 2,930 deletions.
Binary file removed ._data
Binary file not shown.
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@ examples/simple_static_task/hydra_configs/conf/*
!examples/simple_static_task/hydra_configs/conf/prolific_example.yaml
!examples/simple_static_task/hydra_configs/conf/onboarding_example*.yaml
examples/**/build/*
examples/form_composer_demo/preview/*_preview.html

# Form Composer
mephisto/generators/form_composer/data/*.json
mephisto/generators/form_composer/webapp/package-lock.json
mephisto/generators/form_composer/hydra_configs/conf/*.yaml
!mephisto/generators/form_composer/hydra_configs/conf/default.yaml

# PyCharm
.idea
Expand Down
4 changes: 4 additions & 0 deletions docker/entrypoints/server.mturk.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,12 @@
set -e

pip install pytest

cd /mephisto
# Directory for Task results
mkdir -p "data" && chmod 777 "data"
# Directory for Cypress testing
mkdir -p "/root/.cache/cypress" && chmod 777 "/root/.cache/cypress"

mephisto register mturk_sandbox \
name=$MTURK_SANDBOX_NAME \
Expand Down
4 changes: 4 additions & 0 deletions docker/entrypoints/server.prolific.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,12 @@
set -e

pip install pytest black

cd /mephisto
# Directory for Task results
mkdir -p "data" && chmod 777 "data"
# Directory for Cypress testing
mkdir -p "/root/.cache/cypress" && chmod 777 "/root/.cache/cypress"

mephisto register prolific name=prolific api_key=$PROLIFIC_API_KEY

Expand Down
12 changes: 11 additions & 1 deletion docker/envs/env.dev
Original file line number Diff line number Diff line change
@@ -1 +1,11 @@
PROLIFIC_API_KEY="dev"
PROLIFIC_API_KEY="..."

MTURK_SANDBOX_NAME="..."
MTURK_SANDBOX_ACCESS_KEY_ID="..."
MTURK_SANDBOX_SECRET_ACCESS_KEY="..."

AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_DEFAULT_REGION=...

S3_URL_EXPIRATION_MINUTES=60
10 changes: 5 additions & 5 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@

### Customization

- Are you interested in a simple task with barebones HTML files? Reference the **[simple static task](../examples/static_static_task/)**.
- Are you interested in a simple task with barebones HTML files? Reference the **[simple static task](/examples/static_static_task/)**.

- Are you interested in building a rich frontend? Learn about how you can create a **[simple React task](../examples/static_react_task/)**.
- Reference the docs for the **[`mephisto-task` helper package](../packages/mephisto-task/README.md)** for interfacing with the Mephisto server.
- You may also want to use components offered in the **[`annotation-toolkit` component library](../packages/annotation-toolkit/README.md)** to get up-and-running quick. The components of the library can be **[demoed in the storybook](https://annotation-toolkit-storybook.vercel.app/)**.
- Are you interested in building a rich frontend? Learn about how you can create a **[simple React task](/examples/static_react_task/)**.
- Reference the docs for the **[`mephisto-task` helper package](/packages/mephisto-task/README.md)** for interfacing with the Mephisto server.
- You may also want to use components offered in the **[`annotation-toolkit` component library](/packages/annotation-toolkit/README.md)** to get up-and-running quick. The components of the library can be **[demoed in the storybook](https://annotation-toolkit-storybook.vercel.app/)**.

- Are you interested in dialogue-based tasks using the ParlAI framework? You may want to check out the **[example ParlAI demo task](../examples/parlai_chat_task_demo/)**. For UI customizations, we heavily use the **[`bootstrap-chat` helper package](../packages/bootstrap-chat/)**, so you may want to reference the documentation for that as well.
- Are you interested in dialogue-based tasks using the ParlAI framework? You may want to check out the **[example ParlAI demo task](/examples/parlai_chat_task_demo/)**. For UI customizations, we heavily use the **[`bootstrap-chat` helper package](/packages/bootstrap-chat/)**, so you may want to reference the documentation for that as well.
118 changes: 118 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
-->

# Examples

The Mephisto example folders within contain some sample starter code and tasks to demonstrate potential workflows for setting up and working on new tasks.

Mephisto Tasks can be launched (each run is called TaskRun) with a single `docker-compose` command (you will need to have Docker [installed](https://docs.docker.com/engine/install/).)
Expand Down Expand Up @@ -70,6 +71,123 @@ A more complex example featuring worker-generated dynamic input: [mnist](/exampl

---

#### 3. Dynamic form-based task

You can create and modify auto-composed form-based tasks using FormComposer example (its full specs are in this [README](/mephisto/generators/form_composer/README.md)).

There are three FormComposer examples:

##### 3.1. Simple form

This is a single-version form containing no variable tokens.

- Default config file: [example_local_mock.yaml](/examples/form_composer_demo/hydra_configs/conf/example_local_mock.yaml).
- Launch command:
```shell
docker-compose -f docker/docker-compose.dev.yml run \
--build \
--publish 3001:3000 \
--rm mephisto_dc \
python /mephisto/examples/form_composer_demo/run_task.py
```
- Browser page (for the first task unit): [http://localhost:3001/?worker_id=x&assignment_id=1](http://localhost:3001/?worker_id=x&assignment_id=1).
- You should see a Bootstrap-themed form with fields and sections corresponding to the form config in its [task_data.json](/examples/form_composer_demo/data/simple/task_data.json) file.

##### 3.2. Dynamic form

Dynamic form means a multi-version form, where versions are generated by varying values of special tokens embedded into the form config.

- Default config file: [dynamic_example_local_mock.yaml](/examples/form_composer_demo/hydra_configs/conf/dynamic_example_local_mock.yaml).
- Launch command:
```shell
docker-compose -f docker/docker-compose.dev.yml run \
--build \
--publish 3001:3000 \
--rm mephisto_dc \
python /mephisto/examples/form_composer_demo/run_task_dynamic.py
```
- Browser page is same as for the Simple form example

There are variations of dynamic form config that use different providers. To try that, change `run_task_dynamic.py` in the launch command to:

- `run_task_dynamic_ec2_prolific.py` for Prolific (requires valid EC2 credentials)
- `run_task_dynamic_ec2_mturk_sandbox.py` for Mechanical Turk sandbox (requires valid EC2 credentials)

##### 3.3. Dynamic form with presigned URLs

This example builds further upon the Dynamic form example. Here we use presigned URLs (i.e. short-lived URLs for S3 AWS files) in the displayed forms, for data security.

- Set up environment variables (in file `docker/envs/env.dev`):
- Required: valid AWS credentials: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION`
- Required: a private S3 bucket with files that you will embed in the example form (either replace dummy URLs in the configs by hand, or automatically generate new ones with `mephisto form_composer_config` command)
- Optional: `S3_URL_EXPIRATION_MINUTES` (default value is 60 minutes)
- Default config file: [dynamic_example_local_mock.yaml](/examples/form_composer_demo/hydra_configs/conf/dynamic_example_local_mock.yaml).
- Create config: see all options for `form_composer_config` command [here](/mephisto/generators/form_composer/README.md#using-formcomposerconfig-utility). Example command:
```shell
docker-compose -f docker/docker-compose.dev.yml run \
--build \
--publish 3001:3000 \
--rm mephisto_dc \
mephisto form_composer_config --verify --directory "/mephisto/examples/form_composer_demo/data/dynamic_presigned_urls"
```
- Launch command after you generated all configs:
```shell
docker-compose -f docker/docker-compose.dev.yml run \
--build \
--publish 3001:3000 \
--rm mephisto_dc \
python /mephisto/examples/form_composer_demo/run_task_dynamic_presigned_urls.py
```


# End-to-end example with AWS

Putting it altogether, let's prepare and launch a task featuring a form containing one embedded file plus a few other fields. Here we'll assume working in directory `/mephisto/examples/form_composer_demo/data/dynamic_presigned_urls`.

- Adjust `dynamic_presigned_urls_example_ec2_prolific.yaml` task config as needed
- Create `form_config.json` file to define your form fields and layout
- it should contain a token named `file_location`
- for more details see `mephisto/generators/form_composer/README.md`
- Create `separate_token_values_config.json` with desired token values
- Specify your AWS credentials
- Create file `docker/aws_credentials` and populate it with AWS keys info (for infrastructure and Mturk)
- Populate your AWS credentials into `docker/envs/env.local` file (for presigning S3 URLs)
- Clone file `docker/docker-compose.dev.yml` as `docker/docker-compose.local.yml`, and point its `env_file` to `envs/env.local`
- Remove content of folder `/tmp` (if you didn't shut the previous Task run correctly)
- Launch docker containers: `docker-compose -f docker/docker-compose.local.yml up`
- SSH into the running container: `docker exec -it mephisto_dc bash`
- Generate your task data config with these commands:
```shell
mephisto form_composer_config \
--directory "/mephisto/examples/form_composer_demo/data/dynamic_presigned_urls" \
--update-file-location-values "https://your-bucket.s3.amazonaws.com/..." \
--use-presigned-urls

mephisto form_composer_config \
--directory "/mephisto/examples/form_composer_demo/data/dynamic_presigned_urls" \
--permutate-separate-tokens

mephisto form_composer_config \
--directory "/mephisto/examples/form_composer_demo/data/dynamic_presigned_urls" \
--extrapolate-token-sets

mephisto form_composer_config \
--directory "/mephisto/examples/form_composer_demo/data/dynamic_presigned_urls" \
--verify
```
- Launch your task:
```shell
cd /mephisto/examples/form_composer_demo && python run_task_dynamic_presigned_urls_ec2_prolific.py
```
- After the Task is completed by all workers, launch task review app and acces it at [http://localhost:8081](http://localhost:8081) (for more details see `mephisto/review_app/README.md`):
```shell
mephisto review_app -h 0.0.0.0 -p 8000 -d True -f True
```

_Note: if a package build was terminated/failed, or related source code was changed, FormComposer needs to be rebuilt with this command: `mephisto scripts form_composer rebuild_all_apps`._

---

# Your Mephisto project

To read on steps for creating your own custom Mephisto task, please refer to README in the main Mephisto repo.
32 changes: 32 additions & 0 deletions examples/form_composer_demo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
These form-based questionnaires are example of FormComposer task generator.

---

## How to run

1. In repo root, launch containers: `docker-compose -f docker/docker-compose.dev.yml up`
2. SSH into running container to run server: `docker exec -it mephisto_dc bash`
3. Inside the container, run the project with either of these commands:
- Simple form: `cd /mephisto/examples/form_composer_demo && python ./run_task.py`
- Dynamic form: `cd /mephisto/examples/form_composer_demo && python ./run_task_dynamic.py`
- Dynamic form with Prolific on EC2: `cd /mephisto/examples/form_composer_demo && python ./run_task_dynamic_ec2_prolific.py`
- Dynamic form with Mturk on EC2: `cd /mephisto/examples/form_composer_demo && python ./run_task_dynamic_ec2_mturk_sandbox.py`

---

## How to configure

1. For simple form config you need to provide FormComposer with one JSON file - a configuration of your form fields. An example is found in `examples/form_composer_demo/data/simple/task_data.json` file.
2. For dynamic form configs you need two JSON files in `examples/form_composer_demo/data/dynamic` directory:
- Form configuration `form_config.json`
- Token sets values `token_sets_values_config.json`
- To generate extrapolated `task_data.json` config, run this command: `mephisto form_composer_config --extrapolate-token-sets True`
- Note that `task_data.json` file will be overwritten with the resulting config
3. To generate `token_sets_values_config.json` file from token values permutations in `separate_token_values_config.json`, run this command: `mephisto form_composer_config --permutate-separate-tokens`
- Note that `token_sets_values_config.json` file will be overwriten with new sets of tokens values

---

#### Form config

For details on how form config is composed, and how its data fields are validated, please see the main FormComposer's [README.md](/mephisto/generators/form_composer/README.md).
File renamed without changes.
Loading

0 comments on commit 2dd08af

Please sign in to comment.