CLI Testing Tool for Parsing Results to Standard Output

[adreichert]

Summary

This PR adds a simple CLI tool that wraps the LlamaParse constructor.

It is intended to make testing easier. It's useful for quickly parsing files to see the results, visually comparing different models, or quickly inspecting JSON output to see the different fields. Hopefully, it will also help people get started with the tool more quickly. It is not intended to be complete. I've included the options I vary most of the time.

• The output JSON is typically passed to jq -r
• I've been using is for the past several months.

I think others would benefit. I have a more complicated version that will fetch past jobs' results or status using job ids. I'll add more functionality to this file if this PR is approved.

Example Usage

python -m llama_parse.tool parse foo.pdf

Testing

Help Message

python -m llama_parse.tool parse --help

Usage: python -m llama_parse.tool parse [OPTIONS] FILE

  Parse the given file and output the result to the STDOUT

  All supported arguments match those of the LlamaParse constructor. Please
  refer to the official documentation for more information.

Options:
  --api-key <api-key>             Defaults to $LLAMA_CLOUD_API_KEY
  --vendor-multimodal-model-name <model>
  --vendor-multimodal-api-key <vendor-api-key>
  --invalidate-cache
  --result-type <result-type>
  --help                          Show this message and exit.

Example Script

Everything parsed

python -m llama_parse.tool parse \
    $FILE \
    --invalidate-cache > ~/Desktop/a.md

python -m llama_parse.tool parse \
    $FILE \
    --result-type='text' \
    --invalidate-cache > ~/Desktop/a.txt

python -m llama_parse.tool parse \
    $FILE \
    --result-type='json' \
    --invalidate-cache > ~/Desktop/a.json

python -m llama_parse.tool parse \
    $FILE \
    --vendor-multimodal-model-name='openai-gpt4o' \
    --vendor-multimodal-api-key=$VENDOR_KEY \
    --invalidate-cache > ~/Desktop/b.md


python -m llama_parse.tool parse \
    $FILE \
    --result-type='json' \
    --api-key=$LLAMA_CLOUD_API_KEY \
    --vendor-multimodal-model-name='openai-gpt4o' \
    --vendor-multimodal-api-key=$VENDOR_KEY \
    --invalidate-cache > ~/Desktop/b.json

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[logan-markewich]

This looks great! Thanks for the contribution

A few notes:

1. It looks like click wasn't added to the toml dependencies, so any one using this will run into an import error if it's not installed
2. It seems like not every option is included (which is totally fine, there's tons). Maybe @hexapode can comment on if any should be added
3. We can actually make this usable on the command line without needing python -m -- check out this example: https://stackoverflow.com/questions/59286983/how-to-run-a-script-using-pyproject-toml-settings-and-poetry
4. Let's add a section in the readme for CLI usage

[adreichert]

• Updated Readme
• Added Click
• Add CLI command lp-tool

[adreichert]

@logan-markewich Any feedback?

[adreichert]

ResultType.JSON was removed. I'll update the code to address this.

[logan-markewich]

@adreichert actually -- I pushed a rather large refactor lol

[logan-markewich]

Feel free to revert, but its mostly ensuring that the LlamaParse CLI is in sync with the actual client in terms of options

[adreichert]

I'm not going to make any changes. Thank you for including a CLI. I appreciate your time.