Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Return sentences instead of individual tokens for text features during inference #3504

Closed
wants to merge 9 commits into from
Closed

Return sentences instead of individual tokens for text features during inference #3504

wants to merge 9 commits into from

Conversation

arnavgarg1
Copy link
Contributor

@arnavgarg1 arnavgarg1 commented Aug 7, 2023 •
edited
Loading

Some questions:

  1. Should we skip probabilities during LLM inference?
  2. Should we only do this for LLMs?

@arnavgarg1 arnavgarg1 changed the title Return sentences instead of individual tokens for text features Return sentences instead of individual tokens for text features during inference Aug 7, 2023
@github-actions
Copy link

github-actions bot commented Aug 7, 2023 •
edited
Loading

Unit Test Results

  6 files  ±0    6 suites  ±0   1h 9m 56s ⏱️ +10s
34 tests ±0  29 ✔️ ±0    5 💤 ±0  0 ❌ ±0 
88 runs  ±0  72 ✔️ ±0  16 💤 ±0  0 ❌ ±0 

Results for commit afcae60. ± Comparison against base commit 8fa3bc6.

♻️ This comment has been updated with latest results.

@arnavgarg1 arnavgarg1 closed this Aug 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinxzhao justinxzhao justinxzhao left review comments

@tgaddair tgaddair tgaddair approved these changes

@w4nderlust w4nderlust Awaiting requested review from w4nderlust w4nderlust will be requested when the pull request is marked ready for review w4nderlust is a code owner

@geoffreyangus geoffreyangus Awaiting requested review from geoffreyangus geoffreyangus will be requested when the pull request is marked ready for review geoffreyangus is a code owner

@jeffkinnison jeffkinnison Awaiting requested review from jeffkinnison jeffkinnison will be requested when the pull request is marked ready for review jeffkinnison is a code owner

@Infernaught Infernaught Awaiting requested review from Infernaught Infernaught will be requested when the pull request is marked ready for review Infernaught is a code owner

@alexsherstinsky alexsherstinsky Awaiting requested review from alexsherstinsky alexsherstinsky will be requested when the pull request is marked ready for review alexsherstinsky is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@arnavgarg1 @tgaddair @justinxzhao