Skip to content

Issues: huggingface/trl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

82 Open 1,142 Closed
82 Open 1,142 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

PPO manual reward functions
#2363 opened Nov 18, 2024 by schmidtj3
Still not supporting for ChatGLM3 maybe
#2362 opened Nov 18, 2024 by fjy01
7 of 9 tasks
Contributing new distillation related trainers
#2361 opened Nov 16, 2024 by YihanCao123
1 of 3 tasks
Question about the logprobs of the policy-generated sentences in PPO trainer
#2358 opened Nov 15, 2024 by yanghh2000
6 of 9 tasks
PPOTrainer with HuggingFace PreTrainedModelWrapper Models
#2357 opened Nov 14, 2024 by Mrinh212375
7 of 9 tasks
3
How to train from scratch? Can you provide the code ❓ question Seeking clarification or more information
#2356 opened Nov 14, 2024 by sankexin
5 of 9 tasks
2
Dpo Train Issue: max step from 1000 to 996349
#2355 opened Nov 14, 2024 by seTalent
8 of 9 tasks
BUG in the new PPO trainer
#2353 opened Nov 13, 2024 by TingchenFu
7 of 9 tasks
2
KTO: unpair_preference_dataset does not work for datasets with additional columns
#2351 opened Nov 13, 2024 by LuisVasquezBSC
RLOO Checkpoint Issue
#2342 opened Nov 11, 2024 by asparius
2 of 4 tasks
3
[Question] Why is Importance Sampling and Clipping applied in RLOO?
#2341 opened Nov 10, 2024 by shashankg7
3
Multiple Errors with PPOTrainer. error in ppo_trainer.dataloader 🐛 bug Something isn't working 🏋 PPO Related to PPO
#2340 opened Nov 10, 2024 by Debolena7
3
Difference between SFTTrainer and Seq2seqTrainer ❓ question Seeking clarification or more information 🏋 SFT Related to SFT
#2339 opened Nov 9, 2024 by Hyfred
RuntimeError: chunk expects at least a 1-dimensional tensor 🐛 bug Something isn't working 🏋 SFT Related to SFT
#2338 opened Nov 8, 2024 by imrankh46
4 tasks done
@kashif
11
DPO Training DataLoader is not shuffled 🏋 DPO Related to DPO ✨ enhancement New feature or request
#2337 opened Nov 7, 2024 by kaiwenw
4 tasks
1
Accelerator package version problem 🐛 bug Something isn't working 🚀 deepspeed Related to deepspeed 🏋 PPO Related to PPO
#2335 opened Nov 7, 2024 by littleshutong
2 of 4 tasks
2
Set gradient_checkpointing_kwargs in the yaml
#2334 opened Nov 7, 2024 by Galaxy-Husky
RLooTrainer bug when using deepspeed 🐛 bug Something isn't working 🚀 deepspeed Related to deepspeed 🏋 RLOO Related to RLOO
#2329 opened Nov 6, 2024 by macheng6
2 of 4 tasks
4
Support for MiniCPM-V Reinforcement Learning with Direct Preference Optimization (DPO) 🏋 DPO Related to DPO ❓ question Seeking clarification or more information 👁️ VLM Related to Visual Language Models
#2326 opened Nov 5, 2024 by DarioPTWR
2
RLOOTrainer ignores custom DataCollatorWithPadding in favor of default one 🐛 bug Something isn't working 🏋 RLOO Related to RLOO
#2309 opened Nov 2, 2024 by anch0vy
1
Using a different ref_model from model leads to incorrect results ✨ enhancement New feature or request ❓ question Seeking clarification or more information
#2307 opened Nov 1, 2024 by DarshanDeshpande
2 of 4 tasks
1
Code migration suggestions 🏋 DPO Related to DPO ⏳ needs more info Additional information or clarification is required to proceed ❓ question Seeking clarification or more information
#2296 opened Oct 30, 2024 by MonolithFoundation
1
OOM when finetuning Llama3.2-90B on 8xA100 80GB
#2294 opened Oct 29, 2024 by maximilianmordig
2 of 4 tasks
@lewtun
wrong objective/entropy in RLOOTrainer 🐛 bug Something isn't working 🏋 RLOO Related to RLOO
#2281 opened Oct 25, 2024 by serendipity800
1 of 4 tasks
@qgallouedec
Feature Request: String-Based Comparison Reward model for RLOOTrainer ✨ enhancement New feature or request 🏋 RLOO Related to RLOO
#2280 opened Oct 25, 2024 by HiroshigeAoki
1
ProTip! Adding no:label will show everything without a label.