Update SFT QLoRA notebook with **14B** model on free Colab #4336
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| } | ||
| ], | ||
| "source": [ | ||
| "prompt = hf_tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n", |
There was a problem hiding this comment.
I think you can use llm.chat directly here
| "import torch\n", | ||
| "\n", | ||
| "llm = LLM(\n", | ||
| " model=f\"sergiopaniego/{output_dir}-merged\", # Replace with your HF username or organization\n", |
There was a problem hiding this comment.
no need to do it right now but in the future it could be nice to centralize every artifact in hf.co/trl-lib
|
For some reason the new trackio space doesn't render (the previous one does) 
|
| " **model_inputs,\n", | ||
| " max_new_tokens=512\n", | ||
| ")\n", | ||
| "output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()\n", |
There was a problem hiding this comment.
nit: tolist no needed
| "output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()\n", | |
| "output_ids = generated_ids[0][len(model_inputs.input_ids[0]):]\n", |
|
Updated based on comments! Ready for final review/merge
I've tested it downloading the updated notebook from here and it shows the Space when opening it from Colab or VS Code. Maybe the Space was on sleeping state at that point. |
…sft_qlora_notebook
…sft_qlora_notebook
…_lora_qlora.ipynb`
What does this PR do?
Fine tune a 14B model on free Colab using TRL+SFT!
It already uses this PR so its dependent on it
Before submitting
Pull Request section?
to it if that's the case.
Who can review?
@qgallouedec @kashif