Article
Fixing Gradient Accumulation
By
and 5 others
•
•
60Joseph Lee
jiosephlee
AI & ML interests
None yet
Recent Activity
liked
a Space
about 2 months ago
transformers-community/support
new activity
about 2 months ago
transformers-community/support:Clarification on Recent Changes to Loss and Gradient Accumulation
updated
a model
about 2 months ago
jiosephlee/therapeutic_fine_tuning_10M_v2
Organizations
None yet
upvoted
an
article
about 2 months ago
upvoted
2
articles
2 months ago
Article
Improving Hugging Face Training Efficiency Through Packing with Flash Attention
By
and 5 others
•
•
40
Article
Efficient LLM Pretraining: Packed Sequences and Masked Attention
By
•
•
48
upvoted
2
articles
3 months ago
Article
Good answers are not necessarily factual answers: an analysis of hallucination in leading LLMs
By
and 1 other
•
•
41
Article
Saving Memory Using Padding-Free Transformer Layers during Finetuning
By
•
•
19