3outeille/transformers backend (Dense model only)#2048

Merged

tianyu-l merged 132 commits intopytorch:mainfrom

huggingface:3outeille/transformers_backend

Nov 20, 2025

Commits on Sep 6, 2025

Commits on Sep 8, 2025

define own apply_ac for transformer backend instead of reusing llama3
3outeille
committed

Commits on Sep 9, 2025

HF model without any parallelism now train (but grad_norm is high)
3outeille
committed

Commits on Sep 10, 2025

a bit cleaner way to get passed args
3outeille
committed
now same number of params + same attention backend but noticed higher gradnorm and less tps with HF model
3outeille
committed

Commits on Sep 11, 2025

Commits on Sep 15, 2025

Commits on Sep 18, 2025

Commits on Sep 22, 2025

Commits on Sep 23, 2025

beginning parallelism by setting tests
3outeille
committed

Commits on Sep 25, 2025

make FSDP work in a cleaner way (mapping instead of renaming)
3outeille
committed

Commits on Sep 26, 2025

Commits on Sep 30, 2025

Commits on Oct 1, 2025

breaking test
3outeille
committed

Commits on Oct 4, 2025

refactor test
3outeille
committed

Commits on Oct 5, 2025

Commits on Oct 9, 2025

now everything works (1D/2D/3D/4D). need to fix correctness with PP
3outeille
committed

Commits on Oct 14, 2025

fix and uniformize weight init of llama-like model + various fix
3outeille
committed

Commits on Oct 17, 2025

Merge branch 'main' into 3outeille/transformers_backend
3outeille
committed

Commits on Oct 30, 2025

refactor args to make it clearer
3outeille
committed

Commits on Oct 31, 2025

Commits on Nov 1, 2025

Commits on Nov 3, 2025

Commits on Nov 5, 2025

Commits on Nov 7, 2025

fix loading weights in PP by using Module Dict
3outeille
committed

Commits on Nov 18, 2025

Commits on Nov 20, 2025

fix minor stuff
3outeille
committed