Skip to content
Merged
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
8d00b73
float8 tmp save
weifengpy Jun 12, 2024
68d9f61
Merge branch 'main' into fsdp2
weifengpy Jun 19, 2024
4cd5f74
run 8b eager successfully
weifengpy Jun 19, 2024
05a4a06
enable compile
weifengpy Jun 20, 2024
f48a82e
benchmark
weifengpy Jun 20, 2024
14aabfb
1d setup
weifengpy Jun 21, 2024
b88aee9
2d setup
weifengpy Jun 21, 2024
2b4e0c2
2d setup
weifengpy Jun 21, 2024
ad63aba
Merge branch 'main' into fsdp2
weifengpy Jul 3, 2024
71d4dc6
Merge branch 'main' into fsdp2
weifengpy Jul 11, 2024
23536e9
fp8 all-gather FSDP
weifengpy Jul 12, 2024
bdb0fd0
linter
weifengpy Jul 12, 2024
ef0e843
add unit test and restore original toml
weifengpy Jul 12, 2024
c294f6a
add unit test for float8
weifengpy Jul 12, 2024
b58b07b
better doc with original dtype all-gather and value error on fp8
weifengpy Jul 15, 2024
7df10ae
improve config msg
weifengpy Jul 15, 2024
f674012
Merge branch 'pytorch:main' into fsdp2
weifengpy Jul 15, 2024
7dd788c
rename config to enable_fp8_linear and improve comments
weifengpy Jul 16, 2024
faefe27
rename to enable_fp8_linear
weifengpy Jul 16, 2024
7aad066
add 2D test
weifengpy Jul 16, 2024
5040c31
import Optional and NotImplement for delayed scaling
weifengpy Jul 16, 2024
cee653e
remove TP fp8 all-gather from CI
weifengpy Jul 16, 2024
e164285
fix linter
weifengpy Jul 16, 2024
22c71ea
remove redudant check
weifengpy Jul 16, 2024
595f83d
install float8_experimental in CI
weifengpy Jul 16, 2024
68e9f19
Merge branch 'pytorch:main' into fsdp2
weifengpy Jul 16, 2024
9de67ff
import float8_experimental inside enable_fp8_linear
weifengpy Jul 16, 2024
367507f
import float8_experimental only when needed
weifengpy Jul 17, 2024
de3de0e
Merge branch 'pytorch:main' into fsdp2
weifengpy Jul 17, 2024
1ed8dab
skip CI on non-H100 GPUs
weifengpy Jul 17, 2024
2be380d
warning about sm90
weifengpy Jul 17, 2024
eb7d801
Merge branch 'pytorch:main' into fsdp2
weifengpy Jul 17, 2024
1952e4b
Merge branch 'pytorch:main' into fsdp2
weifengpy Jul 18, 2024
121acae
add torch.compile + FSDP2 float8 all-gather to CI
weifengpy Jul 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions test_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,18 @@ def build_test_list():
"FSDP2 with float8 all-gather and precomputed dynamic scales",
"fsdp2_float8_all_gather_precompute_dynamic_scales",
),
OverrideDefinitions(
[
[
"--training.enable_float8_linear",
"--training.enable_fsdp_float8_all_gather",
"--training.precompute_float8_dynamic_scale_for_fsdp",
"--training.compile",
]
],
"FSDP2 with float8 all-gather and precomputed dynamic scales",
"fsdp2_float8_all_gather_precompute_dynamic_scales_compile",
),
OverrideDefinitions(
[
[
Expand Down