fix: [nvbug5300494] Use runtime total gpu memory to calculate kv cache memory and log more memory information #4660
Conversation
|
/bot run |
1 similar comment
|
/bot run |
|
PR_Github #6467 [ run ] triggered by Bot |
|
PR_Github #6467 [ run ] completed with state |
|
/bot run |
|
PR_Github #6510 [ run ] triggered by Bot |
|
/bot run --disable-fail-fast |
|
PR_Github #6514 [ run ] triggered by Bot |
|
PR_Github #6510 [ run ] completed with state |
HuiGao-NV
changed the title
|
PR_Github #6514 [ run ] completed with state |
|
/bot run --disable-fail-fast |
|
PR_Github #6586 [ run ] triggered by Bot |
|
PR_Github #6586 [ run ] completed with state |
|
/bot run --stage-list="H100_PCIe-PyTorch-3" |
|
PR_Github #6651 [ run ] triggered by Bot |
|
PR_Github #6651 [ run ] completed with state |
Change method to compute peak memory Set new peak memory for case test_ptq_quickstart_advanced_mtp Get non-torch memory of starttime of kv memory estimation Signed-off-by: Hui Gao <[email protected]>
|
/bot skip --comment="CI has passed." |
|
PR_Github #6687 [ skip ] triggered by Bot |
|
PR_Github #6687 [ skip ] completed with state |
…e memory and log more memory information (NVIDIA#4660) Signed-off-by: Hui Gao <[email protected]>
…e memory and log more memory information (NVIDIA#4660) Signed-off-by: Hui Gao <[email protected]>
3 participants
Use runtime total gpu memory to calculate kv cache memory and log more memory information.
This can avoid mis-reporting of less kv memory.