Skip to content

Conversation

@pcanal
Copy link
Member

@pcanal pcanal commented Dec 9, 2023

This was reported at art-framework-suite/art#142 and is due to the combination of
(a) We do not generate dictionary for std::tuple instances (b) When TClass::GetClass is called it tries to load the dictonary until there is a full TClass object is in memory (c) The emulated std::tuple TClass are marked as 'not loaded' (d) Searching for the TClass for a templated class will cost memory (during the lookup of the instantiation). (e) TPluginManager::ExecPluginImpl was looking up the TClass for the typle std::type< list of arguments>

The lookup induced in (e) in the user's case (root built with runtime cxx module on) lead to some memory allocation in Clang while trying to find out if there was now a library or dictionary to load.

This fixes #14199

This was reported at art-framework-suite/art#142
and is due to the combination of
(a) We do not generate dictionary for std::tuple instances
(b) When TClass::GetClass is called it tries to load the dictonary until there is a full TClass object is in memory
(c) The emulated std::tuple TClass are marked as 'not loaded'
(d) Searching for the TClass for a templated class will cost memory (during the lookup of the instantiation).
(e) TPluginManager::ExecPluginImpl was looking up the TClass for the typle `std::type< list of arguments>`

The lookup induced in (e)  in the user's case (root built with runtime cxx module on)
lead to some memory allocation in Clang while trying to find out if there was now
a library or dictionary to load.
@phsft-bot
Copy link

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default
How to customize builds

@github-actions
Copy link

github-actions bot commented Dec 9, 2023

Test Results

       10 files         10 suites   2d 2h 18m 0s ⏱️
  2 484 tests   2 476 ✔️ 0 💤 8
23 763 runs  23 755 ✔️ 0 💤 8

For more details on these failures, see this check.

Results for commit 5bf3016.

♻️ This comment has been updated with latest results.

@pcanal pcanal marked this pull request as draft December 11, 2023 08:42
@phsft-bot
Copy link

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default
How to customize builds

@pcanal pcanal marked this pull request as ready for review December 11, 2023 10:07
@pcanal
Copy link
Member Author

pcanal commented Dec 12, 2023

The New CI failures seems unrelated.

@ferdymercury ferdymercury modified the milestones: 6.30/02, 6.30.04 Dec 18, 2023
@greenc-FNAL
Copy link
Contributor

Patch was successfully back-ported to 6.30/02 and reported by Mu2e as resolving art-framework-suite/art#142

Copy link
Member

@dpiparo dpiparo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes! LGTM. I am sure this will help in cases different from Mu2e.

@hahnjo
Copy link
Member

hahnjo commented Jan 22, 2024

It's hard to pin down, but looking through the commit history and its incremental builds it seems that this PR made tutorial-multicore-imt101_parTreeProcessing very likely to fail on Windows 64-bit...

FYI @bellenot

@bellenot
Copy link
Member

bellenot commented Jan 22, 2024

I confirm that these changes break tutorial-multicore-imt101_parTreeProcessing on Windows, with different error messages:

768: Processing C:/root-dev/git/master/tutorials/multicore/imt101_parTreeProcessing.C...
768: Assertion failed: LexLevel == 0 && "cannot use lookahead while lexing", file C:\root-dev\git\master\interpreter\llvm-project\clang\include\clang/Lex/Preprocessor.h, line 1874
768: Assertion CMake Error at C:/root-dev/build/x64/debug/RootTestDriver.cmake:232 (message):
768:   error code: Exit code 0xc0000409


768: Processing C:/root-dev/git/master/tutorials/multicore/imt101_parTreeProcessing.C...
768: Assertion failed: CurDiagID == std::numeric_limits<unsigned>::max() && "Multiple diagnostics in flight at once!", file C:\root-dev\git\master\interpreter\llvm-project\clang\include\clang/Basic/Diagnostic.h, line 1545
768: Assertion failed: (*I)->isCompleCMake Error at C:/root-dev/build/x64/debug/RootTestDriver.cmake:232 (message):
768:   error code: Exit code 0xc0000409


768: Processing C:/root-dev/git/master/tutorials/multicore/imt101_parTreeProcessing.C...
768: Assertion failed: (!m_Consumer->getTransaction() || (m_Consumer->getTransaction() == T)) && "Cannot release different T", file C:\root-dev\git\master\interpreter\cling\lib\Interpreter\IncrementalParser.cpp, line 538
768: CMake Error at C:/root-dev/build/x64/debug/RootTestDriver.cmake:232 (message):
768:   error code: Exit code 0xc0000409


768: Processing C:/root-dev/git/master/tutorials/multicore/imt101_parTreeProcessing.C...
768: Assertion failed: CurLexerKind != CLK_CachingLexer && "already in caching lex mode", file C:\root-dev\git\master\interpreter\llvm-project\clang\lib\Lex\PPCaching.cpp, line 99
768: Assertion failed: CCMake Error at C:/root-dev/build/x64/debug/RootTestDriver.cmake:232 (message):
768:   error code: Exit code 0xc0000409


768: Processing C:/root-dev/git/master/tutorials/multicore/imt101_parTreeProcessing.C...
768: Assertion failed: is(tok::raw_identifier), file C:\root-dev\git\master\interpreter\llvm-project\clang\include\clang/Lex/Token.h, line 208
768: Assertion failed: (CMake Error at C:/root-dev/build/x64/debug/RootTestDriver.cmake:232 (message):
768:   error code: Exit code 0xc0000409


768: Processing C:/root-dev/git/master/tutorials/multicore/imt101_parTreeProcessing.C...
768: Assertion failed: DiagID - DIAG_UPPER_LIMIT < DiagInfo.size() && "Invalid diagnostic ID", file C:\root-dev\git\master\interpreter\llvm-project\clang\lib\Basic\DiagnosticIDs.cpp, line 379
768: Assertion failed: CurLexerCMake Error at C:/root-dev/build/x64/debug/RootTestDriver.cmake:232 (message):
768:   error code: Exit code 0xc0000409


768: Processing C:/root-dev/git/master/tutorials/multicore/imt101_parTreeProcessing.C...
768: Assertion failed: CurLexer && "Got EOF but no current lexer set!", file C:\root-dev\git\master\interpreter\llvm-project\clang\lib\Lex\PPLexerChange.cpp, line 531
768: ACMake Error at C:/root-dev/build/x64/debug/RootTestDriver.cmake:232 (message):
768:   error code: Exit code 0xc0000409

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory hoarding triggered by the TPluginManager

7 participants