Skip to content

Conversation

jeremyd2019
Copy link
Contributor

@jeremyd2019 jeremyd2019 commented May 5, 2025

In #138329, _GNU_SOURCE was added for Cygwin, but when building Clang standalone against an installed LLVM this definition was not picked up, resulting in undefined strnlen. Follow the documentation in https://llvm.org/docs/CMake.html#embedding-llvm-in-your-project and add the LLVM_DEFINITIONS in standalone projects' cmakes.

@llvmbot llvmbot added the clang Clang issues not falling into any other category label May 5, 2025
@llvmbot
Copy link
Member

llvmbot commented May 5, 2025

@llvm/pr-subscribers-lldb
@llvm/pr-subscribers-mlir

@llvm/pr-subscribers-clang

Author: None (jeremyd2019)

Changes

In #138329, _GNU_SOURCE was added for Cygwin, but when building Clang standalone against an installed LLVM this definition was not picked up, resulting in undefined strnlen. Follow the documentation in https://llvm.org/docs/CMake.html#developing-llvm-passes-out-of-source and add the LLVM_DEFINITIONS in Clang's CMakeLists.txt.

Get rid of the list(REMOVE CMAKE_REQUIRED_DEFINITIONS -D_GNU_SOURCE) later, as list(REMOVE) is documented to remove all occurences of the item, not just the one that was just added. Instead, make use of cmake_push_check_state() and cmake_pop_check_state(), which is already used in other cmake files.


Full diff: https://github.com/llvm/llvm-project/pull/138587.diff

1 Files Affected:

  • (modified) clang/CMakeLists.txt (+7-4)
diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index c3f30e2a8e9c0..ab2ac9bc6b9ad 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -68,6 +68,10 @@ if(CLANG_BUILT_STANDALONE)
   option(CLANG_ENABLE_BOOTSTRAP "Generate the clang bootstrap target" OFF)
   option(LLVM_ENABLE_LIBXML2 "Use libxml2 if available." ON)
 
+  separate_arguments(LLVM_DEFINITIONS_LIST NATIVE_COMMAND ${LLVM_DEFINITIONS})
+  add_definitions(${LLVM_DEFINITIONS_LIST})
+  list(APPEND CMAKE_REQUIRED_DEFINITIONS ${LLVM_DEFINITIONS_LIST})
+
   include(AddLLVM)
   include(TableGen)
   include(HandleLLVMOptions)
@@ -183,18 +187,17 @@ check_include_file(sys/resource.h CLANG_HAVE_RLIMITS)
 # This check requires _GNU_SOURCE on linux
 check_include_file(dlfcn.h CLANG_HAVE_DLFCN_H)
 if( CLANG_HAVE_DLFCN_H )
+  include(CMakePushCheckState)
   include(CheckLibraryExists)
   include(CheckSymbolExists)
   check_library_exists(dl dlopen "" HAVE_LIBDL)
+  cmake_push_check_state()
   if( HAVE_LIBDL )
     list(APPEND CMAKE_REQUIRED_LIBRARIES dl)
   endif()
   list(APPEND CMAKE_REQUIRED_DEFINITIONS -D_GNU_SOURCE)
   check_symbol_exists(dladdr dlfcn.h CLANG_HAVE_DLADDR)
-  list(REMOVE_ITEM CMAKE_REQUIRED_DEFINITIONS -D_GNU_SOURCE)
-  if( HAVE_LIBDL )
-    list(REMOVE_ITEM CMAKE_REQUIRED_LIBRARIES dl)
-  endif()
+  cmake_pop_check_state()
 endif()
 
 set(CLANG_RESOURCE_DIR "" CACHE STRING

@jeremyd2019
Copy link
Contributor Author

jeremyd2019 commented May 5, 2025

If this is right, it should probably be done to other standalone-capable projects' CMakeLists.txt also (LLD in particular, for my interests). Actually, it seems there's nothing in LLD that requires _GNU_SOURCE on Cygwin...

@mstorsjo
Copy link
Member

mstorsjo commented May 6, 2025

If this is right, it should probably be done to other standalone-capable projects' CMakeLists.txt also (LLD in particular, for my interests). Actually, it seems there's nothing in LLD that requires _GNU_SOURCE on Cygwin...

Yep, indeed.

I guess the main question is who others regularly build in standalone mode and is familiar with how this is supposed to work, cmake-wise. CC @mgorny @tstellar who seem to have touched things for standalone build mode, and CC @petrhosek for overall cmake review.

@petrhosek
Copy link
Member

I'd slightly prefer doing this as two separate changes, that is the first change to replace the use of list(REMOVE ...) with CMakePushCheckState and the second change to use LLVMConfig.cmake's LLVM_DEFINITIONS since these are technically orthogonal.

I agree that the second change, that is using LLVMConfig.cmake's LLVM_DEFINITIONS, should be applied to other projects that support standalone builds like LLD.

@mstorsjo
Copy link
Member

mstorsjo commented May 6, 2025

I'd slightly prefer doing this as two separate changes, that is the first change to replace the use of list(REMOVE ...) with CMakePushCheckState and the second change to use LLVMConfig.cmake's LLVM_DEFINITIONS since these are technically orthogonal.

Sounds reasonable!

I agree that the second change, that is using LLVMConfig.cmake's LLVM_DEFINITIONS, should be applied to other projects that support standalone builds like LLD.

Ok, so this is an oversight, and it just happens that none of the projects that support standalone builds have been built in configurations where the defines in LLVM_DEFINITIONS have been essential so far?

@jeremyd2019
Copy link
Contributor Author

jeremyd2019 commented May 6, 2025

I agree that the second change, that is using LLVMConfig.cmake's LLVM_DEFINITIONS, should be applied to other projects that support standalone builds like LLD.

Is there a list somewhere or do I just rely on my grep-foo?

  • bolt
  • clang
  • flang
  • lld
  • lldb
  • mlir
  • polly

grep also turned up these, but they don't seem like things that link to LLVM, so probably shouldn't have these defines.

  • compiler-rt
  • libc
  • libclc
  • llvm-libgcc

@jeremyd2019
Copy link
Contributor Author

Also, should all of those projects be updated in one pull request, or do I need to open a half-dozen more?

@mstorsjo
Copy link
Member

mstorsjo commented May 7, 2025

I agree that the second change, that is using LLVMConfig.cmake's LLVM_DEFINITIONS, should be applied to other projects that support standalone builds like LLD.

Is there a list somewhere or do I just rely on my grep-foo?

  • bolt
  • clang
  • flang
  • lld
  • lldb
  • mlir
  • polly

grep also turned up these, but they don't seem like things that link to LLVM, so probably shouldn't have these defines.

  • compiler-rt
  • libc
  • libclc
  • llvm-libgcc

Yes, this seems correct.

Also, should all of those projects be updated in one pull request, or do I need to open a half-dozen more?

I think it's fine to do that kind of project-wide consistency fixes all in one pull request, especially if the change in each of the projects isn't very large and involved.

@mgorny
Copy link
Member

mgorny commented May 7, 2025

Yeah, I think it's a good idea. Thanks for doing that!

@jeremyd2019 jeremyd2019 force-pushed the clang-respect-llvm-definitons branch from a61d253 to b145528 Compare May 7, 2025 17:48
@jeremyd2019 jeremyd2019 changed the title [Clang][CMake] respect LLVMConfig.cmake's LLVM_DEFINITIONS [CMake] respect LLVMConfig.cmake's LLVM_DEFINITIONS May 7, 2025
@jeremyd2019
Copy link
Contributor Author

I rebased this on top of #138783 and adjusted the title and description. Now it should be in a good state to push cmake changes for other projects.

@llvmbot llvmbot added lld lldb mlir BOLT flang Flang issues not falling into any other category labels May 7, 2025
@jeremyd2019 jeremyd2019 changed the title [CMake] respect LLVMConfig.cmake's LLVM_DEFINITIONS [CMake] respect LLVMConfig.cmake's LLVM_DEFINITIONS in standalone builds May 7, 2025
@mstorsjo
Copy link
Member

I rebased this on top of #138783 and adjusted the title and description. Now it should be in a good state to push cmake changes for other projects.

The changes look good, but it looks like the changes from #138783 still show up when viewing the changes; can you check that you've rebased past the merged #138783?

(Also, I take it that no other subprojects than clang need the cmake_push_check_state change?)

@jeremyd2019 jeremyd2019 force-pushed the clang-respect-llvm-definitons branch from 3078853 to be29dee Compare May 15, 2025 16:57
@jeremyd2019
Copy link
Contributor Author

I rebased this on top of #138783 and adjusted the title and description. Now it should be in a good state to push cmake changes for other projects.

The changes look good, but it looks like the changes from #138783 still show up when viewing the changes; can you check that you've rebased past the merged #138783?

I had not - I have now though.

(Also, I take it that no other subprojects than clang need the cmake_push_check_state change?)

No, the other projects were not messing with _GNU_SOURCE like clang was.

Copy link
Member

@mstorsjo mstorsjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

IIRC @petrhosek had commented on this before, and was generally in favour of it, but I'd still leave it open for a couple days if he wants to comment further on it.

@mstorsjo mstorsjo merged commit ba4bd3f into llvm:main May 22, 2025
11 checks passed
@jeremyd2019 jeremyd2019 deleted the clang-respect-llvm-definitons branch May 22, 2025 05:16
@llvm-ci
Copy link
Collaborator

llvm-ci commented May 22, 2025

LLVM Buildbot has detected a new failure on builder llvm-clang-x86_64-gcc-ubuntu-no-asserts running on doug-worker-6 while building bolt,clang,flang,lld,lldb,mlir,polly at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/202/builds/1378

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
...
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-rewritten-binop.cpp (22841 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-multidimensional.cpp (22842 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert.c (22843 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-structured-binding.cpp (22844 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-negative.cpp (22845 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/macro-to-enum.c (22846 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-uppercase.cpp (22847 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-basic.cpp (22848 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-unique-cxx11.cpp (22849 of 88321)
TIMEOUT: AddressSanitizer-x86_64-linux-dynamic :: TestCases/asan_lsan_deadlock.cpp (22850 of 88321)
******************** TEST 'AddressSanitizer-x86_64-linux-dynamic :: TestCases/asan_lsan_deadlock.cpp' FAILED ********************
Exit Code: -9
Timeout: Reached timeout of 900 seconds

Command Output (stderr):
--
/home/buildbot/buildbot-root/gcc-no-asserts/build/./bin/clang  --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only  -m64  -shared-libasan -O0 /home/buildbot/buildbot-root/gcc-no-asserts/llvm-project/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp -o /home/buildbot/buildbot-root/gcc-no-asserts/build/runtimes/runtimes-bins/compiler-rt/test/asan/X86_64LinuxDynamicConfig/TestCases/Output/asan_lsan_deadlock.cpp.tmp # RUN: at line 4
+ /home/buildbot/buildbot-root/gcc-no-asserts/build/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -m64 -shared-libasan -O0 /home/buildbot/buildbot-root/gcc-no-asserts/llvm-project/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp -o /home/buildbot/buildbot-root/gcc-no-asserts/build/runtimes/runtimes-bins/compiler-rt/test/asan/X86_64LinuxDynamicConfig/TestCases/Output/asan_lsan_deadlock.cpp.tmp
env ASAN_OPTIONS=detect_leaks=1 not  /home/buildbot/buildbot-root/gcc-no-asserts/build/runtimes/runtimes-bins/compiler-rt/test/asan/X86_64LinuxDynamicConfig/TestCases/Output/asan_lsan_deadlock.cpp.tmp 2>&1 | FileCheck /home/buildbot/buildbot-root/gcc-no-asserts/llvm-project/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp # RUN: at line 5
+ env ASAN_OPTIONS=detect_leaks=1 not /home/buildbot/buildbot-root/gcc-no-asserts/build/runtimes/runtimes-bins/compiler-rt/test/asan/X86_64LinuxDynamicConfig/TestCases/Output/asan_lsan_deadlock.cpp.tmp
+ FileCheck /home/buildbot/buildbot-root/gcc-no-asserts/llvm-project/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp

--

********************
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-shared-header.cpp (22851 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-extra.cpp (22852 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/macro-to-enum.cpp (22853 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-unique-default-init.cpp (22854 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-unique-inaccessible-ctors.cpp (22855 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-unique-macros.cpp (22856 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-unique-header.cpp (22857 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/pass-by-value-header.cpp (22858 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/pass-by-value-multi-fixes.cpp (22859 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-shared.cpp (22860 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/loop-convert-reverse.cpp (22861 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/pass-by-value-macro-header.cpp (22862 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/raw-string-literal-delimiter.cpp (22863 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/redundant-void-arg.c (22864 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/raw-string-literal-replace-shorter.cpp (22865 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/pass-by-value.cpp (22866 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/make-unique.cpp (22867 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/min-max-use-initializer-list.cpp (22868 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/raw-string-literal.cpp (22869 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/redundant-void-arg-delayed.cpp (22870 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/replace-random-shuffle.cpp (22871 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/unary-static-assert.cpp (22872 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/return-braced-init-list.cpp (22873 of 88321)
PASS: Clang Tools :: clang-tidy/checkers/modernize/replace-auto-ptr.cpp (22874 of 88321)

@llvm-ci
Copy link
Collaborator

llvm-ci commented May 22, 2025

LLVM Buildbot has detected a new failure on builder clang-ppc64-aix running on aix-ppc64 while building bolt,clang,flang,lld,lldb,mlir,polly at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/64/builds/3735

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'lit :: timeout-hang.py' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 13
not env -u FILECHECK_OPTS "/home/llvm/llvm-external-buildbots/workers/env/bin/python3.11" /home/llvm/llvm-external-buildbots/workers/aix-ppc64/clang-ppc64-aix/llvm-project/llvm/utils/lit/lit.py -j1 --order=lexical Inputs/timeout-hang/run-nonexistent.txt  --timeout=1 --param external=0 | "/home/llvm/llvm-external-buildbots/workers/env/bin/python3.11" /home/llvm/llvm-external-buildbots/workers/aix-ppc64/clang-ppc64-aix/build/utils/lit/tests/timeout-hang.py 1
# executed command: not env -u FILECHECK_OPTS /home/llvm/llvm-external-buildbots/workers/env/bin/python3.11 /home/llvm/llvm-external-buildbots/workers/aix-ppc64/clang-ppc64-aix/llvm-project/llvm/utils/lit/lit.py -j1 --order=lexical Inputs/timeout-hang/run-nonexistent.txt --timeout=1 --param external=0
# .---command stderr------------
# | lit.py: /home/llvm/llvm-external-buildbots/workers/aix-ppc64/clang-ppc64-aix/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 1 seconds was requested on the command line. Forcing timeout to be 1 seconds.
# `-----------------------------
# executed command: /home/llvm/llvm-external-buildbots/workers/env/bin/python3.11 /home/llvm/llvm-external-buildbots/workers/aix-ppc64/clang-ppc64-aix/build/utils/lit/tests/timeout-hang.py 1
# .---command stdout------------
# | Testing took as long or longer than timeout
# `-----------------------------
# error: command failed with exit status: 1

--

********************


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BOLT clang Clang issues not falling into any other category flang Flang issues not falling into any other category lld lldb mlir

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants