Skip to content

[libc] Reland #149423 "wchar string conversion functions mb to wc" #150667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
Jul 28, 2025
Merged

[libc] Reland #149423 "wchar string conversion functions mb to wc" #150667

merged 20 commits into from
Jul 28, 2025

Conversation

@sribee8
Copy link
Contributor

@sribee8 sribee8 commented Jul 25, 2025

Added missing includes in the test files for null check

Sriya Pratipati and others added 18 commits July 15, 2025 22:08
started mbstowcs implementation
5757dce
finished mbstowcs implementation
c9d06fc
implemented mbsrtowcs
26a6741
added tests for mbstowcs
0694e91
added final test
1921356
added internal mbsnrtowcs
6bc7497
Merge branch 'main' into mbstowcs-implementation
fbd6453
implemented mbsnrtowcs and tests
40aa6ee
fixed formatting
b3b7b2a
added crash on nullptr for src and moved internal implementation to h…
d8f2d6f 
…eader file
fixed formatting
18e2f26
header file changes
84ad6a9
fixed spacing in cmake file
6595702
fixed typo in cmake
6a55826
alphabetized yaml
eacce44
@sribee8
Merge branch 'main' into mbstowcs-implementation
e116c79
@sribee8
Merge branch 'main' into mbstowcs-implementation
6bc272a
Reland "[libc] wchar string conversion functions mb to wc"
2b155f9 
Added missing includes in the test files for null check
uzairnawaz
uzairnawaz approved these changes Jul 25, 2025
View reviewed changes
Copy link
Contributor

@uzairnawaz uzairnawaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

michaelrj-google
michaelrj-google reviewed Jul 25, 2025
View reviewed changes
Copy link
Contributor

@michaelrj-google michaelrj-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need the LIBC_HAS_SANITIZER check, for the reasons described in this PR: #150341

@sribee8 sribee8 merged commit 741df45 into llvm:main Jul 28, 2025
14 of 19 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Jul 28, 2025

LLVM Buildbot has detected a new failure on builder libc-x86_64-debian-fullbuild-dbg-asan running on libc-x86_64-debian-fullbuild while building libc at step 4 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/171/builds/27172

Here is the relevant piece of the build log for the reference 
Step 4 (annotate) failure: 'python ../llvm-zorg/zorg/buildbot/builders/annotated/libc-linux.py ...' (failure)
...
[ RUN      ] LlvmLibcMBSRToWCSTest.InvalidMiddleByte
[       OK ] LlvmLibcMBSRToWCSTest.InvalidMiddleByte (3 us)
[ RUN      ] LlvmLibcMBSRToWCSTest.NullDestination
[       OK ] LlvmLibcMBSRToWCSTest.NullDestination (5 us)
[ RUN      ] LlvmLibcMBSRToWCSTest.ErrnoChecks
[       OK ] LlvmLibcMBSRToWCSTest.ErrnoChecks (6 us)
[ RUN      ] LlvmLibcMBSRToWCSTest.NullptrCrash
[       OK ] LlvmLibcMBSRToWCSTest.NullptrCrash (653 us)
Ran 10 tests.  PASS: 10  FAIL: 0
[1217/1221] Running unit test libc.test.src.wchar.mbstowcs_test.__unit__
FAILED: libc/test/src/wchar/CMakeFiles/libc.test.src.wchar.mbstowcs_test.__unit__ /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/build/libc/test/src/wchar/CMakeFiles/libc.test.src.wchar.mbstowcs_test.__unit__ 
cd /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/build/libc/test/src/wchar && /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/build/libc/test/src/wchar/libc.test.src.wchar.mbstowcs_test.__unit__.__build__
[==========] Running 10 tests from 1 test suite.
[ RUN      ] LlvmLibcMBSToWCSTest.OneByteOneChar
[       OK ] LlvmLibcMBSToWCSTest.OneByteOneChar (24 us)
[ RUN      ] LlvmLibcMBSToWCSTest.FourByteOneChar
[       OK ] LlvmLibcMBSToWCSTest.FourByteOneChar (6 us)
[ RUN      ] LlvmLibcMBSToWCSTest.MultiByteTwoCharacters
[       OK ] LlvmLibcMBSToWCSTest.MultiByteTwoCharacters (4 us)
[ RUN      ] LlvmLibcMBSToWCSTest.MixedNumberOfBytes
[       OK ] LlvmLibcMBSToWCSTest.MixedNumberOfBytes (6 us)
[ RUN      ] LlvmLibcMBSToWCSTest.ReadLessThanStringLength
[       OK ] LlvmLibcMBSToWCSTest.ReadLessThanStringLength (5 us)
[ RUN      ] LlvmLibcMBSToWCSTest.InvalidFirstByte
[       OK ] LlvmLibcMBSToWCSTest.InvalidFirstByte (2 us)
[ RUN      ] LlvmLibcMBSToWCSTest.InvalidMiddleByte
[       OK ] LlvmLibcMBSToWCSTest.InvalidMiddleByte (3 us)
[ RUN      ] LlvmLibcMBSToWCSTest.NullDestination
[       OK ] LlvmLibcMBSToWCSTest.NullDestination (4 us)
[ RUN      ] LlvmLibcMBSToWCSTest.ErrnoChecks
[       OK ] LlvmLibcMBSToWCSTest.ErrnoChecks (6 us)
[ RUN      ] LlvmLibcMBSToWCSTest.NullptrCrash
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3847002==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x5567c04fa666 bp 0x7ffd1432d330 sp 0x7ffd1432d1a0 T0)
==3847002==The signal is caused by a READ memory access.
==3847002==Hint: address points to the zero page.
    #0 0x5567c04fa666 in __llvm_libc_20_0_0_git::internal::StringConverter<unsigned char>::pushFullCharacter() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/__support/wchar/string_converter.h:37:20
    #1 0x5567c04f9c0c in __llvm_libc_20_0_0_git::internal::StringConverter<unsigned char>::popUTF32() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/__support/wchar/string_converter.h:63:32
    #2 0x5567c04f9000 in __llvm_libc_20_0_0_git::internal::mbsnrtowcs(wchar_t*, char const**, unsigned long, unsigned long, __llvm_libc_20_0_0_git::internal::mbstate*) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/__support/wchar/mbsnrtowcs.h:39:42
    #3 0x5567c04f89a9 in __llvm_libc_20_0_0_git::mbstowcs(wchar_t*, char const*, unsigned long) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/wchar/mbstowcs.cpp:28:14
    #4 0x5567c04f4d2c in LlvmLibcMBSToWCSTest_NullptrCrash::Run()::$_0::operator()() const /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/src/wchar/mbstowcs_test.cpp:168:3
    #5 0x5567c04f4d08 in __llvm_libc_20_0_0_git::testutils::FunctionCaller* __llvm_libc_20_0_0_git::testing::Test::createCallable<LlvmLibcMBSToWCSTest_NullptrCrash::Run()::$_0>(LlvmLibcMBSToWCSTest_NullptrCrash::Run()::$_0)::Callable::operator()() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcTest.h:218:36
    #6 0x5567c0501c3d in __llvm_libc_20_0_0_git::testutils::invoke_in_subprocess(__llvm_libc_20_0_0_git::testutils::FunctionCaller*, int) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/ExecuteFunctionUnix.cpp:54:5
    #7 0x5567c0500351 in __llvm_libc_20_0_0_git::testing::Test::testProcessKilled(__llvm_libc_20_0_0_git::testutils::FunctionCaller*, int, char const*, char const*, __llvm_libc_20_0_0_git::testing::internal::Location) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcDeathTestExecutors.cpp:28:7
    #8 0x5567c04f4b74 in LlvmLibcMBSToWCSTest_NullptrCrash::Run() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/src/wchar/mbstowcs_test.cpp:168:3
    #9 0x5567c0503887 in __llvm_libc_20_0_0_git::testing::Test::runTests(__llvm_libc_20_0_0_git::testing::TestOptions const&) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcTest.cpp:165:8
    #10 0x5567c0541ef8 in main /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcTestMain.cpp:59:10
    #11 0x7f541fa46249 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
Step 8 (libc-unit-tests) failure: libc-unit-tests (failure)
...
[ RUN      ] LlvmLibcMBSRToWCSTest.InvalidMiddleByte
[       OK ] LlvmLibcMBSRToWCSTest.InvalidMiddleByte (3 us)
[ RUN      ] LlvmLibcMBSRToWCSTest.NullDestination
[       OK ] LlvmLibcMBSRToWCSTest.NullDestination (5 us)
[ RUN      ] LlvmLibcMBSRToWCSTest.ErrnoChecks
[       OK ] LlvmLibcMBSRToWCSTest.ErrnoChecks (6 us)
[ RUN      ] LlvmLibcMBSRToWCSTest.NullptrCrash
[       OK ] LlvmLibcMBSRToWCSTest.NullptrCrash (653 us)
Ran 10 tests.  PASS: 10  FAIL: 0
[1217/1221] Running unit test libc.test.src.wchar.mbstowcs_test.__unit__
FAILED: libc/test/src/wchar/CMakeFiles/libc.test.src.wchar.mbstowcs_test.__unit__ /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/build/libc/test/src/wchar/CMakeFiles/libc.test.src.wchar.mbstowcs_test.__unit__ 
cd /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/build/libc/test/src/wchar && /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/build/libc/test/src/wchar/libc.test.src.wchar.mbstowcs_test.__unit__.__build__
[==========] Running 10 tests from 1 test suite.
[ RUN      ] LlvmLibcMBSToWCSTest.OneByteOneChar
[       OK ] LlvmLibcMBSToWCSTest.OneByteOneChar (24 us)
[ RUN      ] LlvmLibcMBSToWCSTest.FourByteOneChar
[       OK ] LlvmLibcMBSToWCSTest.FourByteOneChar (6 us)
[ RUN      ] LlvmLibcMBSToWCSTest.MultiByteTwoCharacters
[       OK ] LlvmLibcMBSToWCSTest.MultiByteTwoCharacters (4 us)
[ RUN      ] LlvmLibcMBSToWCSTest.MixedNumberOfBytes
[       OK ] LlvmLibcMBSToWCSTest.MixedNumberOfBytes (6 us)
[ RUN      ] LlvmLibcMBSToWCSTest.ReadLessThanStringLength
[       OK ] LlvmLibcMBSToWCSTest.ReadLessThanStringLength (5 us)
[ RUN      ] LlvmLibcMBSToWCSTest.InvalidFirstByte
[       OK ] LlvmLibcMBSToWCSTest.InvalidFirstByte (2 us)
[ RUN      ] LlvmLibcMBSToWCSTest.InvalidMiddleByte
[       OK ] LlvmLibcMBSToWCSTest.InvalidMiddleByte (3 us)
[ RUN      ] LlvmLibcMBSToWCSTest.NullDestination
[       OK ] LlvmLibcMBSToWCSTest.NullDestination (4 us)
[ RUN      ] LlvmLibcMBSToWCSTest.ErrnoChecks
[       OK ] LlvmLibcMBSToWCSTest.ErrnoChecks (6 us)
[ RUN      ] LlvmLibcMBSToWCSTest.NullptrCrash
AddressSanitizer:DEADLYSIGNAL
=================================================================
==3847002==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x5567c04fa666 bp 0x7ffd1432d330 sp 0x7ffd1432d1a0 T0)
==3847002==The signal is caused by a READ memory access.
==3847002==Hint: address points to the zero page.
    #0 0x5567c04fa666 in __llvm_libc_20_0_0_git::internal::StringConverter<unsigned char>::pushFullCharacter() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/__support/wchar/string_converter.h:37:20
    #1 0x5567c04f9c0c in __llvm_libc_20_0_0_git::internal::StringConverter<unsigned char>::popUTF32() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/__support/wchar/string_converter.h:63:32
    #2 0x5567c04f9000 in __llvm_libc_20_0_0_git::internal::mbsnrtowcs(wchar_t*, char const**, unsigned long, unsigned long, __llvm_libc_20_0_0_git::internal::mbstate*) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/__support/wchar/mbsnrtowcs.h:39:42
    #3 0x5567c04f89a9 in __llvm_libc_20_0_0_git::mbstowcs(wchar_t*, char const*, unsigned long) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/src/wchar/mbstowcs.cpp:28:14
    #4 0x5567c04f4d2c in LlvmLibcMBSToWCSTest_NullptrCrash::Run()::$_0::operator()() const /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/src/wchar/mbstowcs_test.cpp:168:3
    #5 0x5567c04f4d08 in __llvm_libc_20_0_0_git::testutils::FunctionCaller* __llvm_libc_20_0_0_git::testing::Test::createCallable<LlvmLibcMBSToWCSTest_NullptrCrash::Run()::$_0>(LlvmLibcMBSToWCSTest_NullptrCrash::Run()::$_0)::Callable::operator()() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcTest.h:218:36
    #6 0x5567c0501c3d in __llvm_libc_20_0_0_git::testutils::invoke_in_subprocess(__llvm_libc_20_0_0_git::testutils::FunctionCaller*, int) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/ExecuteFunctionUnix.cpp:54:5
    #7 0x5567c0500351 in __llvm_libc_20_0_0_git::testing::Test::testProcessKilled(__llvm_libc_20_0_0_git::testutils::FunctionCaller*, int, char const*, char const*, __llvm_libc_20_0_0_git::testing::internal::Location) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcDeathTestExecutors.cpp:28:7
    #8 0x5567c04f4b74 in LlvmLibcMBSToWCSTest_NullptrCrash::Run() /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/src/wchar/mbstowcs_test.cpp:168:3
    #9 0x5567c0503887 in __llvm_libc_20_0_0_git::testing::Test::runTests(__llvm_libc_20_0_0_git::testing::TestOptions const&) /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcTest.cpp:165:8
    #10 0x5567c0541ef8 in main /home/llvm-libc-buildbot/buildbot-worker/libc-x86_64-debian-fullbuild/libc-x86_64-debian-fullbuild-dbg-asan/llvm-project/libc/test/UnitTest/LibcTestMain.cpp:59:10
    #11 0x7f541fa46249 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

@sribee8 sribee8 mentioned this pull request Jul 28, 2025
Merged
sribee8 added a commit that referenced this pull request Jul 28, 2025
@sribee8
Revert "[libc] Reland #149423 "wchar string conversion functions mb t…
1381ad4 
…o wc"" (#151016)

Reverts #150667
llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Jul 28, 2025
@sribee8 @github-actions
Automerge: Revert "[libc] Reland #149423 "wchar string conversion fun…
e7a03f0 
…ctions mb to wc"" (#151016)

Reverts llvm/llvm-project#150667
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@uzairnawaz uzairnawaz uzairnawaz approved these changes

@michaelrj-google michaelrj-google michaelrj-google approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sribee8 @llvm-ci @uzairnawaz @michaelrj-google