Skip to content

Conversation

@janvorli
Copy link
Member

@janvorli janvorli commented Dec 10, 2021

The issue seems to be caused by a bug in the test. The problematic call
to TestCreateMutex is passed nullptr as the name parameter. It
then calls convert helper on it to convert it to wide char. However,
the convert helper doesn't check whether it is nullptr or not and
ends up returning a pointer to a memory with possibly random data,
that is returned by malloc(0). The returned pointer is then passed to
the CreateMutex PAL api that probably ends up attempting to get the
length of the name or something. And depending on the random data, it
sometimes fails.

The fix is to change the convert function to handle nullptr so that
it returns nullptr too.

Close #62336

@janvorli janvorli added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Dec 10, 2021
@janvorli janvorli self-assigned this Dec 10, 2021
@ghost ghost added the area-PAL-coreclr label Dec 10, 2021
@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@janvorli
Copy link
Member Author

/azp run runtime-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

The issue seems to be caused by a bug in the test. The problematic call
to `TestCreateMutex` is passed `nullptr` as the `name` parameter. It
then calls `convert` helper on it to convert it to wide char. However,
the `convert` helper doesn't check whether it is `nullptr` or not and
ends up returning a pointer to a memory with possibly random data,
that is returned by `malloc(0)`. The returned pointer is then passed to
the CreateMutex PAL api that probably ends up attempting to get the
length of the name or something. And depending on the random data, it
sometimes fails.

The fix is to change the `convert` function to handle `nullptr` so that
it returns `nullptr` too.
@janvorli janvorli force-pushed the test-namedmutex-failure branch from 9b8433b to 68c008c Compare December 14, 2021 21:29
@janvorli janvorli changed the title Attempt to repro named mutex pal test issue Fix named mutex PAL test issue Dec 14, 2021
@janvorli janvorli requested a review from trylek December 14, 2021 21:31
@janvorli janvorli removed the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Dec 14, 2021
@janvorli janvorli added this to the 7.0.0 milestone Dec 14, 2021
Copy link
Member

@trylek trylek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, fixing tests is often equally or more challenging than fixing the actual runtime behavior, thanks Jan for figuring this out!

@janvorli janvorli merged commit d0be66c into dotnet:main Dec 14, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Jan 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

threading/NamedMutex/test1/paltest_namedmutex_test1 failed

2 participants