-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Allow to specify file allocation size #51111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
fe24ee2
add the new ctor to the ref assembly
adamsitnik c127580
add tests
adamsitnik f3af047
add new ctor
adamsitnik 5d004fd
initial Windows implementation
adamsitnik bbac6d7
fix the build
adamsitnik 5451439
remove duplicated Interop code
adamsitnik 29b1223
Merge remote-tracking branch 'upstream/main' into fileAllocationSize
adamsitnik 07cfc75
fix the full framework build
adamsitnik 945c917
initial Unix implementation
adamsitnik 0b566aa
Merge remote-tracking branch 'upstream/main' into fileAllocationSize
adamsitnik 3bf1f95
some polishing
adamsitnik ed1f18d
remove unused using and extra empty lines from existing tests
adamsitnik f645e67
the active issue link should point to a valid issue
adamsitnik 909c86e
the tests should actually verify whether FileStream is async or not
adamsitnik 6789382
remove duplicated consts
adamsitnik dbe44d4
proper tests for Windows
adamsitnik 3571a23
implement Unix tests and make them pass by moving fallocate call afte…
adamsitnik cbe2277
fix the Windows build...
adamsitnik c490439
align macOS implementation with Linux
adamsitnik 7842eba
polishing
adamsitnik b2cc9f3
handle files that are too large for the current file system
adamsitnik ed5808f
add a test for "file was too large" exception
adamsitnik e250d96
and handling for EFBIG
adamsitnik df155e2
fix the Unix build
adamsitnik b6929e3
fix a wrong assert
adamsitnik 6875a9f
try to enable the new tests for Browser build
adamsitnik 0a0edad
Apply suggestions from code review
adamsitnik c46ac66
apply other suggestions for the code review, get rid of the managed a…
adamsitnik 142fbb1
Merge remote-tracking branch 'upstream/main' into fileAllocationSize
adamsitnik 83f119d
disable it for WASM
adamsitnik 37b9245
address code review feedback: remove unused path parameter, use stack…
adamsitnik c2f995e
address code review feedback: if NtCreateFile fails, we should immedi…
adamsitnik 2b00680
Apply suggestions from code review
adamsitnik 60102b0
Merge remote-tracking branch 'upstream/main' into fileAllocationSize
adamsitnik 4065f43
introduce FileStreamOptions
adamsitnik b0c8243
address code review feedback: throw ArgumentOutOfRangeException when …
adamsitnik 2145dc1
address code review feedback: move error codes
adamsitnik 5eed718
SECURITY_ANONYMOUS
adamsitnik 4993dbb
Merge remote-tracking branch 'upstream/main' into fileAllocationSize
adamsitnik b9987a8
address code review feedback
adamsitnik 9728078
address code review feedback and clarify which values are supported a…
adamsitnik 8648f65
fix XML docs
adamsitnik File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
17 changes: 17 additions & 0 deletions
17
src/libraries/Common/src/Interop/Unix/System.Native/Interop.PosixFAllocate.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using System.Runtime.InteropServices; | ||
| using Microsoft.Win32.SafeHandles; | ||
|
|
||
| internal static partial class Interop | ||
| { | ||
| internal static partial class Sys | ||
| { | ||
| /// <summary> | ||
| /// Returns -1 on ENOSPC, -2 on EFBIG. On success or ignorable error, 0 is returned. | ||
| /// </summary> | ||
| [DllImport(Libraries.SystemNative, EntryPoint = "SystemNative_PosixFAllocate", SetLastError = false)] | ||
| internal static extern int PosixFAllocate(SafeFileHandle fd, long offset, long length); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
75 changes: 75 additions & 0 deletions
75
src/libraries/Common/src/Interop/Windows/Interop.SECURITY_QUALITY_OF_SERVICE.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| internal static partial class Interop | ||
| { | ||
| /// <summary> | ||
| /// <a href="https://docs.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-security_quality_of_service">SECURITY_QUALITY_OF_SERVICE</a> structure. | ||
| /// Used to support client impersonation. Client specifies this to a server to allow | ||
| /// it to impersonate the client. | ||
| /// </summary> | ||
| internal unsafe struct SECURITY_QUALITY_OF_SERVICE | ||
| { | ||
| public uint Length; | ||
| public ImpersonationLevel ImpersonationLevel; | ||
| public ContextTrackingMode ContextTrackingMode; | ||
| public BOOLEAN EffectiveOnly; | ||
|
|
||
| public unsafe SECURITY_QUALITY_OF_SERVICE(ImpersonationLevel impersonationLevel, ContextTrackingMode contextTrackingMode, bool effectiveOnly) | ||
| { | ||
| Length = (uint)sizeof(SECURITY_QUALITY_OF_SERVICE); | ||
| ImpersonationLevel = impersonationLevel; | ||
| ContextTrackingMode = contextTrackingMode; | ||
| EffectiveOnly = effectiveOnly ? BOOLEAN.TRUE : BOOLEAN.FALSE; | ||
| } | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/aa379572.aspx">SECURITY_IMPERSONATION_LEVEL</a> enumeration values. | ||
| /// [SECURITY_IMPERSONATION_LEVEL] | ||
| /// </summary> | ||
| public enum ImpersonationLevel : uint | ||
| { | ||
| /// <summary> | ||
| /// The server process cannot obtain identification information about the client and cannot impersonate the client. | ||
| /// [SecurityAnonymous] | ||
| /// </summary> | ||
| Anonymous, | ||
|
|
||
| /// <summary> | ||
| /// The server process can obtain identification information about the client, but cannot impersonate the client. | ||
| /// [SecurityIdentification] | ||
| /// </summary> | ||
| Identification, | ||
|
|
||
| /// <summary> | ||
| /// The server process can impersonate the client's security context on it's local system. | ||
| /// [SecurityImpersonation] | ||
| /// </summary> | ||
| Impersonation, | ||
|
|
||
| /// <summary> | ||
| /// The server process can impersonate the client's security context on remote systems. | ||
| /// [SecurityDelegation] | ||
| /// </summary> | ||
| Delegation | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// <a href="https://msdn.microsoft.com/en-us/library/cc234317.aspx">SECURITY_CONTEXT_TRACKING_MODE</a> | ||
| /// </summary> | ||
| public enum ContextTrackingMode : byte | ||
| { | ||
| /// <summary> | ||
| /// The server is given a snapshot of the client's security context. | ||
| /// [SECURITY_STATIC_TRACKING] | ||
| /// </summary> | ||
| Static = 0x00, | ||
|
|
||
| /// <summary> | ||
| /// The server is continually updated with changes. | ||
| /// [SECURITY_DYNAMIC_TRACKING] | ||
| /// </summary> | ||
| Dynamic = 0x01 | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -2,39 +2,47 @@ | |||||||||||
| // The .NET Foundation licenses this file to you under the MIT license. | ||||||||||||
|
|
||||||||||||
| using System; | ||||||||||||
| using System.Diagnostics; | ||||||||||||
| using System.IO; | ||||||||||||
| using System.Runtime.InteropServices; | ||||||||||||
|
|
||||||||||||
| internal static partial class Interop | ||||||||||||
| { | ||||||||||||
| internal static partial class NtDll | ||||||||||||
| { | ||||||||||||
| internal const uint NT_ERROR_STATUS_DISK_FULL = 0xC000007F; | ||||||||||||
| internal const uint NT_ERROR_STATUS_FILE_TOO_LARGE = 0xC0000904; | ||||||||||||
| internal const uint NT_STATUS_INVALID_PARAMETER = 0xC000000D; | ||||||||||||
|
|
||||||||||||
| // https://msdn.microsoft.com/en-us/library/bb432380.aspx | ||||||||||||
| // https://msdn.microsoft.com/en-us/library/windows/hardware/ff566424.aspx | ||||||||||||
| [DllImport(Libraries.NtDll, CharSet = CharSet.Unicode, ExactSpelling = true)] | ||||||||||||
| private static extern unsafe int NtCreateFile( | ||||||||||||
| private static extern unsafe uint NtCreateFile( | ||||||||||||
| out IntPtr FileHandle, | ||||||||||||
| DesiredAccess DesiredAccess, | ||||||||||||
| ref OBJECT_ATTRIBUTES ObjectAttributes, | ||||||||||||
| out IO_STATUS_BLOCK IoStatusBlock, | ||||||||||||
| long* AllocationSize, | ||||||||||||
| System.IO.FileAttributes FileAttributes, | ||||||||||||
| System.IO.FileShare ShareAccess, | ||||||||||||
| FileAttributes FileAttributes, | ||||||||||||
| FileShare ShareAccess, | ||||||||||||
| CreateDisposition CreateDisposition, | ||||||||||||
| CreateOptions CreateOptions, | ||||||||||||
| void* EaBuffer, | ||||||||||||
| uint EaLength); | ||||||||||||
|
|
||||||||||||
| internal static unsafe (int status, IntPtr handle) CreateFile( | ||||||||||||
| internal static unsafe (uint status, IntPtr handle) CreateFile( | ||||||||||||
| ReadOnlySpan<char> path, | ||||||||||||
| IntPtr rootDirectory, | ||||||||||||
| CreateDisposition createDisposition, | ||||||||||||
| DesiredAccess desiredAccess = DesiredAccess.FILE_GENERIC_READ | DesiredAccess.SYNCHRONIZE, | ||||||||||||
| System.IO.FileShare shareAccess = System.IO.FileShare.ReadWrite | System.IO.FileShare.Delete, | ||||||||||||
| System.IO.FileAttributes fileAttributes = 0, | ||||||||||||
| FileShare shareAccess = FileShare.ReadWrite | FileShare.Delete, | ||||||||||||
| FileAttributes fileAttributes = 0, | ||||||||||||
| CreateOptions createOptions = CreateOptions.FILE_SYNCHRONOUS_IO_NONALERT, | ||||||||||||
| ObjectAttributes objectAttributes = ObjectAttributes.OBJ_CASE_INSENSITIVE, | ||||||||||||
| void* eaBuffer = null, | ||||||||||||
| uint eaLength = 0) | ||||||||||||
| uint eaLength = 0, | ||||||||||||
| long* preallocationSize = null, | ||||||||||||
| SECURITY_QUALITY_OF_SERVICE* securityQualityOfService = null) | ||||||||||||
| { | ||||||||||||
| fixed (char* c = &MemoryMarshal.GetReference(path)) | ||||||||||||
| { | ||||||||||||
|
|
@@ -48,14 +56,15 @@ internal static unsafe (int status, IntPtr handle) CreateFile( | |||||||||||
| OBJECT_ATTRIBUTES attributes = new OBJECT_ATTRIBUTES( | ||||||||||||
| &name, | ||||||||||||
| objectAttributes, | ||||||||||||
| rootDirectory); | ||||||||||||
| rootDirectory, | ||||||||||||
| securityQualityOfService); | ||||||||||||
|
|
||||||||||||
| int status = NtCreateFile( | ||||||||||||
| uint status = NtCreateFile( | ||||||||||||
| out IntPtr handle, | ||||||||||||
| desiredAccess, | ||||||||||||
| ref attributes, | ||||||||||||
| out IO_STATUS_BLOCK statusBlock, | ||||||||||||
| AllocationSize: null, | ||||||||||||
| AllocationSize: preallocationSize, | ||||||||||||
| fileAttributes, | ||||||||||||
| shareAccess, | ||||||||||||
| createDisposition, | ||||||||||||
|
|
@@ -67,6 +76,122 @@ internal static unsafe (int status, IntPtr handle) CreateFile( | |||||||||||
| } | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| internal static unsafe (uint status, IntPtr handle) CreateFile(ReadOnlySpan<char> path, FileMode mode, FileAccess access, FileShare share, FileOptions options, long preallocationSize) | ||||||||||||
| { | ||||||||||||
| // For mitigating local elevation of privilege attack through named pipes | ||||||||||||
| // make sure we always call NtCreateFile with SECURITY_ANONYMOUS so that the | ||||||||||||
| // named pipe server can't impersonate a high privileged client security context | ||||||||||||
| SECURITY_QUALITY_OF_SERVICE securityQualityOfService = new SECURITY_QUALITY_OF_SERVICE( | ||||||||||||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @carlossanlop @jozkee this is to mimic the following logic: runtime/src/libraries/System.Private.CoreLib/src/System/IO/Strategies/FileStreamHelpers.Windows.cs Lines 83 to 87 in 88211b9
|
||||||||||||
| ImpersonationLevel.Anonymous, // SECURITY_ANONYMOUS | ||||||||||||
| ContextTrackingMode.Static, | ||||||||||||
| effectiveOnly: false); | ||||||||||||
|
|
||||||||||||
| return CreateFile( | ||||||||||||
| path: path, | ||||||||||||
| rootDirectory: IntPtr.Zero, | ||||||||||||
| createDisposition: GetCreateDisposition(mode), | ||||||||||||
| desiredAccess: GetDesiredAccess(access, mode, options), | ||||||||||||
| shareAccess: GetShareAccess(share), | ||||||||||||
| fileAttributes: GetFileAttributes(options), | ||||||||||||
| createOptions: GetCreateOptions(options), | ||||||||||||
| objectAttributes: GetObjectAttributes(share), | ||||||||||||
| preallocationSize: &preallocationSize, | ||||||||||||
| securityQualityOfService: &securityQualityOfService); | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| private static CreateDisposition GetCreateDisposition(FileMode mode) | ||||||||||||
| { | ||||||||||||
| switch (mode) | ||||||||||||
| { | ||||||||||||
| case FileMode.CreateNew: | ||||||||||||
| return CreateDisposition.FILE_CREATE; | ||||||||||||
| case FileMode.Create: | ||||||||||||
| return CreateDisposition.FILE_SUPERSEDE; | ||||||||||||
| case FileMode.OpenOrCreate: | ||||||||||||
| case FileMode.Append: // has extra handling in GetDesiredAccess | ||||||||||||
| return CreateDisposition.FILE_OPEN_IF; | ||||||||||||
| case FileMode.Truncate: | ||||||||||||
| return CreateDisposition.FILE_OVERWRITE; | ||||||||||||
| default: | ||||||||||||
| Debug.Assert(mode == FileMode.Open); // the enum value is validated in FileStream ctor | ||||||||||||
| return CreateDisposition.FILE_OPEN; | ||||||||||||
| } | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| private static DesiredAccess GetDesiredAccess(FileAccess access, FileMode fileMode, FileOptions options) | ||||||||||||
| { | ||||||||||||
| DesiredAccess result = 0; | ||||||||||||
|
|
||||||||||||
| if ((access & FileAccess.Read) != 0) | ||||||||||||
| { | ||||||||||||
| result |= DesiredAccess.FILE_GENERIC_READ; | ||||||||||||
| } | ||||||||||||
| if ((access & FileAccess.Write) != 0) | ||||||||||||
| { | ||||||||||||
| result |= DesiredAccess.FILE_GENERIC_WRITE; | ||||||||||||
| } | ||||||||||||
| if (fileMode == FileMode.Append) | ||||||||||||
| { | ||||||||||||
| result |= DesiredAccess.FILE_APPEND_DATA; | ||||||||||||
| } | ||||||||||||
| if ((options & FileOptions.Asynchronous) == 0) | ||||||||||||
| { | ||||||||||||
| result |= DesiredAccess.SYNCHRONIZE; // required by FILE_SYNCHRONOUS_IO_NONALERT | ||||||||||||
| } | ||||||||||||
| if ((options & FileOptions.DeleteOnClose) != 0 || fileMode == FileMode.Create) | ||||||||||||
| { | ||||||||||||
| result |= DesiredAccess.DELETE; // required by FILE_DELETE_ON_CLOSE and FILE_SUPERSEDE (which deletes a file if it exists) | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| return result; | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| private static FileShare GetShareAccess(FileShare share) | ||||||||||||
| => share & ~FileShare.Inheritable; // FileShare.Inheritable is handled in GetObjectAttributes | ||||||||||||
|
|
||||||||||||
| private static FileAttributes GetFileAttributes(FileOptions options) | ||||||||||||
| => (options & FileOptions.Encrypted) != 0 ? FileAttributes.Encrypted : 0; | ||||||||||||
|
|
||||||||||||
| // FileOptions.Encrypted is handled in GetFileAttributes | ||||||||||||
| private static CreateOptions GetCreateOptions(FileOptions options) | ||||||||||||
| { | ||||||||||||
| // Every directory is just a directory FILE. | ||||||||||||
| // FileStream does not allow for opening directories on purpose. | ||||||||||||
| // FILE_NON_DIRECTORY_FILE is used to ensure that | ||||||||||||
| CreateOptions result = CreateOptions.FILE_NON_DIRECTORY_FILE; | ||||||||||||
|
|
||||||||||||
| if ((options & FileOptions.WriteThrough) != 0) | ||||||||||||
| { | ||||||||||||
| result |= CreateOptions.FILE_WRITE_THROUGH; | ||||||||||||
| } | ||||||||||||
| if ((options & FileOptions.RandomAccess) != 0) | ||||||||||||
| { | ||||||||||||
| result |= CreateOptions.FILE_RANDOM_ACCESS; | ||||||||||||
| } | ||||||||||||
| if ((options & FileOptions.SequentialScan) != 0) | ||||||||||||
| { | ||||||||||||
| result |= CreateOptions.FILE_SEQUENTIAL_ONLY; | ||||||||||||
| } | ||||||||||||
| if ((options & FileOptions.DeleteOnClose) != 0) | ||||||||||||
| { | ||||||||||||
| result |= CreateOptions.FILE_DELETE_ON_CLOSE; // has extra handling in GetDesiredAccess | ||||||||||||
| } | ||||||||||||
| if ((options & FileOptions.Asynchronous) == 0) | ||||||||||||
| { | ||||||||||||
| // it's async by default, so we need to disable it when async was not requested | ||||||||||||
| result |= CreateOptions.FILE_SYNCHRONOUS_IO_NONALERT; // has extra handling in GetDesiredAccess | ||||||||||||
| } | ||||||||||||
| if (((int)options & 0x20000000) != 0) // NoBuffering | ||||||||||||
| { | ||||||||||||
| result |= CreateOptions.FILE_NO_INTERMEDIATE_BUFFERING; | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| return result; | ||||||||||||
| } | ||||||||||||
|
|
||||||||||||
| private static ObjectAttributes GetObjectAttributes(FileShare share) | ||||||||||||
| => (share & FileShare.Inheritable) != 0 ? ObjectAttributes.OBJ_INHERIT : 0; | ||||||||||||
|
|
||||||||||||
| /// <summary> | ||||||||||||
| /// File creation disposition when calling directly to NT APIs. | ||||||||||||
| /// </summary> | ||||||||||||
|
|
||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.