[PATCH v7 0/2] MR6904: ntdll.dll: Update NtQueryDirectoryFile to align with current Windows behaviour
This PR updates the behaviour of `NtQueryDirectoryFile`, bringing it in line with current Windows behaviour. The need for this update was discovered when attempting to build the Unreal Engine with MSVC under Wine. In certain cases conditional include statements do not behave as expected, due to MSVC depending on undocumented behaviour of `NtQueryDirectoryFile`. We ran tests on multiple versions of Windows, and discovered that the behaviour has changed since the original Wine implementation, but the documentation has not. The source code for our test tool, and a set of results can be found [here](https://github.com/TensorWorks/NtQueryDirectoryFile-Test). As of Windows 8, calling `NtQueryDirectoryFile` with a re-used handle, a new mask, and setting the `RestartScan` flag to True, causes the cached results to be erased and a new scan to be performed with the updated mask. Currently, Wine performs as did earlier versions of Windows, where the changed mask is ignored, and the cache is reused. This can cause `NtQueryDirectoryFile` under Wine to falsely report that files exist, when they do not. This PR corrects this behaviour, invalidating the cache when required. Implementing this exposed further undocumented behaviour of `NtQueryDirectoryFile`, where a search for a non-existent file will return either `STATUS_NO_MORE_FILES` or `STATUS_NO_SUCH_FILE`, depending on whether or not the handle had been previously used regardless of the value of `RestartScan`. This was reflected in a `winetest` which allowed for the response to be either `STATUS_SUCCESS` or `STATUS_NO_MORE_FILES`. This patch also adds unit tests for the new behaviour of `NtQueryDirectoryFile`. These tests pass when running `winetest` under Windows, and under Wine with these changes in place, but they will fail under older versions of Wine. If run under older versions of Windows the test suite will detect that this functionality is not supported, and will not run the updated tests. -- v7: ntdll: Update NtQueryDirectoryFile to purge cache if scan is reset with a new mask ntdll: Test updated NtQueryDirectoryFile mask reset behaviour https://gitlab.winehq.org/wine/wine/-/merge_requests/6904
From: Eugene McArdle <eugene(a)tensorworks.com.au> --- dlls/ntdll/tests/directory.c | 240 +++++++++++++++++++++++++++++++++++ 1 file changed, 240 insertions(+) diff --git a/dlls/ntdll/tests/directory.c b/dlls/ntdll/tests/directory.c index 52ff9936560..6aec2ff6408 100644 --- a/dlls/ntdll/tests/directory.c +++ b/dlls/ntdll/tests/directory.c @@ -964,6 +964,245 @@ done: pRtlFreeUnicodeString(&ntdirname); } +static void set_up_mask_test(const char *testdir) +{ + BOOL ret; + char buf[MAX_PATH + 5]; + HANDLE h; + + ret = CreateDirectoryA(testdir, NULL); + ok( ret, "couldn't create dir '%s', error %ld\n", testdir, GetLastError() ); + + sprintf(buf, "%s\\%s", testdir, "a-file.h"); + h = CreateFileA(buf, GENERIC_READ|GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, + FILE_ATTRIBUTE_NORMAL, 0); + ok( h != INVALID_HANDLE_VALUE, "failed to create temp file '%s'\n", buf ); + CloseHandle(h); + + sprintf(buf, "%s\\%s", testdir, "another-file.h"); + h = CreateFileA(buf, GENERIC_READ|GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, + FILE_ATTRIBUTE_NORMAL, 0); + ok( h != INVALID_HANDLE_VALUE, "failed to create temp file '%s'\n", buf ); + CloseHandle(h); +} + +static void tear_down_mask_test(const char *testdir) +{ + int ret; + char buf[MAX_PATH]; + + sprintf(buf, "%s\\%s", testdir, "a-file.h"); + ret = DeleteFileA(buf); + ok( ret || (GetLastError() == ERROR_FILE_NOT_FOUND) || (GetLastError() == ERROR_PATH_NOT_FOUND), + "Failed to rm %s, error %ld\n", buf, GetLastError() ); + + sprintf(buf, "%s\\%s", testdir, "another-file.h"); + ret = DeleteFileA(buf); + ok( ret || (GetLastError() == ERROR_FILE_NOT_FOUND) || (GetLastError() == ERROR_PATH_NOT_FOUND), + "Failed to rm %s, error %ld\n", buf, GetLastError() ); + + RemoveDirectoryA(testdir); +} + + +/* Performs a query with NtQueryDirectoryFile() */ +static BOOL test_NtQueryDirectoryFile_mask(HANDLE handle, BOOL restart_scan, UNICODE_STRING *mask, + NTSTATUS expected_status, BOOL check_name_and_length, BOOL validate_only) +{ + NTSTATUS status; + IO_STATUS_BLOCK io; + UINT data_size = sizeof(FILE_DIRECTORY_INFORMATION) + (MAX_PATH * sizeof(wchar_t)); + BYTE data[8192]; + FILE_DIRECTORY_INFORMATION *dir_info; + WCHAR *name; + ULONG name_len; + WCHAR *mask_value = {0}; + ULONG mask_len = 0; + + if (mask) + { + mask_len = mask->Length / sizeof(WCHAR); + mask_value = mask->Buffer; + } + + /* Perform the query */ + status = pNtQueryDirectoryFile( handle, NULL, NULL, NULL, &io, data, data_size, + FileDirectoryInformation, TRUE, mask, restart_scan ); + + if (validate_only && status != expected_status) return FALSE; + + todo_wine + ok( status == expected_status, "unexpected status : 0x%lx Test settings: file mask: '%s', restart: %d, expected status: 0x%lx\n", + status, wine_dbgstr_wn(mask_value, mask_len), restart_scan, expected_status ); + + /* Verify the results if required */ + if (status == 0 && check_name_and_length) + { + dir_info = (FILE_DIRECTORY_INFORMATION *)data; + name = dir_info->FileName; + name_len = dir_info->FileNameLength / sizeof(WCHAR); + + todo_wine + ok( name_len == mask_len, "unexpected filename length %lu, expected %lu\n", name_len, mask_len ); + todo_wine + ok( !memcmp(name, mask_value, mask_len), "unexpected filename %s, expected %s\n", + wine_dbgstr_wn(name, name_len), wine_dbgstr_wn(mask_value, mask_len) ); + } + return TRUE; +} + +static void test_NtQueryDirectoryFile_change_mask(void) +{ + NTSTATUS status; + HANDLE dirh; + HANDLE dirh_test_multiple_handles; + HANDLE dirh_test_fresh_null; + HANDLE dirh_test_fresh_empty; + char testdir[MAX_PATH]; + OBJECT_ATTRIBUTES attr; + IO_STATUS_BLOCK io; + UNICODE_STRING ntdirname; + WCHAR testdir_w[MAX_PATH]; + + UNICODE_STRING atestfile; + UNICODE_STRING anothertestfile; + UNICODE_STRING notatestfile; + UNICODE_STRING testmask; + UNICODE_STRING emptymask; + + pRtlInitUnicodeString(&atestfile, L"a-file.h"); + pRtlInitUnicodeString(&anothertestfile, L"another-file.h"); + pRtlInitUnicodeString(¬atestfile, L"not-a-file.h"); + pRtlInitUnicodeString(&testmask, L"*.h"); + pRtlInitUnicodeString(&emptymask, L""); + + /* Clean up from prior aborted run, if any, then set up test files */ + ok( GetTempPathA(MAX_PATH, testdir), "couldn't get temp dir\n" ); + strcat(testdir, "mask.tmp"); + tear_down_mask_test(testdir); + set_up_mask_test(testdir); + + pRtlMultiByteToUnicodeN(testdir_w, sizeof(testdir_w), NULL, testdir, strlen(testdir) + 1); + if (!pRtlDosPathNameToNtPathName_U(testdir_w, &ntdirname, NULL, NULL)) + { + ok( 0, "RtlDosPathNametoNtPathName_U failed\n" ); + goto done; + } + InitializeObjectAttributes(&attr, &ntdirname, OBJ_CASE_INSENSITIVE, 0, NULL); + + /* Open a handle for our test directory */ + status = pNtOpenFile(&dirh, SYNCHRONIZE | FILE_LIST_DIRECTORY, &attr, &io, FILE_SHARE_READ, + FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_FOR_BACKUP_INTENT | FILE_DIRECTORY_FILE); + ok( status == STATUS_SUCCESS, "failed to open dir '%s', ret 0x%lx, error %ld\n", testdir, status, GetLastError() ); + if (status != STATUS_SUCCESS) + { + skip("can't test if we can't open the directory\n"); + goto done; + } + + /* Open a handle for our test directory to test multiple handles */ + status = pNtOpenFile(&dirh_test_multiple_handles, SYNCHRONIZE | FILE_LIST_DIRECTORY, &attr, &io, FILE_SHARE_READ, + FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_FOR_BACKUP_INTENT | FILE_DIRECTORY_FILE); + ok( status == STATUS_SUCCESS, "failed to open second handle to dir '%s', ret 0x%lx, error %ld\n", testdir, status, GetLastError() ); + if (status != STATUS_SUCCESS) + { + skip("can't test if we can't open a second handle to the directory\n"); + goto done; + } + + /* Open a handle for our test directory to test null masks with a fresh handle */ + status = pNtOpenFile(&dirh_test_fresh_null, SYNCHRONIZE | FILE_LIST_DIRECTORY, &attr, &io, FILE_SHARE_READ, + FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_FOR_BACKUP_INTENT | FILE_DIRECTORY_FILE); + ok( status == STATUS_SUCCESS, "failed to open handle to dir '%s' for testing NULL masks, ret 0x%lx, error %ld\n", testdir, status, GetLastError() ); + if (status != STATUS_SUCCESS) + { + skip("can't test NULL masks if we can't open an additional handle to the directory\n"); + goto done; + } + + /* Open a handle for our test directory to test empty masks with a fresh handle */ + status = pNtOpenFile(&dirh_test_fresh_empty, SYNCHRONIZE | FILE_LIST_DIRECTORY, &attr, &io, FILE_SHARE_READ, + FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_FOR_BACKUP_INTENT | FILE_DIRECTORY_FILE); + ok( status == STATUS_SUCCESS, "failed to open handle to dir '%s' for testing empty masks, ret 0x%lx, error %ld\n", testdir, status, GetLastError() ); + if (status != STATUS_SUCCESS) + { + skip("can't test empty masks if we can't open an additional handle to the directory\n"); + goto done; + } + + /* Verify that NtQueryDirectoryFile mask reset behaviour introduced in Windows 8 is supported */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, TRUE); + if (!winetest_platform_is_wine && !test_NtQueryDirectoryFile_mask(dirh, TRUE, ¬atestfile, STATUS_NO_MORE_FILES, TRUE, TRUE)) + { + skip("Win8+ NtQueryDirectoryFile mask reset behaviour not supported\n"); + goto done; + } + + /* Test that caches for two handles to a single directory do not interfere with each other*/ + /* Search for a non-existent file, putting it into a state with an empty cache */ + /* This also confirms that using a non-existent mask with a fresh handle returns a different status*/ + test_NtQueryDirectoryFile_mask(dirh_test_multiple_handles, TRUE, ¬atestfile, STATUS_NO_SUCH_FILE, TRUE, FALSE); + /* Confirm that handle is in a state where it fails to find atestfile */ + test_NtQueryDirectoryFile_mask(dirh_test_multiple_handles, FALSE, &atestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + /* Confirm that another handle is able to find atestfile */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + + /* All searches for `notatestfile` are expected to fail */ + /* Tests should also fail if the scan is not reset, and the mask changes to an incompatible one */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, &anothertestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, ¬atestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, ¬atestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &anothertestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + + /* Test mask that matches multiple files, do not check results against the mask */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, ¬atestfile, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, ¬atestfile, STATUS_NO_MORE_FILES, FALSE, FALSE); + + /* Test NULL mask with a previously used handle that last returned an error */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, NULL, STATUS_NO_MORE_FILES, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, NULL, STATUS_NO_MORE_FILES, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, NULL, STATUS_NO_MORE_FILES, FALSE, FALSE); + + /* Test NULL mask with a previously used handle that last returned STATUS_SUCCESS */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, TRUE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, NULL, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, NULL, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, NULL, STATUS_SUCCESS, FALSE, FALSE); + + /* Test NULL mask with a fresh handle */ + test_NtQueryDirectoryFile_mask(dirh_test_fresh_null, TRUE, NULL, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh_test_fresh_null, FALSE, NULL, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh_test_fresh_null, TRUE, NULL, STATUS_SUCCESS, FALSE, FALSE); + + /* Test empty mask with a previously used handle that last returned an error */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, ¬atestfile, STATUS_NO_MORE_FILES, FALSE, TRUE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, &emptymask, STATUS_NO_MORE_FILES, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &emptymask, STATUS_NO_MORE_FILES, FALSE, FALSE); + + /* Test empty mask with a previously used handle that last returned STATUS_SUCCESS */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, TRUE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + + /* Test empty mask with a fresh handle */ + test_NtQueryDirectoryFile_mask(dirh_test_fresh_empty, TRUE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh_test_fresh_empty, FALSE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + + /* Cleanup */ +done: + tear_down_mask_test(testdir); + pRtlFreeUnicodeString(&ntdirname); + pRtlFreeUnicodeString(&atestfile); + pRtlFreeUnicodeString(&anothertestfile); + pRtlFreeUnicodeString(¬atestfile); +} + static NTSTATUS get_file_id( FILE_INTERNAL_INFORMATION *info, const WCHAR *root, const WCHAR *name ) { OBJECT_ATTRIBUTES attr; @@ -1157,5 +1396,6 @@ START_TEST(directory) test_directory_sort( sysdir ); test_NtQueryDirectoryFile(); test_NtQueryDirectoryFile_case(); + test_NtQueryDirectoryFile_change_mask(); test_redirection(); } -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/6904
From: Eugene McArdle <eugene(a)tensorworks.com.au> --- dlls/ntdll/tests/directory.c | 3 -- dlls/ntdll/unix/file.c | 57 ++++++++++++++++++++++++++++++++++-- 2 files changed, 54 insertions(+), 6 deletions(-) diff --git a/dlls/ntdll/tests/directory.c b/dlls/ntdll/tests/directory.c index 6aec2ff6408..9d7ed2f4841 100644 --- a/dlls/ntdll/tests/directory.c +++ b/dlls/ntdll/tests/directory.c @@ -1031,7 +1031,6 @@ static BOOL test_NtQueryDirectoryFile_mask(HANDLE handle, BOOL restart_scan, UNI if (validate_only && status != expected_status) return FALSE; - todo_wine ok( status == expected_status, "unexpected status : 0x%lx Test settings: file mask: '%s', restart: %d, expected status: 0x%lx\n", status, wine_dbgstr_wn(mask_value, mask_len), restart_scan, expected_status ); @@ -1042,9 +1041,7 @@ static BOOL test_NtQueryDirectoryFile_mask(HANDLE handle, BOOL restart_scan, UNI name = dir_info->FileName; name_len = dir_info->FileNameLength / sizeof(WCHAR); - todo_wine ok( name_len == mask_len, "unexpected filename length %lu, expected %lu\n", name_len, mask_len ); - todo_wine ok( !memcmp(name, mask_value, mask_len), "unexpected filename %s, expected %s\n", wine_dbgstr_wn(name, name_len), wine_dbgstr_wn(mask_value, mask_len) ); } diff --git a/dlls/ntdll/unix/file.c b/dlls/ntdll/unix/file.c index 13dc36b42ff..097aaaca4b9 100644 --- a/dlls/ntdll/unix/file.c +++ b/dlls/ntdll/unix/file.c @@ -225,6 +225,7 @@ struct dir_data struct file_identity id; /* directory file identity */ struct dir_data_names *names; /* directory file names */ struct dir_data_buffer *buffer; /* head of data buffers list */ + UNICODE_STRING mask; /* the mask used when creating the cache entry */ }; static const unsigned int dir_data_buffer_initial_size = 4096; @@ -565,6 +566,7 @@ static void free_dir_data( struct dir_data *data ) free( buffer ); } free( data->names ); + if (data->mask.Buffer) free( data->mask.Buffer ); free( data ); } @@ -1595,7 +1597,7 @@ static BOOL append_entry( struct dir_data *data, const char *long_name, TRACE( "long %s short %s mask %s\n", debugstr_w( long_nameW ), debugstr_w( short_nameW ), debugstr_us( mask )); - if (mask && !match_filename( long_nameW, long_len, mask )) + if (mask && mask->Length !=0 && !match_filename( long_nameW, long_len, mask )) { if (!short_len) return TRUE; /* no short name to match */ if (!match_filename( short_nameW, short_len, mask )) return TRUE; @@ -2606,6 +2608,14 @@ static NTSTATUS init_cached_dir_data( struct dir_data **data_ret, int fd, const return status; } + /* if a mask was specified then copy it into the cache entry */ + if (mask && mask->Length) + { + data->mask.Length = data->mask.MaximumLength = mask->Length; + if (!(data->mask.Buffer = calloc( 1, mask->Length ))) return STATUS_NO_MEMORY; + memcpy(data->mask.Buffer, mask->Buffer, mask->Length); + } + /* sort filenames, but not "." and ".." */ i = 0; if (i < data->count && !strcmp( data->names[i].unix_name, "." )) i++; @@ -2628,17 +2638,35 @@ static NTSTATUS init_cached_dir_data( struct dir_data **data_ret, int fd, const } +/*********************************************************************** + * ustring_equal + * + * Simplified version of RtlEqualUnicodeString that performs only case-sensitive comparisons. + */ +static BOOLEAN ustring_equal( const UNICODE_STRING *a, const UNICODE_STRING *b ) +{ + USHORT length_a = (a ? a->Length : 0); + USHORT length_b = (b ? b->Length : 0); + if (length_a != length_b) return FALSE; + + if (length_a == 0 && length_b == 0) return TRUE; + return (!memcmp(a->Buffer, b->Buffer, a->Length)); +} + + /*********************************************************************** * get_cached_dir_data * * Retrieve the cached directory data, or initialize it if necessary. */ static unsigned int get_cached_dir_data( HANDLE handle, struct dir_data **data_ret, int fd, - const UNICODE_STRING *mask ) + const UNICODE_STRING *mask, BOOLEAN restart_scan, + BOOLEAN *fresh_handle ) { unsigned int i; int entry = -1, free_entries[16]; unsigned int status; + *fresh_handle = TRUE; SERVER_START_REQ( get_directory_cache_entry ) { @@ -2675,6 +2703,21 @@ static unsigned int get_cached_dir_data( HANDLE handle, struct dir_data **data_r dir_data_cache_size = size; } + /* If we have an existing cache entry then set the flag to indicate that the handle is not fresh */ + if (dir_data_cache[entry]) *fresh_handle = FALSE; + + /* If we have an existing cache entry, but restart_scan is true and a new non-empty mask + was specified then we need to invalidate the existing cache entry and create a new one + */ + if (dir_data_cache[entry] && restart_scan && mask && mask->Length != 0 && + !ustring_equal(&dir_data_cache[entry]->mask, mask)) + { + TRACE( "invalidating existing cache entry for handle %p, old mask: \"%s\", new mask: \"%s\"\n", + handle, debugstr_us(&(dir_data_cache[entry]->mask)), debugstr_us(mask)); + free_dir_data( dir_data_cache[entry] ); + dir_data_cache[entry] = NULL; + } + if (!dir_data_cache[entry]) status = init_cached_dir_data( &dir_data_cache[entry], fd, mask ); *data_ret = dir_data_cache[entry]; @@ -2694,6 +2737,7 @@ NTSTATUS WINAPI NtQueryDirectoryFile( HANDLE handle, HANDLE event, PIO_APC_ROUTI enum server_fd_type type; struct dir_data *data; unsigned int status; + BOOLEAN fresh_handle = TRUE; TRACE("(%p %p %p %p %p %p 0x%08x 0x%08x 0x%08x %s 0x%08x\n", handle, event, apc_routine, apc_context, io, buffer, @@ -2746,7 +2790,7 @@ NTSTATUS WINAPI NtQueryDirectoryFile( HANDLE handle, HANDLE event, PIO_APC_ROUTI cwd = open( ".", O_RDONLY ); if (fchdir( fd ) != -1) { - if (!(status = get_cached_dir_data( handle, &data, fd, mask ))) + if (!(status = get_cached_dir_data( handle, &data, fd, mask, restart_scan, &fresh_handle ))) { union file_directory_info *last_info = NULL; @@ -2768,6 +2812,13 @@ NTSTATUS WINAPI NtQueryDirectoryFile( HANDLE handle, HANDLE event, PIO_APC_ROUTI } else status = errno_to_status( errno ); + /* Return STATUS_NO_MORE_FILES if a mask did not match a file and the handle had previously been used */ + if (status == STATUS_NO_SUCH_FILE && !fresh_handle) + { + status = STATUS_NO_MORE_FILES; + io->Status = status; + } + mutex_unlock( &dir_mutex ); if (needs_close) close( fd ); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/6904
Hi, It looks like your patch introduced the new failures shown below. Please investigate and fix them before resubmitting your patch. If they are not new, fixing them anyway would help a lot. Otherwise please ask for the known failures list to be updated. The tests also ran into some preexisting test failures. If you know how to fix them that would be helpful. See the TestBot job for the details: The full results can be found at: https://testbot.winehq.org/JobDetails.pl?Key=150436 Your paranoid android. === w7pro64 (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w864 (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w1064v1507 (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w1064v1809 (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w1064_2qxl (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w1064_adm (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w1064_tsign (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w10pro64 (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w10pro64_ar (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w10pro64_ja (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w10pro64_zh_CN (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374) === w11pro64_amd (64 bit report) === Report validation errors: ntdll:directory crashed (c0000374)
@epo I've added several tests including testing null masks and empty masks, which exposed a shortcoming in the cache reset in a very specific edge case. That's also been resolved in this pair of commits. I've squashed everything into two commits, and as requested I've flagged tests with `todo_wine` in the first, and then in the second I add the updated Wine behaviour, and remove the `todo_wine` flags. I haven't yet rearranged the code in file.c, pending your response to my question above. Once we're happy with this I'll rebase the PR to bring it up-to-date -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_90843
@epo Can I ask you to take a look at the latest version of these changes please? If there are any outstanding issues please let me know :thumbsup: -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_91817
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ + todo_wine + ok( status == expected_status, "unexpected status : 0x%lx Test settings: file mask: '%s', restart: %d, expected status: 0x%lx\n", + status, wine_dbgstr_wn(mask_value, mask_len), restart_scan, expected_status ); + + /* Verify the results if required */ + if (status == 0 && check_name_and_length) + { + dir_info = (FILE_DIRECTORY_INFORMATION *)data; + name = dir_info->FileName; + name_len = dir_info->FileNameLength / sizeof(WCHAR); + + todo_wine + ok( name_len == mask_len, "unexpected filename length %lu, expected %lu\n", name_len, mask_len ); + todo_wine + ok( !memcmp(name, mask_value, mask_len), "unexpected filename %s, expected %s\n", any reason to test half of the string?
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92913
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ WCHAR *mask_value = {0}; + ULONG mask_len = 0; + + if (mask) + { + mask_len = mask->Length / sizeof(WCHAR); + mask_value = mask->Buffer; + } + + /* Perform the query */ + status = pNtQueryDirectoryFile( handle, NULL, NULL, NULL, &io, data, data_size, + FileDirectoryInformation, TRUE, mask, restart_scan ); + + if (validate_only && status != expected_status) return FALSE; + + todo_wine this generates tons of failures...
as you're fixing it in next patch, it's fine to use todo_wine_if(status != expected_status) same remark applies to the other todo_wine calls in this function -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92914
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ goto done; + } + InitializeObjectAttributes(&attr, &ntdirname, OBJ_CASE_INSENSITIVE, 0, NULL); + + /* Open a handle for our test directory */ + status = pNtOpenFile(&dirh, SYNCHRONIZE | FILE_LIST_DIRECTORY, &attr, &io, FILE_SHARE_READ, + FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_FOR_BACKUP_INTENT | FILE_DIRECTORY_FILE); + ok( status == STATUS_SUCCESS, "failed to open dir '%s', ret 0x%lx, error %ld\n", testdir, status, GetLastError() ); + if (status != STATUS_SUCCESS) + { + skip("can't test if we can't open the directory\n"); + goto done; + } + + /* Open a handle for our test directory to test multiple handles */ + status = pNtOpenFile(&dirh_test_multiple_handles, SYNCHRONIZE | FILE_LIST_DIRECTORY, &attr, &io, FILE_SHARE_READ, nitpick: for readibility of the tests, you'd rather move that close to the part of the tests where you use the created handle
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92912
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ sprintf(buf, "%s\\%s", testdir, "a-file.h"); + ret = DeleteFileA(buf); + ok( ret || (GetLastError() == ERROR_FILE_NOT_FOUND) || (GetLastError() == ERROR_PATH_NOT_FOUND), + "Failed to rm %s, error %ld\n", buf, GetLastError() ); + + sprintf(buf, "%s\\%s", testdir, "another-file.h"); + ret = DeleteFileA(buf); + ok( ret || (GetLastError() == ERROR_FILE_NOT_FOUND) || (GetLastError() == ERROR_PATH_NOT_FOUND), + "Failed to rm %s, error %ld\n", buf, GetLastError() ); + + RemoveDirectoryA(testdir); +} + + +/* Performs a query with NtQueryDirectoryFile() */ +static BOOL test_NtQueryDirectoryFile_mask(HANDLE handle, BOOL restart_scan, UNICODE_STRING *mask, it looks to me that the test would be more complete if instead of check_name_and_length argument you'd pass a unicode string (assuming all STATUS_SUCCESS should have a matching UNICODE_STRING), while the status returning an error should have a NULL unicode string. and the test below for the returned string should be tested agains that string and not the input mask
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92915
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ test_NtQueryDirectoryFile_mask(dirh, TRUE, &emptymask, STATUS_NO_MORE_FILES, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &emptymask, STATUS_NO_MORE_FILES, FALSE, FALSE); + + /* Test empty mask with a previously used handle that last returned STATUS_SUCCESS */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, TRUE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + + /* Test empty mask with a fresh handle */ + test_NtQueryDirectoryFile_mask(dirh_test_fresh_empty, TRUE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh_test_fresh_empty, FALSE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + + /* Cleanup */ +done: + tear_down_mask_test(testdir); + pRtlFreeUnicodeString(&ntdirname); RtlInitUnicodeString doesn't allocate any memory, so UNICODE_STRING shouldn't be freed
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92918
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, &anothertestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, ¬atestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, ¬atestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &anothertestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + + /* Test mask that matches multiple files, do not check results against the mask */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, ¬atestfile, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, ¬atestfile, STATUS_NO_MORE_FILES, FALSE, FALSE); + + /* Test NULL mask with a previously used handle that last returned an error */ nitpick: it would be more readable if each test sequece was independent of previous one (even if this requires adding another step to the sequence)
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92920
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, TRUE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + + /* Test empty mask with a fresh handle */ + test_NtQueryDirectoryFile_mask(dirh_test_fresh_empty, TRUE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh_test_fresh_empty, FALSE, &emptymask, STATUS_SUCCESS, FALSE, FALSE); + + /* Cleanup */ +done: + tear_down_mask_test(testdir); + pRtlFreeUnicodeString(&ntdirname); + pRtlFreeUnicodeString(&atestfile); + pRtlFreeUnicodeString(&anothertestfile); + pRtlFreeUnicodeString(¬atestfile); +} but you're leaking all the created handles... from comment above, for the ones that are really local to a couple of lines, NtClose should be put within the same code area... dirh should be closed here
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92919
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ /* Clean up from prior aborted run, if any, then set up test files */ + ok( GetTempPathA(MAX_PATH, testdir), "couldn't get temp dir\n" ); + strcat(testdir, "mask.tmp"); + tear_down_mask_test(testdir); + set_up_mask_test(testdir); + + pRtlMultiByteToUnicodeN(testdir_w, sizeof(testdir_w), NULL, testdir, strlen(testdir) + 1); + if (!pRtlDosPathNameToNtPathName_U(testdir_w, &ntdirname, NULL, NULL)) + { + ok( 0, "RtlDosPathNametoNtPathName_U failed\n" ); + goto done; + } + InitializeObjectAttributes(&attr, &ntdirname, OBJ_CASE_INSENSITIVE, 0, NULL); + + /* Open a handle for our test directory */ + status = pNtOpenFile(&dirh, SYNCHRONIZE | FILE_LIST_DIRECTORY, &attr, &io, FILE_SHARE_READ, nitpick: indentation is wrong
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92916
eric pouech (@epo) commented about dlls/ntdll/tests/directory.c:
+ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &atestfile, STATUS_SUCCESS, TRUE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, &anothertestfile, STATUS_NO_MORE_FILES, TRUE, FALSE); + + /* Test mask that matches multiple files, do not check results against the mask */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, ¬atestfile, STATUS_SUCCESS, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, ¬atestfile, STATUS_NO_MORE_FILES, FALSE, FALSE); + + /* Test NULL mask with a previously used handle that last returned an error */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, NULL, STATUS_NO_MORE_FILES, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, FALSE, NULL, STATUS_NO_MORE_FILES, FALSE, FALSE); + test_NtQueryDirectoryFile_mask(dirh, TRUE, NULL, STATUS_NO_MORE_FILES, FALSE, FALSE); + + /* Test NULL mask with a previously used handle that last returned STATUS_SUCCESS */ + test_NtQueryDirectoryFile_mask(dirh, TRUE, &testmask, STATUS_SUCCESS, FALSE, TRUE); IMO, theres' no reason validate_only should be TRUE here (same applies below)
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92917
eric pouech (@epo) commented about dlls/ntdll/unix/file.c:
TRACE( "long %s short %s mask %s\n", debugstr_w( long_nameW ), debugstr_w( short_nameW ), debugstr_us( mask ));
- if (mask && !match_filename( long_nameW, long_len, mask )) + if (mask && mask->Length !=0 && !match_filename( long_nameW, long_len, mask ))
not sure it's needed -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92922
eric pouech (@epo) commented about dlls/ntdll/unix/file.c:
struct file_identity id; /* directory file identity */ struct dir_data_names *names; /* directory file names */ struct dir_data_buffer *buffer; /* head of data buffers list */ + UNICODE_STRING mask; /* the mask used when creating the cache entry */
the only use of keeping that string is to avoid to flush the cache when passing rescan + the same string as the previous one... I'm not convinced it's worth it -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92921
eric pouech (@epo) commented about dlls/ntdll/unix/file.c:
+ USHORT length_b = (b ? b->Length : 0); + if (length_a != length_b) return FALSE; + + if (length_a == 0 && length_b == 0) return TRUE; + return (!memcmp(a->Buffer, b->Buffer, a->Length)); +} + + /*********************************************************************** * get_cached_dir_data * * Retrieve the cached directory data, or initialize it if necessary. */ static unsigned int get_cached_dir_data( HANDLE handle, struct dir_data **data_ret, int fd, - const UNICODE_STRING *mask ) + const UNICODE_STRING *mask, BOOLEAN restart_scan, you should be able to write the changes without adding the fresh_handle parameter:
* insert the NO_SUCH_FILE =\> NO_MORE_FILES fixup after calling init_cache_data_dir (in the case the cache entry already exists) * you'll have to tweak the calling code to always set io-\>Status = status (this will in turn show a failing test in other directory tests, but native Win10 here has the same behavior, so this test must be adapted first) -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92923
@eugenemcardle hi... sorry for the late reply, been busy on some other stuffs a bunch of comments to improve the merge request (the overall logic looks ok, the tests are covering most of the interesting aspects, mostly code cleanup, simplification & readbility) -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_92924
On Mon Jan 27 13:49:14 2025 +0000, eric pouech wrote:
the only use of keeping that string is to avoid to flush the cache when passing rescan + the same string as the previous one... I'm not convinced it's worth it Removing this field and its associated checks would result in flushing the cache every time the scan is restarted, regardless of whether the mask has changed. This would represent something of a departure from the existing approach to caching, which is to reuse the cache wherever possible.
Given that flushing the cache involves making an RPC call to the Wineserver, there is the potential that a minor difference in behaviour here could have meaningful performance implications for applications that restart scans frequently (particularly if applications do so concurrently from multiple processes or threads, resulting in contention over the Wineserver RPC call). We'll conduct some performance analysis to determine the impact on relevant workloads and post the results here to inform further discussion. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/6904#note_93143
participants (5)
-
Adam Rehn (＠adamrehn) -
eric pouech (＠epo)
-
Eugene McArdle
-
Eugene McArdle (＠eugenemcardle)
-
Marvin