[PATCH v28 0/8] MR5769: Add support for large page backed file mapping objects on Linux.
This MR adds support for creating file mapping objects backed by large pages on Linux, by making the following changes:
## wineserver
* On Linux, `create_temp_file` will first attempt to use memfds as the backing fd. If it fails, it'll return to the current codepath, creating a temporary file in either the server or config directory. * The created memfd will be sealed against writes, if the caller requesting the appropriate page protection flags. * This removes the requirement that FDs be only created on filesystems/directories that aren't `noexec`. * In the server method `create_mapping` , if large pages have been requested by the caller, hold that the calling thread's token holds `SeLockMemoryPrivilege` . * Additionally, add `SeLockMemoryPrivilege` to the list of privileges enabled for the Administrator.
## `ntdll`
* Add `virtual_get_min_large_page_size` and its exported wrapper `wine_unix_get_min_large_page_size`. * On Linux, the minimum page size is determined by going through `/sys/kernel/mm/hugepages`. If hugepage support was not detected, `STATUS_NOT_SUPPORTED` is returned instead. On other platforms, the older hard-coded value of 2*1024*1024 is returned instead. * `NtCreateSection` will validate certain parameters if large pages are requested. Specifically, it will return STATUS_INVALID_PARAMETER if the requested mapping is not anonymous/unnamed, or the size is not a multiple of the minimum supported page size.
## `kernelbase`
* `GetLargePageMinimum` will use `wine_unix_get_min_large_page_size`.
## `kernel32/tests`
* Add new test test_large_page_file_mapping, which validates privilege enforcements and parameter validation while creating large pages backed file mapping obejcts. The tests are skipped if `GetLargePageMinimum` returns 0.
-- v28: psapi: Add tests for querying information for large pages. ntdll: Use PAGEMAP_SCAN ioctl to implement get_working_set_ex, if available. kernel32: Add tests for large pages support. ntdll: Support allocating virtual memory, creating and mapping files backed by large pages (SEC_LARGE_PAGES, MEM_LARGE_PAGES).
From: Vibhav Pant vibhavp@gmail.com
Also, add SeLockMemoryPrivilege to the list of admin privileges. --- server/mapping.c | 6 ++++++ server/security.h | 1 + server/token.c | 2 ++ 3 files changed, 9 insertions(+)
diff --git a/server/mapping.c b/server/mapping.c index 92eb0c9f076..8f7103dcf14 100644 --- a/server/mapping.c +++ b/server/mapping.c @@ -1442,6 +1442,12 @@ DECL_HANDLER(create_mapping)
if (!objattr) return;
+ if ((req->flags & SEC_LARGE_PAGES) && !thread_single_check_privilege( current, SeLockMemoryPrivilege )) + { + set_error( STATUS_PRIVILEGE_NOT_HELD ); + return; + } + if ((mapping = create_mapping( root, &name, objattr->attributes, req->size, req->flags, req->file_handle, req->file_access, sd ))) { diff --git a/server/security.h b/server/security.h index f4dff679179..7a135e90928 100644 --- a/server/security.h +++ b/server/security.h @@ -23,6 +23,7 @@
#include <sys/types.h>
+extern const struct luid SeLockMemoryPrivilege; extern const struct luid SeIncreaseQuotaPrivilege; extern const struct luid SeSecurityPrivilege; extern const struct luid SeTakeOwnershipPrivilege; diff --git a/server/token.c b/server/token.c index da7f0bb7ff2..b1fd23af492 100644 --- a/server/token.c +++ b/server/token.c @@ -42,6 +42,7 @@
#define MAX_SUBAUTH_COUNT 1
+const struct luid SeLockMemoryPrivilege = { 4, 0}; const struct luid SeIncreaseQuotaPrivilege = { 5, 0 }; const struct luid SeTcbPrivilege = { 7, 0 }; const struct luid SeSecurityPrivilege = { 8, 0 }; @@ -782,6 +783,7 @@ struct token *token_create_admin( unsigned primary, int impersonation_level, int { SeManageVolumePrivilege, 0 }, { SeImpersonatePrivilege, SE_PRIVILEGE_ENABLED }, { SeCreateGlobalPrivilege, SE_PRIVILEGE_ENABLED }, + { SeLockMemoryPrivilege, SE_PRIVILEGE_ENABLED }, }; /* note: we don't include non-builtin groups here for the user - * telling us these is the job of a client-side program */
From: Vibhav Pant vibhavp@gmail.com
On Linux, get the minimum supported hugepage size from directory entries /sys/kernel/mm/hugepages, and set the LargePageMinimum field in user_shared_data to it, if supported. --- server/mapping.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+)
diff --git a/server/mapping.c b/server/mapping.c index 8f7103dcf14..30be548fd53 100644 --- a/server/mapping.c +++ b/server/mapping.c @@ -29,6 +29,14 @@ #include <sys/stat.h> #include <sys/mman.h> #include <unistd.h> +#ifdef HAVE_MEMFD_CREATE +#include <linux/memfd.h> +#endif +#ifdef __linux__ /* For detecting hugepage support. */ +#include <dirent.h> +#include <errno.h> +#include <limits.h> +#endif
#include "ntstatus.h" #define WIN32_NO_STATUS @@ -1414,6 +1422,66 @@ obj_locator_t get_shared_object_locator( const volatile void *object_shm ) return locator; }
+#ifdef __linux__ +static size_t linux_get_min_hugepage_size( void ) +{ + DIR *sysfs_hugepages; + struct dirent *supported_size; + size_t min_size = 0; + size_t total_supported_sizes = 0; + + errno = 0; + sysfs_hugepages = opendir("/sys/kernel/mm/hugepages"); + if (sysfs_hugepages == NULL) + { + return -1; + } + + while(1) + { + long hugepage_size; + + supported_size = readdir( sysfs_hugepages ); + if ( supported_size == NULL ) + { + break; + } + if (strncmp( supported_size->d_name, "hugepages-", 10) != 0) + { + continue; + } + errno = 0; + hugepage_size = strtol( &supported_size->d_name[10], NULL, 10 ); + if (hugepage_size == 0 || hugepage_size == LONG_MAX || hugepage_size == LONG_MIN) + { + if (errno != 0) + { + fprintf( stderr, + "could not parse page size from directory entry '%s': %s\n", + supported_size->d_name, strerror( errno ) ); + } + continue; + } + hugepage_size *= 1024; + min_size = ( total_supported_sizes == 0 ) + ? hugepage_size + : ( hugepage_size < min_size ? hugepage_size : min_size ); + total_supported_sizes++; + } + + closedir(sysfs_hugepages); + return min_size; +} +#endif /* __linux__ */ + +static size_t get_min_large_page_size( void ) +{ +#ifdef __linux__ + return linux_get_min_hugepage_size(); +#endif + return 0; +} + struct object *create_user_data_mapping( struct object *root, const struct unicode_str *name, unsigned int attr, const struct security_descriptor *sd ) { @@ -1425,8 +1493,15 @@ struct object *create_user_data_mapping( struct object *root, const struct unico ptr = mmap( NULL, mapping->size, PROT_WRITE, MAP_SHARED, get_unix_fd( mapping->fd ), 0 ); if (ptr != MAP_FAILED) { + ULONG min_large_page_size; + user_shared_data = ptr; user_shared_data->SystemCall = 1; + min_large_page_size = get_min_large_page_size(); + if (min_large_page_size != 0) + { + user_shared_data->LargePageMinimum = min_large_page_size; + } } return &mapping->obj; }
From: Vibhav Pant vibhavp@gmail.com
--- dlls/kernelbase/memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/dlls/kernelbase/memory.c b/dlls/kernelbase/memory.c index 46b155b7a47..f5a1358b0e9 100644 --- a/dlls/kernelbase/memory.c +++ b/dlls/kernelbase/memory.c @@ -41,6 +41,7 @@ WINE_DEFAULT_DEBUG_CHANNEL(heap); WINE_DECLARE_DEBUG_CHANNEL(virtual); WINE_DECLARE_DEBUG_CHANNEL(globalmem);
+static const struct _KUSER_SHARED_DATA *user_shared_data = (struct _KUSER_SHARED_DATA *)0x7ffe0000;
static CROSS_PROCESS_WORK_LIST *open_cross_process_connection( HANDLE process ) @@ -143,7 +144,7 @@ BOOL WINAPI DECLSPEC_HOTPATCH FlushInstructionCache( HANDLE process, LPCVOID add */ SIZE_T WINAPI GetLargePageMinimum(void) { - return 2 * 1024 * 1024; + return user_shared_data->LargePageMinimum; }
From: Vibhav Pant vibhavp@gmail.com
On Linux, try creating a memfd first to implement create_temp_file. If a large page mapping is requested (SEC_LARGE_PAGES), create a memfd with the MFD_HUGETLB flags. If SEC_COMMIT is passed, allocate sufficient space to back the mapping with posix_fallocate.
Additionally, ensure when SEC_LARGE_PAGES is requested, ensure that SEC_COMMIT is passed as well. --- configure | 12 +++++ configure.ac | 2 + include/config.h.in | 6 +++ server/mapping.c | 118 ++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 134 insertions(+), 4 deletions(-)
diff --git a/configure b/configure index c506c72ad8b..15cf8f98459 100755 --- a/configure +++ b/configure @@ -7989,6 +7989,12 @@ if test "x$ac_cv_header_linux_major_h" = xyes then : printf "%s\n" "#define HAVE_LINUX_MAJOR_H 1" >>confdefs.h
+fi +ac_fn_c_check_header_compile "$LINENO" "linux/memfd.h" "ac_cv_header_linux_memfd_h" "$ac_includes_default" +if test "x$ac_cv_header_linux_memfd_h" = xyes +then : + printf "%s\n" "#define HAVE_LINUX_MEMFD_H 1" >>confdefs.h + fi ac_fn_c_check_header_compile "$LINENO" "linux/param.h" "ac_cv_header_linux_param_h" "$ac_includes_default" if test "x$ac_cv_header_linux_param_h" = xyes @@ -20813,6 +20819,12 @@ if test "x$ac_cv_func_mach_continuous_time" = xyes then : printf "%s\n" "#define HAVE_MACH_CONTINUOUS_TIME 1" >>confdefs.h
+fi +ac_fn_c_check_func "$LINENO" "memfd_create" "ac_cv_func_memfd_create" +if test "x$ac_cv_func_memfd_create" = xyes +then : + printf "%s\n" "#define HAVE_MEMFD_CREATE 1" >>confdefs.h + fi ac_fn_c_check_func "$LINENO" "pipe2" "ac_cv_func_pipe2" if test "x$ac_cv_func_pipe2" = xyes diff --git a/configure.ac b/configure.ac index 7fe5477daf7..8e68a09ad0a 100644 --- a/configure.ac +++ b/configure.ac @@ -381,6 +381,7 @@ AC_CHECK_HEADERS(\ linux/input.h \ linux/ioctl.h \ linux/major.h \ + linux/memfd.h \ linux/param.h \ linux/serial.h \ linux/types.h \ @@ -2080,6 +2081,7 @@ AC_CHECK_FUNCS(\ getrandom \ kqueue \ mach_continuous_time \ + memfd_create \ pipe2 \ port_create \ posix_fadvise \ diff --git a/include/config.h.in b/include/config.h.in index aa066a87ad7..bbd5a57a280 100644 --- a/include/config.h.in +++ b/include/config.h.in @@ -180,6 +180,9 @@ /* Define to 1 if you have the <linux/major.h> header file. */ #undef HAVE_LINUX_MAJOR_H
+/* Define to 1 if you have the <linux/memfd.h> header file. */ +#undef HAVE_LINUX_MEMFD_H + /* Define to 1 if you have the <linux/param.h> header file. */ #undef HAVE_LINUX_PARAM_H
@@ -219,6 +222,9 @@ /* Define to 1 if you have the <mach-o/loader.h> header file. */ #undef HAVE_MACH_O_LOADER_H
+/* Define to 1 if you have the 'memfd_create' function. */ +#undef HAVE_MEMFD_CREATE + /* Define to 1 if you have the <mntent.h> header file. */ #undef HAVE_MNTENT_H
diff --git a/server/mapping.c b/server/mapping.c index 30be548fd53..04b35d6ec4a 100644 --- a/server/mapping.c +++ b/server/mapping.c @@ -355,13 +355,117 @@ static int check_current_dir_for_exec(void) return (ret != MAP_FAILED); }
+#ifdef HAVE_MEMFD_CREATE +static void make_memfd_name( const char *prefix, ULONG sec_flags, char *dest ) +{ + /* memfd names are restricted to 250 bytes including \0, but are only meant for debugging, with + * duplicate names not causing any side-effects. */ + + if (prefix != NULL) + { + snprintf( dest, 250, "map-%s-%x", prefix, (int)sec_flags ); + } + else + { + snprintf( dest, 250, "map-%x", (int)sec_flags ); + } +} + +static int create_memfd( ULONG file_access, ULONG sec_flags, file_pos_t new_size, int *err ) +{ + int fd; + off_t size = new_size; + char memfd_name[256]; + unsigned int memfd_flags = MFD_ALLOW_SEALING; + unsigned int seal_flags = F_SEAL_SEAL | F_SEAL_WRITE; + +#ifdef HAVE_POSIX_FALLOCATE + if (sizeof(file_pos_t) > sizeof(off_t) && size != new_size) + { + *err = EINVAL; + return -1; + } +#endif +#ifdef MFD_EXEC + memfd_flags |= MFD_EXEC; +#endif + + make_memfd_name( NULL, sec_flags, memfd_name ); + + if (sec_flags & SEC_LARGE_PAGES) + { + memfd_flags |= MFD_HUGETLB; + } + fd = memfd_create( memfd_name, memfd_flags ); + if (fd == -1) + { + if (errno == EINVAL && (sec_flags & SEC_LARGE_PAGES)) + { + /* MFD_HUGETLB & MFD_ALLOW_SEALING is only available in Linux >= 4.16. Lets try creating + * one without HUGETLB */ + fd = memfd_create( memfd_name, memfd_flags & ~MFD_HUGETLB ); + if (fd == -1) + { + *err = errno; + return -1; + } + } + else + { + *err = errno; + return -1; + } + } +#ifdef HAVE_POSIX_FALLOCATE + if (sec_flags & SEC_COMMIT) + { + *err = posix_fallocate( fd, 0, size ); + if (*err != 0) + { + close( fd ); + return -1; + } + } +#endif + if (ftruncate( fd, size ) == -1) + { + *err = errno; + close( fd ); + return -1; + } + if (file_access & FILE_WRITE_DATA) + { + seal_flags &= ~F_SEAL_WRITE; + } + + if (fcntl( fd, F_ADD_SEALS, F_SEAL_SEAL, seal_flags ) == -1) + { + *err = errno; + close( fd ); + return -1; + } + return fd; +} +#endif /* defined(HAVE_MEMFD_CREATE) */ + /* create a temp file for anonymous mappings */ -static int create_temp_file( file_pos_t size ) +static int create_temp_file( ULONG file_access, ULONG sec_flags, file_pos_t size ) { static int temp_dir_fd = -1; char tmpfn[16]; int fd;
+#ifdef HAVE_MEMFD_CREATE + int err = 0; + + fd = create_memfd( file_access, sec_flags, size, &err); + if (fd != -1) + { + return fd; + } + +#endif + if (temp_dir_fd == -1) { temp_dir_fd = server_dir_fd; @@ -646,7 +750,7 @@ static int build_shared_mapping( struct mapping *mapping, int fd,
/* create a temp file for the mapping */
- if ((shared_fd = create_temp_file( total_size )) == -1) return 0; + if ((shared_fd = create_temp_file( FILE_WRITE_DATA, 0, total_size )) == -1) return 0; if (!(file = create_file_for_fd( shared_fd, FILE_GENERIC_READ|FILE_GENERIC_WRITE, 0 ))) return 0;
if (!(buffer = malloc( max_size ))) goto error; @@ -978,7 +1082,12 @@ static struct ranges *create_ranges(void)
static unsigned int get_mapping_flags( obj_handle_t handle, unsigned int flags ) { - switch (flags & (SEC_IMAGE | SEC_RESERVE | SEC_COMMIT | SEC_FILE)) + if (flags & SEC_LARGE_PAGES) + { + if (!(flags & SEC_COMMIT)) + goto error; + } + switch (flags & (SEC_IMAGE | SEC_RESERVE | SEC_COMMIT | SEC_FILE )) { case SEC_IMAGE: if (flags & (SEC_WRITECOMBINE | SEC_LARGE_PAGES)) break; @@ -993,6 +1102,7 @@ static unsigned int get_mapping_flags( obj_handle_t handle, unsigned int flags ) if (handle) return SEC_FILE | (flags & (SEC_NOCACHE | SEC_WRITECOMBINE)); return flags; } +error: set_error( STATUS_INVALID_PARAMETER ); return 0; } @@ -1082,7 +1192,7 @@ static struct mapping *create_mapping( struct object *root, const struct unicode } if ((flags & SEC_RESERVE) && !(mapping->committed = create_ranges())) goto error; mapping->size = (mapping->size + page_mask) & ~((mem_size_t)page_mask); - if ((unix_fd = create_temp_file( mapping->size )) == -1) goto error; + if ((unix_fd = create_temp_file( file_access, flags, mapping->size )) == -1) goto error; if (!(mapping->fd = create_anonymous_fd( &mapping_fd_ops, unix_fd, &mapping->obj, FILE_SYNCHRONOUS_IO_NONALERT ))) goto error; allow_fd_caching( mapping->fd );
From: Vibhav Pant vibhavp@gmail.com
--- dlls/ntdll/unix/server.c | 2 +- dlls/ntdll/unix/sync.c | 15 ++- dlls/ntdll/unix/unix_private.h | 2 +- dlls/ntdll/unix/virtual.c | 166 ++++++++++++++++++++++++--------- 4 files changed, 139 insertions(+), 46 deletions(-)
diff --git a/dlls/ntdll/unix/server.c b/dlls/ntdll/unix/server.c index f3ffd99c3fc..00aeea4fb0f 100644 --- a/dlls/ntdll/unix/server.c +++ b/dlls/ntdll/unix/server.c @@ -1019,7 +1019,7 @@ static BOOL add_fd_to_cache( HANDLE handle, int fd, enum server_fd_type type, else { void *ptr = anon_mmap_alloc( FD_CACHE_BLOCK_SIZE * sizeof(union fd_cache_entry), - PROT_READ | PROT_WRITE ); + PROT_READ | PROT_WRITE, FALSE ); if (ptr == MAP_FAILED) return FALSE; fd_cache[entry] = ptr; } diff --git a/dlls/ntdll/unix/sync.c b/dlls/ntdll/unix/sync.c index 4b2d7c1ccbc..737cc8ed865 100644 --- a/dlls/ntdll/unix/sync.c +++ b/dlls/ntdll/unix/sync.c @@ -2058,6 +2058,19 @@ NTSTATUS WINAPI NtCreateSection( HANDLE *handle, ACCESS_MASK access, const OBJEC
*handle = 0;
+ if (sec_flags & SEC_LARGE_PAGES) + { + SIZE_T min_size = user_shared_data->LargePageMinimum; + if (file != NULL || size == NULL) + { + return STATUS_INVALID_PARAMETER; + } + + if (min_size == 0 || size->QuadPart % min_size != 0) + { + return STATUS_INVALID_PARAMETER; + } + } switch (protect & 0xff) { case PAGE_READONLY: @@ -2442,7 +2455,7 @@ static union tid_alert_entry *get_tid_alert_entry( HANDLE tid ) if (!tid_alert_blocks[block_idx]) { static const size_t size = TID_ALERT_BLOCK_SIZE * sizeof(union tid_alert_entry); - void *ptr = anon_mmap_alloc( size, PROT_READ | PROT_WRITE ); + void *ptr = anon_mmap_alloc( size, PROT_READ | PROT_WRITE, FALSE ); if (ptr == MAP_FAILED) return NULL; if (InterlockedCompareExchangePointer( (void **)&tid_alert_blocks[block_idx], ptr, NULL )) munmap( ptr, size ); /* someone beat us to it */ diff --git a/dlls/ntdll/unix/unix_private.h b/dlls/ntdll/unix/unix_private.h index fcb3de83a29..2ce10b62a1c 100644 --- a/dlls/ntdll/unix/unix_private.h +++ b/dlls/ntdll/unix/unix_private.h @@ -252,7 +252,7 @@ extern unsigned int alloc_object_attributes( const OBJECT_ATTRIBUTES *attr, stru extern NTSTATUS system_time_precise( void *args );
extern void *anon_mmap_fixed( void *start, size_t size, int prot, int flags ); -extern void *anon_mmap_alloc( size_t size, int prot ); +extern void *anon_mmap_alloc( size_t size, int prot, BOOL large_pages ); extern void virtual_init(void); extern ULONG_PTR get_system_affinity_mask(void); extern void virtual_get_system_info( SYSTEM_BASIC_INFORMATION *info, BOOL wow64 ); diff --git a/dlls/ntdll/unix/virtual.c b/dlls/ntdll/unix/virtual.c index 3981905bcd3..afa98187be4 100644 --- a/dlls/ntdll/unix/virtual.c +++ b/dlls/ntdll/unix/virtual.c @@ -195,6 +195,12 @@ static struct list teb_list = LIST_INIT( teb_list ); #define MAP_NORESERVE 0 #endif
+#if defined( MAP_HUGETLB ) && defined( MAP_LOCKED ) +#define MMAP_LARGE_PAGES_FLAG (MAP_HUGETLB | MAP_LOCKED) +#else +#define MMAP_LARGE_PAGES_FLAG 0 +#endif + #ifdef _WIN64 /* on 64-bit the page protection bytes use a 2-level table */ static const size_t pages_vprot_shift = 20; static const size_t pages_vprot_mask = (1 << 20) - 1; @@ -232,9 +238,10 @@ void *anon_mmap_fixed( void *start, size_t size, int prot, int flags ) }
/* allocate anonymous mmap() memory at any address */ -void *anon_mmap_alloc( size_t size, int prot ) +void *anon_mmap_alloc( size_t size, int prot, BOOL large_pages ) { - return mmap( NULL, size, prot, MAP_PRIVATE | MAP_ANON, -1, 0 ); + return mmap( NULL, size, prot, + MAP_PRIVATE | MAP_ANON | (large_pages ? MMAP_LARGE_PAGES_FLAG : 0), -1, 0 ); }
@@ -1004,7 +1011,7 @@ static BOOL alloc_pages_vprot( const void *addr, size_t size ) for (i = idx >> pages_vprot_shift; i < (end + pages_vprot_mask) >> pages_vprot_shift; i++) { if (pages_vprot[i]) continue; - if ((ptr = anon_mmap_alloc( pages_vprot_mask + 1, PROT_READ | PROT_WRITE )) == MAP_FAILED) + if ((ptr = anon_mmap_alloc( pages_vprot_mask + 1, PROT_READ | PROT_WRITE, FALSE )) == MAP_FAILED) { ERR( "anon mmap error %s for vprot table, size %08lx\n", strerror(errno), pages_vprot_mask + 1 ); return FALSE; @@ -1282,13 +1289,14 @@ static struct wine_rb_entry *find_view_inside_range( void **base_ptr, void **end * retrying inside it, and return where it actually succeeded, or NULL. */ static void* try_map_free_area( void *base, void *end, ptrdiff_t step, - void *start, size_t size, int unix_prot ) + void *start, size_t size, int unix_prot, BOOL large_pages ) { void *ptr; + int flags = large_pages ? MMAP_LARGE_PAGES_FLAG : 0;
while (start && base <= start && (char*)start + size <= (char*)end) { - if ((ptr = anon_mmap_tryfixed( start, size, unix_prot, 0 )) != MAP_FAILED) return start; + if ((ptr = anon_mmap_tryfixed( start, size, unix_prot, flags )) != MAP_FAILED) return start; TRACE( "Found free area is already mapped, start %p.\n", start ); if (errno != EEXIST) { @@ -1313,7 +1321,7 @@ static void* try_map_free_area( void *base, void *end, ptrdiff_t step, * Find a free area between views inside the specified range and map it. * virtual_mutex must be held by caller. */ -static void *map_free_area( void *base, void *end, size_t size, int top_down, int unix_prot, size_t align_mask ) +static void *map_free_area( void *base, void *end, size_t size, int top_down, int unix_prot, size_t align_mask, BOOL large_pages ) { struct wine_rb_entry *first = find_view_inside_range( &base, &end, top_down ); ptrdiff_t step = top_down ? -(align_mask + 1) : (align_mask + 1); @@ -1328,7 +1336,7 @@ static void *map_free_area( void *base, void *end, size_t size, int top_down, in { struct file_view *view = WINE_RB_ENTRY_VALUE( first, struct file_view, entry ); if ((start = try_map_free_area( (char *)view->base + view->size, (char *)start + size, step, - start, size, unix_prot ))) break; + start, size, unix_prot, large_pages ))) break; start = ROUND_ADDR( (char *)view->base - size, align_mask ); /* stop if remaining space is not large enough */ if (!start || start >= end || start < base) return NULL; @@ -1344,7 +1352,7 @@ static void *map_free_area( void *base, void *end, size_t size, int top_down, in { struct file_view *view = WINE_RB_ENTRY_VALUE( first, struct file_view, entry ); if ((start = try_map_free_area( start, view->base, step, - start, size, unix_prot ))) break; + start, size, unix_prot, large_pages ))) break; start = ROUND_ADDR( (char *)view->base + view->size + align_mask, align_mask ); /* stop if remaining space is not large enough */ if (!start || start >= end || (char *)end - (char *)start < size) return NULL; @@ -1353,7 +1361,7 @@ static void *map_free_area( void *base, void *end, size_t size, int top_down, in }
if (!first) - start = try_map_free_area( base, end, step, start, size, unix_prot ); + start = try_map_free_area( base, end, step, start, size, unix_prot, large_pages );
if (!start) ERR( "couldn't map free area in range %p-%p, size %p\n", base, end, (void *)size ); @@ -1494,7 +1502,7 @@ static struct file_view *alloc_view(void) } if (view_block_start == view_block_end) { - void *ptr = anon_mmap_alloc( view_block_size, PROT_READ | PROT_WRITE ); + void *ptr = anon_mmap_alloc( view_block_size, PROT_READ | PROT_WRITE, FALSE ); if (ptr == MAP_FAILED) return NULL; view_block_start = ptr; view_block_end = view_block_start + view_block_size / sizeof(*view_block_start); @@ -1871,10 +1879,11 @@ static void *find_reserved_free_area_outside_preloader( void *start, void *end, * virtual_mutex must be held by caller. */ static void *map_reserved_area( void *limit_low, void *limit_high, size_t size, int top_down, - int unix_prot, size_t align_mask ) + int unix_prot, size_t align_mask, BOOL large_pages ) { void *ptr = NULL; struct reserved_area *area = LIST_ENTRY( ptr, struct reserved_area, entry ); + int flags = large_pages ? MMAP_LARGE_PAGES_FLAG : 0;
if (top_down) { @@ -1906,7 +1915,7 @@ static void *map_reserved_area( void *limit_low, void *limit_high, size_t size, if (ptr) break; } } - if (ptr && anon_mmap_fixed( ptr, size, unix_prot, 0 ) != ptr) ptr = NULL; + if (ptr && anon_mmap_fixed( ptr, size, unix_prot, flags ) != ptr) ptr = NULL; return ptr; }
@@ -1916,12 +1925,13 @@ static void *map_reserved_area( void *limit_low, void *limit_high, size_t size, * Map a memory area at a fixed address. * virtual_mutex must be held by caller. */ -static NTSTATUS map_fixed_area( void *base, size_t size, unsigned int vprot ) +static NTSTATUS map_fixed_area( void *base, size_t size, unsigned int vprot, BOOL large_pages ) { int unix_prot = get_unix_prot(vprot); struct reserved_area *area; NTSTATUS status; char *start = base, *end = (char *)base + size; + int flags = large_pages ? MMAP_LARGE_PAGES_FLAG : 0;
if (find_view_range( base, size )) return STATUS_CONFLICTING_ADDRESSES;
@@ -1934,19 +1944,19 @@ static NTSTATUS map_fixed_area( void *base, size_t size, unsigned int vprot ) if (area_end <= start) continue; if (area_start > start) { - if (anon_mmap_tryfixed( start, area_start - start, unix_prot, 0 ) == MAP_FAILED) goto failed; + if (anon_mmap_tryfixed( start, area_start - start, unix_prot, flags ) == MAP_FAILED) goto failed; start = area_start; } if (area_end >= end) { - if (anon_mmap_fixed( start, end - start, unix_prot, 0 ) == MAP_FAILED) goto failed; + if (anon_mmap_fixed( start, end - start, unix_prot, flags ) == MAP_FAILED) goto failed; return STATUS_SUCCESS; } - if (anon_mmap_fixed( start, area_end - start, unix_prot, 0 ) == MAP_FAILED) goto failed; + if (anon_mmap_fixed( start, area_end - start, unix_prot, flags ) == MAP_FAILED) goto failed; start = area_end; }
- if (anon_mmap_tryfixed( start, end - start, unix_prot, 0 ) == MAP_FAILED) goto failed; + if (anon_mmap_tryfixed( start, end - start, unix_prot, flags ) == MAP_FAILED) goto failed; return STATUS_SUCCESS;
failed: @@ -1979,7 +1989,9 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size, int top_down = alloc_type & MEM_TOP_DOWN; void *ptr; NTSTATUS status; + BOOL large_pages = (alloc_type & MEM_LARGE_PAGES) != 0;
+ if (large_pages) vprot |= SEC_LARGE_PAGES; if (alloc_type & MEM_REPLACE_PLACEHOLDER) { struct file_view *view; @@ -1997,6 +2009,14 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size, return STATUS_SUCCESS; }
+ if (large_pages) + { + if (user_shared_data->LargePageMinimum == 0) + return STATUS_INVALID_PARAMETER; + if (size == 0 || size % user_shared_data->LargePageMinimum != 0) + return STATUS_INVALID_PARAMETER; + } + if (limit_high && limit_low >= limit_high) return STATUS_INVALID_PARAMETER;
if (base) @@ -2005,7 +2025,8 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size, if (limit_low && base < (void *)limit_low) return STATUS_CONFLICTING_ADDRESSES; if (limit_high && is_beyond_limit( base, size, (void *)limit_high )) return STATUS_CONFLICTING_ADDRESSES; if (is_beyond_limit( base, size, host_addr_space_limit )) return STATUS_CONFLICTING_ADDRESSES; - if ((status = map_fixed_area( base, size, vprot ))) return status; + if (large_pages && ((UINT_PTR)base % user_shared_data->LargePageMinimum) != 0) return STATUS_INVALID_PARAMETER; + if ((status = map_fixed_area( base, size, vprot, large_pages ))) return status; if (is_beyond_limit( base, size, working_set_limit )) working_set_limit = address_space_limit; ptr = base; } @@ -2021,7 +2042,7 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size, if (limit_low && (void *)limit_low > start) start = (void *)limit_low; if (limit_high && (void *)limit_high < end) end = (char *)limit_high + 1;
- if ((ptr = map_reserved_area( start, end, size, top_down, get_unix_prot(vprot), align_mask ))) + if ((ptr = map_reserved_area( start, end, size, top_down, get_unix_prot(vprot), align_mask, large_pages ))) { TRACE( "got mem in reserved area %p-%p\n", ptr, (char *)ptr + size ); goto done; @@ -2029,7 +2050,7 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size,
if (start > address_space_start || end < host_addr_space_limit || top_down) { - if (!(ptr = map_free_area( start, end, size, top_down, get_unix_prot(vprot), align_mask ))) + if (!(ptr = map_free_area( start, end, size, top_down, get_unix_prot(vprot), align_mask, large_pages ))) return STATUS_NO_MEMORY; TRACE( "got mem with map_free_area %p-%p\n", ptr, (char *)ptr + size ); goto done; @@ -2037,7 +2058,7 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size,
for (;;) { - if ((ptr = anon_mmap_alloc( view_size, get_unix_prot(vprot) )) == MAP_FAILED) + if ((ptr = anon_mmap_alloc( view_size, get_unix_prot(vprot), large_pages )) == MAP_FAILED) { status = (errno == ENOMEM) ? STATUS_NO_MEMORY : STATUS_INVALID_PARAMETER; ERR( "anon mmap error %s, size %p, unix_prot %#x\n", @@ -2055,6 +2076,18 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size, done: status = create_view( view_ret, ptr, size, vprot ); if (status != STATUS_SUCCESS) unmap_area( ptr, size ); + if (large_pages && mlock(ptr, size) != 0) + { + unmap_area( ptr, size ); + switch (errno) + { + case EPERM: + status = STATUS_ACCESS_DENIED; + break; + default: + status = STATUS_NO_MEMORY; + } + } return status; }
@@ -2082,10 +2115,17 @@ static NTSTATUS map_file_into_view( struct file_view *view, int fd, size_t start prot |= PROT_EXEC; }
+#ifdef MAP_LOCKED + if (vprot & SEC_LARGE_PAGES) + { + flags |= MAP_LOCKED; + } +#endif /* only try mmap if media is not removable (or if we require write access) */ if (!removable || (flags & MAP_SHARED)) { - if (mmap( (char *)view->base + start, size, prot, flags, fd, offset ) != MAP_FAILED) + ptr = mmap( (char *)view->base + start, size, prot, flags, fd, offset ); + if (ptr != MAP_FAILED) goto done;
switch (errno) @@ -2118,8 +2158,14 @@ static NTSTATUS map_file_into_view( struct file_view *view, int fd, size_t start }
/* Reserve the memory with an anonymous mmap */ - ptr = anon_mmap_fixed( (char *)view->base + start, size, PROT_READ | PROT_WRITE, 0 ); - if (ptr == MAP_FAILED) + ptr = anon_mmap_fixed( (char *)view->base + start, size, PROT_READ | PROT_WRITE, +#ifdef MAP_LOCKED + vprot & SEC_LARGE_PAGES ? MAP_LOCKED : 0 +#else + 0 +#endif + ); + if ( ptr == MAP_FAILED ) { ERR( "anon mmap error %s, range %p-%p\n", strerror(errno), (char *)view->base + start, (char *)view->base + start + size ); @@ -2130,6 +2176,16 @@ static NTSTATUS map_file_into_view( struct file_view *view, int fd, size_t start if (prot != (PROT_READ|PROT_WRITE)) mprotect( ptr, size, prot ); /* Set the right protection */ done: set_page_vprot( (char *)view->base + start, size, vprot ); + if ( vprot & SEC_LARGE_PAGES && mlock( ptr, size ) != 0 ) + { + switch (errno) + { + case EPERM: + return STATUS_ACCESS_DENIED; + default: + return STATUS_NO_MEMORY; + } + } return STATUS_SUCCESS; }
@@ -3179,6 +3235,16 @@ static unsigned int virtual_map_section( HANDLE handle, PVOID *addr_ptr, ULONG_P base = *addr_ptr; offset.QuadPart = offset_ptr ? offset_ptr->QuadPart : 0; if (offset.QuadPart >= full_size) return STATUS_INVALID_PARAMETER; + + if (sec_flags & SEC_LARGE_PAGES) + { + if (!*size_ptr || user_shared_data->LargePageMinimum == 0 || + *size_ptr % user_shared_data->LargePageMinimum != 0) + return STATUS_INVALID_PARAMETER; + if (base && ((UINT_PTR)base % user_shared_data->LargePageMinimum != 0)) + return STATUS_INVALID_PARAMETER; + } + if (*size_ptr) { size = *size_ptr; @@ -3204,7 +3270,8 @@ static unsigned int virtual_map_section( HANDLE handle, PVOID *addr_ptr, ULONG_P
server_enter_uninterrupted_section( &virtual_mutex, &sigset );
- res = map_view( &view, base, size, alloc_type, vprot, limit_low, limit_high, 0 ); + res = map_view( &view, base, size, alloc_type, vprot, limit_low, limit_high, + sec_flags & SEC_LARGE_PAGES ? ( user_shared_data->LargePageMinimum - 1 ) : 0 ); if (res) goto done;
TRACE( "handle=%p size=%lx offset=%s\n", handle, size, wine_dbgstr_longlong(offset.QuadPart) ); @@ -3270,7 +3337,7 @@ static void *alloc_virtual_heap( SIZE_T size ) mmap_remove_reserved_area( ret, size ); return ret; } - return anon_mmap_alloc( size, PROT_READ | PROT_WRITE ); + return anon_mmap_alloc( size, PROT_READ | PROT_WRITE, FALSE ); }
/*********************************************************************** @@ -4470,33 +4537,45 @@ static NTSTATUS allocate_virtual_memory( void **ret, SIZE_T *size_ptr, ULONG typ sigset_t sigset; SIZE_T size = *size_ptr; NTSTATUS status = STATUS_SUCCESS; + BOOL large_pages = (type & MEM_LARGE_PAGES) != 0;
/* Round parameters to a page boundary */
if (is_beyond_limit( 0, size, working_set_limit )) return STATUS_WORKING_SET_LIMIT_RANGE; - + if (large_pages && (user_shared_data->LargePageMinimum == 0 || size == 0 || + size % user_shared_data->LargePageMinimum != 0 || + (type & (MEM_RESERVE | MEM_COMMIT)) != (MEM_RESERVE | MEM_COMMIT))) + return STATUS_INVALID_PARAMETER; if (*ret) { - if (type & MEM_RESERVE && !(type & MEM_REPLACE_PLACEHOLDER)) /* Round down to 64k boundary */ - base = ROUND_ADDR( *ret, granularity_mask ); + if (large_pages) + { + if ((UINT_PTR)*ret % user_shared_data->LargePageMinimum != 0) return STATUS_INVALID_PARAMETER; + base = *ret; + } else - base = ROUND_ADDR( *ret, page_mask ); - size = (((UINT_PTR)*ret + size + page_mask) & ~page_mask) - (UINT_PTR)base; - - /* disallow low 64k, wrap-around and kernel space */ - if (((char *)base < (char *)0x10000) || - ((char *)base + size < (char *)base) || - is_beyond_limit( base, size, address_space_limit )) { - /* address 1 is magic to mean DOS area */ - if (!base && *ret == (void *)1 && size == 0x110000) is_dos_memory = TRUE; - else return STATUS_INVALID_PARAMETER; + if (type & MEM_RESERVE && + !(type & MEM_REPLACE_PLACEHOLDER )) /* Round down to 64k boundary */ + base = ROUND_ADDR( *ret, granularity_mask ); + else + base = ROUND_ADDR( *ret, page_mask ); + size = (((UINT_PTR)*ret + size + page_mask) & ~page_mask) - (UINT_PTR)base; + + /* disallow low 64k, wrap-around and kernel space */ + if (((char *)base < (char *)0x10000 ) || ((char *)base + size < (char *)base) || + is_beyond_limit( base, size, address_space_limit )) + { + /* address 1 is magic to mean DOS area */ + if (!base && *ret == (void *)1 && size == 0x110000) is_dos_memory = TRUE; + else return STATUS_INVALID_PARAMETER; + } } } else { base = NULL; - size = (size + page_mask) & ~page_mask; + if (!large_pages) size = (size + page_mask) & ~page_mask; }
/* Compute the alloc type flags */ @@ -4527,7 +4606,7 @@ static NTSTATUS allocate_virtual_memory( void **ret, SIZE_T *size_ptr, ULONG typ if (vprot & VPROT_WRITECOPY) status = STATUS_INVALID_PAGE_PROTECTION; else if (is_dos_memory) status = allocate_dos_memory( &view, vprot ); else status = map_view( &view, base, size, type, vprot, limit_low, limit_high, - align ? align - 1 : granularity_mask ); + align ? align - 1 : (large_pages ? user_shared_data->LargePageMinimum - 1 : granularity_mask) );
if (status == STATUS_SUCCESS) base = view->base; } @@ -4584,7 +4663,7 @@ static NTSTATUS allocate_virtual_memory( void **ret, SIZE_T *size_ptr, ULONG typ NTSTATUS WINAPI NtAllocateVirtualMemory( HANDLE process, PVOID *ret, ULONG_PTR zero_bits, SIZE_T *size_ptr, ULONG type, ULONG protect ) { - static const ULONG type_mask = MEM_COMMIT | MEM_RESERVE | MEM_TOP_DOWN | MEM_WRITE_WATCH | MEM_RESET; + static const ULONG type_mask = MEM_COMMIT | MEM_RESERVE | MEM_TOP_DOWN | MEM_WRITE_WATCH | MEM_RESET | MEM_LARGE_PAGES; ULONG_PTR limit;
TRACE("%p %p %08lx %x %08x\n", process, *ret, *size_ptr, (int)type, (int)protect ); @@ -4724,7 +4803,8 @@ NTSTATUS WINAPI NtAllocateVirtualMemoryEx( HANDLE process, PVOID *ret, SIZE_T *s ULONG count ) { static const ULONG type_mask = MEM_COMMIT | MEM_RESERVE | MEM_TOP_DOWN | MEM_WRITE_WATCH - | MEM_RESET | MEM_RESERVE_PLACEHOLDER | MEM_REPLACE_PLACEHOLDER; + | MEM_RESET | MEM_RESERVE_PLACEHOLDER | MEM_REPLACE_PLACEHOLDER + | MEM_LARGE_PAGES; ULONG_PTR limit_low = 0; ULONG_PTR limit_high = 0; ULONG_PTR align = 0;
From: Vibhav Pant vibhavp@gmail.com
--- dlls/kernel32/tests/virtual.c | 169 ++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+)
diff --git a/dlls/kernel32/tests/virtual.c b/dlls/kernel32/tests/virtual.c index 5fa8a1b5266..3aa7d343272 100644 --- a/dlls/kernel32/tests/virtual.c +++ b/dlls/kernel32/tests/virtual.c @@ -38,6 +38,8 @@ static HINSTANCE hkernel32, hkernelbase, hntdll; static SYSTEM_INFO si; static BOOL is_wow64; +static SIZE_T(WINAPI *pGetLargePageMinimum)(void); +static NTSTATUS (WINAPI *pRtlAdjustPrivilege)(ULONG,BOOLEAN,BOOLEAN,PBOOLEAN); static UINT (WINAPI *pGetWriteWatch)(DWORD,LPVOID,SIZE_T,LPVOID*,ULONG_PTR*,ULONG*); static UINT (WINAPI *pResetWriteWatch)(LPVOID,SIZE_T); static NTSTATUS (WINAPI *pNtAreMappedFilesTheSame)(PVOID,PVOID); @@ -559,6 +561,92 @@ static void test_VirtualAlloc(void) ok(VirtualFree(addr1, 0, MEM_RELEASE), "VirtualFree failed\n"); }
+static void test_VirtualAlloc_large_pages(void) +{ + SIZE_T size; + NTSTATUS status; + HANDLE process_token; + HANDLE token; + BOOLEAN enabled; + + if (!pGetLargePageMinimum) + { + win_skip( "No GetLargePageMinimum support.\n" ); + return; + } + if (!pRtlAdjustPrivilege) + { + win_skip( "No RtlAdjustPrivilege support.\n" ); + return; + } + + size = pGetLargePageMinimum(); + if (size == 0) + { + trace( "No large pages support, skipping test.\n" ); + return; + } + ok( OpenProcessToken( GetCurrentProcess(), TOKEN_DUPLICATE | TOKEN_QUERY, &process_token ), + "OpenProcessToken failed (%ld)\n", GetLastError() ); + ok( DuplicateToken( process_token, SecurityImpersonation, &token ), + "DuplicateToken failed (%ld)\n", GetLastError() ); + ok( ImpersonateLoggedOnUser( token ), "ImpersonateLoggedOnUser failed (%ld)\n", + GetLastError() ); + status = pRtlAdjustPrivilege( SE_LOCK_MEMORY_PRIVILEGE, TRUE, TRUE, &enabled ); + if (status != STATUS_SUCCESS) + { + trace( "Couldn't get SE_LOCK_MEMORY_PRIVILEGE (%ld), skipping large page file " + "mapping test.\n", + status ); + } + else + { + void *addr; + + SetLastError(0xdeadbeef); + addr = VirtualAlloc( NULL, 0, MEM_LARGE_PAGES | MEM_RESERVE | MEM_COMMIT, + PAGE_READWRITE ); + ok( addr == NULL && GetLastError() == ERROR_INVALID_PARAMETER, + "VirtualAlloc should fail with %d (got %ld)\n", ERROR_INVALID_PARAMETER, + GetLastError() ); + + SetLastError(0xdeadbeef); + addr = VirtualAlloc( NULL, size - 1, MEM_LARGE_PAGES | MEM_RESERVE | MEM_COMMIT, + PAGE_READWRITE ); + ok( addr == NULL && GetLastError() == ERROR_INVALID_PARAMETER, + "VirtualAlloc should fail with %d (got %ld)\n", ERROR_INVALID_PARAMETER, + GetLastError() ); + + SetLastError(0xdeadbeef); + addr = VirtualAlloc (NULL, size, MEM_LARGE_PAGES, PAGE_READWRITE); + ok( addr == NULL && GetLastError() == ERROR_INVALID_PARAMETER, + "VirtualAlloc should fail with %d (got %ld)\n", ERROR_INVALID_PARAMETER, + GetLastError() ); + + SetLastError(0xdeadbeef); + addr = + VirtualAlloc( NULL, size, MEM_LARGE_PAGES | MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE ); + ok( addr != NULL, "VirtualAlloc failed (%ld)\n", GetLastError() ); + SetLastError(0xdeadbeef); + ok(VirtualFree(addr, 0, MEM_RELEASE), "VirtualFree failed (%ld)\n", GetLastError()); + + SetLastError(0xdeadbeef); + addr = + VirtualAlloc( (void *)((UINT_PTR)addr - 1), size, MEM_LARGE_PAGES | MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE ); + ok( addr == NULL && GetLastError() == ERROR_INVALID_PARAMETER, + "VirtualAlloc should fail with %d (got %ld)\n", ERROR_INVALID_PARAMETER, + GetLastError() ); + + SetLastError(0xdeadbeef); + addr = + VirtualAlloc( addr, size, MEM_LARGE_PAGES | MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE ); + ok( addr != NULL, "VirtualAlloc failed (%ld)\n", GetLastError() ); + SetLastError(0xdeadbeef); + ok(VirtualFree(addr, 0, MEM_RELEASE), "VirtualFree failed (%ld)\n", GetLastError()); + } + ok( RevertToSelf(), "RevertToSelf failed (%ld)\n", GetLastError() ); +} + static void test_MapViewOfFile(void) { static const char testfile[] = "testfile.xxx"; @@ -1328,6 +1416,83 @@ static void test_MapViewOfFile(void) DeleteFileA(testfile); }
+static void test_large_page_file_mapping( void ) +{ + SIZE_T size; + BOOLEAN enabled; + NTSTATUS status; + DWORD err; + HANDLE file; + HANDLE token; + HANDLE process_token; + + if (!pGetLargePageMinimum) + { + win_skip( "No GetLargePageMinimum support.\n" ); + return; + } + if (!pRtlAdjustPrivilege) + { + win_skip( "No RtlAdjustPrivilege support.\n" ); + return; + } + + size = pGetLargePageMinimum(); + if (size == 0) + { + trace( "No large pages support, skipping test.\n" ); + return; + } + ok( OpenProcessToken( GetCurrentProcess(), TOKEN_DUPLICATE | TOKEN_QUERY, &process_token ), + "OpenProcessToken failed (%ld)\n", GetLastError() ); + ok( DuplicateToken( process_token, SecurityImpersonation, &token ), + "DuplicateToken failed (%ld)\n", GetLastError() ); + ok( ImpersonateLoggedOnUser( token ), "ImpersonateLoggedOnUser failed (%ld)\n", + GetLastError() ); + status = pRtlAdjustPrivilege( SE_LOCK_MEMORY_PRIVILEGE, TRUE, TRUE, &enabled ); + if (status != STATUS_SUCCESS) + { + trace( "Couldn't get SE_LOCK_MEMORY_PRIVILEGE (%ld), skipping large page file " + "mapping test.\n", + status ); + } + else + { + file = + CreateFileMappingW( INVALID_HANDLE_VALUE, NULL, + PAGE_READWRITE | SEC_LARGE_PAGES | SEC_COMMIT, 0, size - 1, NULL ); + err = GetLastError(); + ok( file == NULL && err == ERROR_INVALID_PARAMETER, + "CreateFileMappingW should fail with ERROR_INVALID_PARAMETER (got %ld instead)\n", + err ); + if (file != NULL) CloseHandle( file ); + + file = + CreateFileMappingW( INVALID_HANDLE_VALUE, NULL, + PAGE_READWRITE | SEC_LARGE_PAGES | SEC_COMMIT, 0, size * 2, NULL ); + ok( file != NULL, "CreateFileMappingW failed (%ld)\n", GetLastError() ); + if (file != NULL) + { + void *addr; + addr = MapViewOfFile( file, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, size * 2 ); + ok( addr != NULL, "MapViewOfFile failed (%ld)\n", GetLastError() ); + UnmapViewOfFile( addr ); + CloseHandle( file ); + } + + file = CreateFileMappingW( INVALID_HANDLE_VALUE, NULL, + PAGE_READWRITE | SEC_LARGE_PAGES | SEC_RESERVE, 0, size, NULL ); + err = GetLastError(); + ok( file == NULL && err == ERROR_INVALID_PARAMETER, + "CreateFileMappingW should have failed with ERROR_INVALID_PARAMETER (got %ld " + "instead)\n", + err ); + if (file != NULL) CloseHandle( file ); + } + + ok( RevertToSelf(), "RevertToSelf failed (%ld)\n", GetLastError() ); +} +
static void test_NtAreMappedFilesTheSame(void) { @@ -4408,6 +4573,8 @@ START_TEST(virtual) hkernelbase = GetModuleHandleA("kernelbase.dll"); hntdll = GetModuleHandleA("ntdll.dll");
+ pGetLargePageMinimum = (void *)GetProcAddress(hkernel32, "GetLargePageMinimum"); + pRtlAdjustPrivilege = (void *)GetProcAddress(hntdll, "RtlAdjustPrivilege"); pGetWriteWatch = (void *) GetProcAddress(hkernel32, "GetWriteWatch"); pResetWriteWatch = (void *) GetProcAddress(hkernel32, "ResetWriteWatch"); pGetProcessDEPPolicy = (void *)GetProcAddress( hkernel32, "GetProcessDEPPolicy" ); @@ -4435,10 +4602,12 @@ START_TEST(virtual) test_shared_memory_ro(FALSE, FILE_MAP_COPY|FILE_MAP_WRITE); test_mappings(); test_CreateFileMapping_protection(); + test_large_page_file_mapping(); test_VirtualAlloc_protection(); test_VirtualProtect(); test_VirtualAllocEx(); test_VirtualAlloc(); + test_VirtualAlloc_large_pages(); test_MapViewOfFile(); test_NtAreMappedFilesTheSame(); test_CreateFileMapping();
From: Vibhav Pant vibhavp@gmail.com
--- configure | 10 ++++++ configure.ac | 1 + dlls/ntdll/unix/virtual.c | 65 ++++++++++++++++++++++++++++++++++++--- include/config.h.in | 3 ++ 4 files changed, 74 insertions(+), 5 deletions(-)
diff --git a/configure b/configure index 15cf8f98459..844ee5409e0 100755 --- a/configure +++ b/configure @@ -21065,6 +21065,16 @@ then : printf "%s\n" "#define HAVE_REQUEST_SENSE 1" >>confdefs.h
+fi + +ac_fn_c_check_type "$LINENO" "struct pm_scan_arg" "ac_cv_type_struct_pm_scan_arg" "#include <linux/fs.h> +" +if test "x$ac_cv_type_struct_pm_scan_arg" = xyes +then : + +printf "%s\n" "#define HAVE_STRUCT_PM_SCAN_ARG 1" >>confdefs.h + + fi
diff --git a/configure.ac b/configure.ac index 8e68a09ad0a..8a03a10aa55 100644 --- a/configure.ac +++ b/configure.ac @@ -2121,6 +2121,7 @@ dnl **** Check for types ****
AC_C_INLINE AC_CHECK_TYPES([request_sense],,,[#include <linux/cdrom.h>]) +AC_CHECK_TYPES([struct pm_scan_arg],,,[#include <linux/fs.h>])
AC_CHECK_TYPES([struct xinpgen],,, [#include <sys/types.h> diff --git a/dlls/ntdll/unix/virtual.c b/dlls/ntdll/unix/virtual.c index afa98187be4..f740f852b34 100644 --- a/dlls/ntdll/unix/virtual.c +++ b/dlls/ntdll/unix/virtual.c @@ -63,6 +63,10 @@ # include <mach/mach_init.h> # include <mach/mach_vm.h> #endif +#ifdef HAVE_STRUCT_PM_SCAN_ARG +#include <linux/fs.h> +#include <sys/ioctl.h> +#endif
#include "ntstatus.h" #define WIN32_NO_STATUS @@ -5215,6 +5219,9 @@ static NTSTATUS get_working_set_ex( HANDLE process, LPCVOID addr, { #if !defined(HAVE_LIBPROCSTAT) static int pagemap_fd = -2; +#ifdef HAVE_STRUCT_PM_SCAN_ARG + BOOL pagemap_ioctl_avail = TRUE; +#endif #endif MEMORY_WORKING_SET_EX_INFORMATION *p; sigset_t sigset; @@ -5302,15 +5309,63 @@ static NTSTATUS get_working_set_ex( HANDLE process, LPCVOID addr, get_committed_size( view, p->VirtualAddress, &vprot, VPROT_COMMITTED ) && (vprot & VPROT_COMMITTED)) { - if (pagemap_fd == -1 || - pread( pagemap_fd, &pagemap, sizeof(pagemap), ((UINT_PTR)p->VirtualAddress >> page_shift) * sizeof(pagemap) ) != sizeof(pagemap)) + if (pagemap_fd != -1) + { +#ifdef HAVE_STRUCT_PM_SCAN_ARG + if (pagemap_ioctl_avail) + { + UINT_PTR addr = (UINT_PTR)(ROUND_ADDR(view->base, page_mask)); + struct page_region output_buf; + int count; + struct pm_scan_arg scan_arg = { + .size = sizeof( scan_arg ), + .flags = 0, + .start = addr, + .end = addr + page_size, + .vec = (UINT_PTR)&output_buf, + .vec_len = 1, + .max_pages = 1, + .category_inverted = 0, + .category_mask = PAGE_IS_PRESENT, + .category_anyof_mask = PAGE_IS_HUGE | PAGE_IS_PRESENT | PAGE_IS_FILE, + .return_mask = PAGE_IS_HUGE | PAGE_IS_PRESENT | PAGE_IS_FILE, + }; + + memset(&output_buf, 0, sizeof(output_buf)); + count = ioctl(pagemap_fd, PAGEMAP_SCAN, &scan_arg); + if (count == -1) + { + WARN("ioctl(PAGEMAP_SCAN, %p) failed: '%s'\n", (void *)addr, strerror(errno)); + pagemap_ioctl_avail = FALSE; + } + else if (count > 0) + { + assert( output_buf.start <= addr && output_buf.end > addr ); + p->VirtualAttributes.Valid = !(vprot & VPROT_GUARD) && (vprot & 0x0f) && (output_buf.categories & PAGE_IS_PRESENT); + p->VirtualAttributes.Shared = !is_view_valloc( view ) && (output_buf.categories & PAGE_IS_FILE); + p->VirtualAttributes.LargePage = p->VirtualAttributes.Valid && (output_buf.categories & PAGE_IS_HUGE) && (view->protect & SEC_LARGE_PAGES); + if (p->VirtualAttributes.LargePage) + p->VirtualAttributes.Locked = TRUE; + } + } + if (!pagemap_ioctl_avail) +#endif + { + if (pread( pagemap_fd, &pagemap, sizeof(pagemap), ((UINT_PTR)p->VirtualAddress >> page_shift) * sizeof(pagemap) ) != sizeof(pagemap)) + { + /* If we don't have pagemap information, default to invalid. */ + pagemap = 0; + } + + p->VirtualAttributes.Valid = !(vprot & VPROT_GUARD) && (vprot & 0x0f) && (pagemap >> 63); + p->VirtualAttributes.Shared = !is_view_valloc( view ) && ((pagemap >> 61) & 1); + } + } + else { - /* If we don't have pagemap information, default to invalid. */ pagemap = 0; }
- p->VirtualAttributes.Valid = !(vprot & VPROT_GUARD) && (vprot & 0x0f) && (pagemap >> 63); - p->VirtualAttributes.Shared = !is_view_valloc( view ) && ((pagemap >> 61) & 1); if (p->VirtualAttributes.Shared && p->VirtualAttributes.Valid) p->VirtualAttributes.ShareCount = 1; /* FIXME */ if (p->VirtualAttributes.Valid) diff --git a/include/config.h.in b/include/config.h.in index bbd5a57a280..54816765623 100644 --- a/include/config.h.in +++ b/include/config.h.in @@ -432,6 +432,9 @@ /* Define to 1 if `mt_gstat' is a member of `struct mtget'. */ #undef HAVE_STRUCT_MTGET_MT_GSTAT
+/* Define to 1 if the system has the type `struct pm_scan_arg'. */ +#undef HAVE_STRUCT_PM_SCAN_ARG + /* Define to 1 if `sin6_scope_id' is a member of `struct sockaddr_in6'. */ #undef HAVE_STRUCT_SOCKADDR_IN6_SIN6_SCOPE_ID
From: Vibhav Pant vibhavp@gmail.com
--- dlls/psapi/tests/Makefile.in | 2 +- dlls/psapi/tests/psapi_main.c | 74 +++++++++++++++++++++++++++++++++-- 2 files changed, 72 insertions(+), 4 deletions(-)
diff --git a/dlls/psapi/tests/Makefile.in b/dlls/psapi/tests/Makefile.in index c22fe61ef67..0ff885b524b 100644 --- a/dlls/psapi/tests/Makefile.in +++ b/dlls/psapi/tests/Makefile.in @@ -1,5 +1,5 @@ TESTDLL = psapi.dll -IMPORTS = psapi user32 +IMPORTS = psapi user32 kernelbase
SOURCES = \ psapi_main.c diff --git a/dlls/psapi/tests/psapi_main.c b/dlls/psapi/tests/psapi_main.c index e7e5e5f04e6..b974d44e5c8 100644 --- a/dlls/psapi/tests/psapi_main.c +++ b/dlls/psapi/tests/psapi_main.c @@ -41,6 +41,8 @@ static BOOL (WINAPI *pIsWow64Process)(HANDLE, BOOL *); static BOOL (WINAPI *pWow64DisableWow64FsRedirection)(void **); static BOOL (WINAPI *pWow64RevertWow64FsRedirection)(void *); static BOOL (WINAPI *pQueryWorkingSetEx)(HANDLE, PVOID, DWORD); +static SIZE_T(WINAPI *pGetLargePageMinimum)(void); +static NTSTATUS (WINAPI *pRtlAdjustPrivilege)(ULONG,BOOLEAN,BOOLEAN,PBOOLEAN);
static BOOL wow64; static char** main_argv; @@ -53,6 +55,8 @@ static BOOL init_func_ptrs(void) pWow64DisableWow64FsRedirection = (void *)GetProcAddress(GetModuleHandleA("kernel32.dll"), "Wow64DisableWow64FsRedirection"); pWow64RevertWow64FsRedirection = (void *)GetProcAddress(GetModuleHandleA("kernel32.dll"), "Wow64RevertWow64FsRedirection"); pQueryWorkingSetEx = (void *)GetProcAddress(GetModuleHandleA("psapi.dll"), "QueryWorkingSetEx"); + pGetLargePageMinimum = (void *)GetProcAddress(GetModuleHandleA("kernel32.dll"), "GetLargePageMinimum"); + pRtlAdjustPrivilege = (void *)GetProcAddress(GetModuleHandleA("ntdll.dll"), "RtlAdjustPrivilege"); return TRUE; }
@@ -1015,7 +1019,7 @@ static void test_GetModuleFileNameEx(void) return; ok(ret == strlen(szModExPath), "szModExPath="%s" ret=%ld\n", szModExPath, ret); GetModuleFileNameA(NULL, szModPath, sizeof(szModPath)); - ok(!strncmp(szModExPath, szModPath, MAX_PATH), + ok(!strncmp(szModExPath, szModPath, MAX_PATH), "szModExPath="%s" szModPath="%s"\n", szModExPath, szModPath);
SetLastError(0xdeadbeef); @@ -1119,7 +1123,7 @@ static void test_ws_functions(void) SetLastError(0xdeadbeef); ret = InitializeProcessForWsWatch(ws_handle); ok(ret == 1, "failed with %ld\n", GetLastError()); - + addr = VirtualAlloc(NULL, 1, MEM_COMMIT, PAGE_READWRITE); if(!addr) return; @@ -1145,7 +1149,7 @@ static void test_ws_functions(void)
todo_wine ok(0, "GetWsChanges didn't find our page\n"); } - + free_page: VirtualFree(addr, 0, MEM_RELEASE); } @@ -1237,6 +1241,69 @@ static void test_QueryWorkingSetEx(void) check_QueryWorkingSetEx(addr, "valloc,free", FALSE, 0, 0, FALSE); }
+static void test_large_pages( void ) +{ + SIZE_T size; + NTSTATUS status; + HANDLE process_token; + HANDLE token; + BOOLEAN enabled; + + if (!pGetLargePageMinimum) + { + win_skip( "No GetLargePageMinimum support.\n" ); + return; + } + if (!pRtlAdjustPrivilege) + { + win_skip( "No RtlAdjustPrivilege support.\n" ); + return; + } + + size = pGetLargePageMinimum(); + if (size == 0) + { + trace( "No large pages support, skipping test.\n" ); + return; + } + ok( OpenProcessToken( GetCurrentProcess(), TOKEN_DUPLICATE | TOKEN_QUERY, &process_token ), + "OpenProcessToken failed (%ld)\n", GetLastError() ); + ok( DuplicateToken( process_token, SecurityImpersonation, &token ), + "DuplicateToken failed (%ld)\n", GetLastError() ); + ok( ImpersonateLoggedOnUser( token ), "ImpersonateLoggedOnUser failed (%ld)\n", + GetLastError() ); + status = pRtlAdjustPrivilege( SE_LOCK_MEMORY_PRIVILEGE, TRUE, TRUE, &enabled ); + if (status != STATUS_SUCCESS) + { + trace( "Couldn't get SE_LOCK_MEMORY_PRIVILEGE (%ld), skipping large page file " + "mapping test.\n", + status ); + } + else + { + void *addr; + + SetLastError( 0xdeadbeef ); + addr = + VirtualAlloc( NULL, size, MEM_LARGE_PAGES | MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE ); + ok( addr != NULL, "VirtualAlloc failed (%ld)\n", GetLastError() ); + if (addr != NULL) + { + PSAPI_WORKING_SET_EX_INFORMATION info = { 0 }; + + info.VirtualAddress = addr; + SetLastError( 0xdeadbeef ); + ok( QueryWorkingSetEx( GetCurrentProcess(), &info, sizeof( info ) ), + "QueryWorkingSet failed (%ld)\n", GetLastError() ); + ok( info.VirtualAttributes.Valid, "Expected VirtualAttributes.Valid to be TRUE\n" ); + ok( info.VirtualAttributes.LargePage, "Expected VirtualAttributes.Large to be TRUE\n" ); + ok( info.VirtualAttributes.Locked, "Expected VirtualAttributes.Locked to be TRUE\n" ); + SetLastError( 0xdeadbeef ); + ok( VirtualFree( addr, 0, MEM_RELEASE ), "VirtualFree failed (%ld)\n", GetLastError() ); + } + } +} + START_TEST(psapi_main) { DWORD pid = GetCurrentProcessId(); @@ -1268,6 +1335,7 @@ START_TEST(psapi_main) test_GetModuleBaseName(); test_QueryWorkingSetEx(); test_ws_functions(); + test_large_pages();
CloseHandle(hpSR); CloseHandle(hpQI);
Hi,
It looks like your patch introduced the new failures shown below. Please investigate and fix them before resubmitting your patch. If they are not new, fixing them anyway would help a lot. Otherwise please ask for the known failures list to be updated.
The tests also ran into some preexisting test failures. If you know how to fix them that would be helpful. See the TestBot job for the details:
The full results can be found at: https://testbot.winehq.org/JobDetails.pl?Key=146557
Your paranoid android.
=== debian11 (32 bit report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11 (32 bit ar:MA report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11 (32 bit de report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11 (32 bit fr report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11 (32 bit he:IL report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11 (32 bit hi:IN report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11 (32 bit ja:JP report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11 (32 bit zh:CN report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11b (32 bit WoW report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (8) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (8) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (8)
=== debian11b (64 bit WoW report) ===
kernel32: virtual.c:1478: Test failed: MapViewOfFile failed (8) virtual.c:629: Test failed: VirtualAlloc failed (87) virtual.c:631: Test failed: VirtualFree failed (-559038737) virtual.c:643: Test failed: VirtualAlloc failed (87) virtual.c:645: Test failed: VirtualFree failed (-559038737)
psapi: psapi_main.c:1289: Test failed: VirtualAlloc failed (87)
-
Marvin -
Vibhav Pant -
Vibhav Pant (＠vibhavp)