February 2018 - Wine-Devel - list.winehq.org

[PATCH] wmc: Fix a memory leak (Pinpoint)
by Zhiyi Zhang Feb. 26, 2018

Feb. 26, 2018

Fix 44600 Although these build tools only run for a short period of time. These memory leaks should be fixed anyway. Signed-off-by: Zhiyi Zhang <yi.gd.cn(a)gmail.com> --- tools/wmc/po.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)

1 0

[PATCH] wrc: Fix memory leaks (Pinpoint)
by Zhiyi Zhang Feb. 26, 2018

Feb. 26, 2018

Fix 44610,44611 Although these build tools only run for a short period of time. These memory leaks should be fixed anyway. Signed-off-by: Zhiyi Zhang <yi.gd.cn(a)gmail.com> --- tools/wrc/po.c | 17 ++++++++++++----- tools/wrc/wrc.c | 17 ++++++++++------- 2 files changed, 22 insertions(+), 12 deletions(-)

1 0

[PATCH v3 00/10] winevulkan
by Roderick Colenbrander Feb. 26, 2018

Feb. 26, 2018

Hi all, This is a revised version of my previous patches. The main change is the inclusion of the make_vulkan script aside from some minor changes. Thanks, Roderick Roderick Colenbrander (10): vulkan: Add initial Wine vulkan header. winevulkan: Add stub ICD. winevulkan: Implement vk_icdNegotiateICDInterfaceVersion. winevulkan: Implement global Vulkan function stubs and loading. winevulkan: Define vulkan driver interface. winex11: Add Vulkan stubs. winex11: Implement vkEnumerateInstanceExtensionProperties. winevulkan/winex11: Implement vkCreateInstance. winevulkan: Add instance funcs stubs. winevulkan: Implement vkDestroyInstance. configure.ac | 10 + dlls/gdi32/Makefile.in | 3 +- dlls/gdi32/dibdrv/dc.c | 2 + dlls/gdi32/driver.c | 6 + dlls/gdi32/enhmfdrv/dc.c | 1 + dlls/gdi32/enhmfdrv/init.c | 1 + dlls/gdi32/freetype.c | 1 + dlls/gdi32/gdi32.spec | 3 + dlls/gdi32/mfdrv/init.c | 1 + dlls/gdi32/path.c | 1 + dlls/gdi32/vulkan.c | 41 + dlls/wineandroid.drv/init.c | 1 + dlls/winemac.drv/gdi.c | 1 + dlls/wineps.drv/init.c | 1 + dlls/winevulkan/Makefile.in | 8 + dlls/winevulkan/version.rc | 27 + dlls/winevulkan/vk.xml | 6514 ++++++++++++++++++++++++++++++++++++++ dlls/winevulkan/vulkan.c | 224 ++ dlls/winevulkan/vulkan.py | 1450 +++++++++ dlls/winevulkan/vulkan_private.h | 57 + dlls/winevulkan/vulkan_thunks.c | 102 + dlls/winevulkan/vulkan_thunks.h | 12 + dlls/winevulkan/winevulkan.spec | 2 + dlls/winex11.drv/Makefile.in | 1 + dlls/winex11.drv/init.c | 16 + dlls/winex11.drv/vulkan.c | 152 + dlls/winex11.drv/x11drv.h | 1 + dlls/winex11.drv/xrender.c | 1 + include/config.h.in | 3 + include/wine/gdi_driver.h | 5 +- include/wine/vulkan.h | 2423 ++++++++++++++ include/wine/vulkan_driver.h | 23 + 32 files changed, 11092 insertions(+), 2 deletions(-) create mode 100644 dlls/gdi32/vulkan.c create mode 100644 dlls/winevulkan/Makefile.in create mode 100644 dlls/winevulkan/version.rc create mode 100644 dlls/winevulkan/vk.xml create mode 100644 dlls/winevulkan/vulkan.c create mode 100755 dlls/winevulkan/vulkan.py create mode 100644 dlls/winevulkan/vulkan_private.h create mode 100644 dlls/winevulkan/vulkan_thunks.c create mode 100644 dlls/winevulkan/vulkan_thunks.h create mode 100644 dlls/winevulkan/winevulkan.spec create mode 100644 dlls/winex11.drv/vulkan.c create mode 100644 include/wine/vulkan.h create mode 100644 include/wine/vulkan_driver.h -- 2.14.3

1 11

[PATCH 4/5] wined3d: Explicitly calculate the sub-resource level in is_full_clear().
by Henri Verbeet Feb. 26, 2018

Feb. 26, 2018

Signed-off-by: Henri Verbeet <hverbeet(a)codeweavers.com> --- dlls/wined3d/device.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/dlls/wined3d/device.c b/dlls/wined3d/device.c index 6e275e1..18b39b8 100644 --- a/dlls/wined3d/device.c +++ b/dlls/wined3d/device.c @@ -200,10 +200,14 @@ void device_context_remove(struct wined3d_device *device, struct wined3d_context device->contexts = new_array; } -static BOOL is_full_clear(const struct wined3d_surface *target, const RECT *draw_rect, const RECT *clear_rect) +static BOOL is_full_clear(const struct wined3d_texture *texture, unsigned int sub_resource_idx, + const RECT *draw_rect, const RECT *clear_rect) { - unsigned int height = wined3d_texture_get_level_height(target->container, target->texture_level); - unsigned int width = wined3d_texture_get_level_width(target->container, target->texture_level); + unsigned int width, height, level; + + level = sub_resource_idx % texture->level_count; + width = wined3d_texture_get_level_width(texture, level); + height = wined3d_texture_get_level_height(texture, level); /* partial draw rect */ if (draw_rect->left || draw_rect->top || draw_rect->right < width || draw_rect->bottom < height) @@ -262,7 +266,8 @@ void device_clear_render_targets(struct wined3d_device *device, UINT rt_count, c { struct wined3d_texture *rt = wined3d_texture_from_resource(rtv->resource); - if (flags & WINED3DCLEAR_TARGET && !is_full_clear(target, draw_rect, rect_count ? clear_rect : NULL)) + if (flags & WINED3DCLEAR_TARGET && !is_full_clear(rt, rtv->sub_resource_idx, + draw_rect, rect_count ? clear_rect : NULL)) wined3d_texture_load_location(rt, rtv->sub_resource_idx, context, rtv->resource->draw_binding); else wined3d_texture_prepare_location(rt, rtv->sub_resource_idx, context, rtv->resource->draw_binding); @@ -289,7 +294,7 @@ void device_clear_render_targets(struct wined3d_device *device, UINT rt_count, c struct wined3d_texture *ds = wined3d_texture_from_resource(dsv->resource); if (flags & (WINED3DCLEAR_ZBUFFER | WINED3DCLEAR_STENCIL) - && !is_full_clear(depth_stencil, draw_rect, rect_count ? clear_rect : NULL)) + && !is_full_clear(ds, dsv->sub_resource_idx, draw_rect, rect_count ? clear_rect : NULL)) wined3d_texture_load_location(ds, dsv->sub_resource_idx, context, ds_location); else wined3d_texture_prepare_location(ds, dsv->sub_resource_idx, context, ds_location); -- 2.1.4

1 0

[PATCH 3/5] wined3d: Explicitly calculate the sub-resource level in arbfp_blitter_blit().
by Henri Verbeet Feb. 26, 2018

Feb. 26, 2018

Signed-off-by: Henri Verbeet <hverbeet(a)codeweavers.com> --- dlls/wined3d/arb_program_shader.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/dlls/wined3d/arb_program_shader.c b/dlls/wined3d/arb_program_shader.c index 9823de2..f86c167 100644 --- a/dlls/wined3d/arb_program_shader.c +++ b/dlls/wined3d/arb_program_shader.c @@ -7825,6 +7825,8 @@ static DWORD arbfp_blitter_blit(struct wined3d_blitter *blitter, enum wined3d_bl == WINED3D_LOCATION_DRAWABLE && !wined3d_resource_is_offscreen(&src_texture->resource)) { + unsigned int src_level = src_sub_resource_idx % src_texture->level_count; + /* Without FBO blits transferring from the drawable to the texture is * expensive, because we have to flip the data in sysmem. Since we can * flip in the blitter, we don't actually need that flip anyway. So we @@ -7833,8 +7835,8 @@ static DWORD arbfp_blitter_blit(struct wined3d_blitter *blitter, enum wined3d_bl surface_load_fb_texture(src_surface, FALSE, context); s = *src_rect; - s.top = wined3d_texture_get_level_height(src_texture, src_surface->texture_level) - s.top; - s.bottom = wined3d_texture_get_level_height(src_texture, src_surface->texture_level) - s.bottom; + s.top = wined3d_texture_get_level_height(src_texture, src_level) - s.top; + s.bottom = wined3d_texture_get_level_height(src_texture, src_level) - s.bottom; src_rect = &s; } else -- 2.1.4

1 0

[PATCH 2/5] wined3d: Explicitly calculate the sub-resource level in arbfp_blit_set().
by Henri Verbeet Feb. 26, 2018

Feb. 26, 2018

Signed-off-by: Henri Verbeet <hverbeet(a)codeweavers.com> --- dlls/wined3d/arb_program_shader.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/dlls/wined3d/arb_program_shader.c b/dlls/wined3d/arb_program_shader.c index 36e69a9..9823de2 100644 --- a/dlls/wined3d/arb_program_shader.c +++ b/dlls/wined3d/arb_program_shader.c @@ -7567,9 +7567,9 @@ static GLuint arbfp_gen_plain_shader(const struct wined3d_gl_info *gl_info, cons /* Context activation is done by the caller. */ static HRESULT arbfp_blit_set(struct wined3d_arbfp_blitter *blitter, struct wined3d_context *context, - const struct wined3d_surface *surface, const struct wined3d_color_key *color_key) + const struct wined3d_texture *texture, unsigned int sub_resource_idx, + const struct wined3d_color_key *color_key) { - const struct wined3d_texture *texture = surface->container; enum complex_fixup fixup; const struct wined3d_gl_info *gl_info = context->gl_info; struct wine_rb_entry *entry; @@ -7577,10 +7577,12 @@ static HRESULT arbfp_blit_set(struct wined3d_arbfp_blitter *blitter, struct wine struct arbfp_blit_desc *desc; struct wined3d_color float_color_key[2]; struct wined3d_vec4 size; + unsigned int level; GLuint shader; - size.x = wined3d_texture_get_level_pow2_width(texture, surface->texture_level); - size.y = wined3d_texture_get_level_pow2_height(texture, surface->texture_level); + level = sub_resource_idx % texture->level_count; + size.x = wined3d_texture_get_level_pow2_width(texture, level); + size.y = wined3d_texture_get_level_pow2_height(texture, level); size.z = 1.0f; size.w = 1.0f; @@ -7875,7 +7877,7 @@ static DWORD arbfp_blitter_blit(struct wined3d_blitter *blitter, enum wined3d_bl color_key = &alpha_test_key; } - arbfp_blit_set(arbfp_blitter, context, src_surface, color_key); + arbfp_blit_set(arbfp_blitter, context, src_texture, src_sub_resource_idx, color_key); /* Draw a textured quad */ draw_textured_quad(src_texture, src_sub_resource_idx, context, src_rect, dst_rect, filter); -- 2.1.4

1 0

[PATCH 1/5] wined3d: Pass a wined3d_rendertarget_info structure to context_find_fbo_entry().
by Henri Verbeet Feb. 26, 2018

Feb. 26, 2018

Signed-off-by: Henri Verbeet <hverbeet(a)codeweavers.com> --- dlls/wined3d/context.c | 122 ++++++++++++++++++++++++++++--------------------- 1 file changed, 71 insertions(+), 51 deletions(-) diff --git a/dlls/wined3d/context.c b/dlls/wined3d/context.c index 4929b84..d552c8c 100644 --- a/dlls/wined3d/context.c +++ b/dlls/wined3d/context.c @@ -413,7 +413,7 @@ static inline DWORD context_generate_rt_mask_from_resource(struct wined3d_resour } static inline void context_set_fbo_key_for_render_target(const struct wined3d_context *context, - struct wined3d_fbo_entry_key *key, unsigned int idx, struct wined3d_rendertarget_info *render_target, + struct wined3d_fbo_entry_key *key, unsigned int idx, const struct wined3d_rendertarget_info *render_target, DWORD location) { unsigned int sub_resource_idx = render_target->sub_resource_idx; @@ -487,28 +487,21 @@ static inline void context_set_fbo_key_for_render_target(const struct wined3d_co } static void context_generate_fbo_key(const struct wined3d_context *context, - struct wined3d_fbo_entry_key *key, struct wined3d_rendertarget_info *render_targets, - struct wined3d_surface *depth_stencil_surface, DWORD color_location, + struct wined3d_fbo_entry_key *key, const struct wined3d_rendertarget_info *render_targets, + const struct wined3d_rendertarget_info *depth_stencil, DWORD color_location, DWORD ds_location) { - struct wined3d_rendertarget_info depth_stencil = {{0}}; unsigned int i; key->rb_namespace = 0; - if (depth_stencil_surface) - { - depth_stencil.resource = &depth_stencil_surface->container->resource; - depth_stencil.sub_resource_idx = surface_get_sub_resource_idx(depth_stencil_surface); - depth_stencil.layer_count = 1; - } - context_set_fbo_key_for_render_target(context, key, 0, &depth_stencil, ds_location); + context_set_fbo_key_for_render_target(context, key, 0, depth_stencil, ds_location); for (i = 0; i < context->gl_info->limits.buffers; ++i) context_set_fbo_key_for_render_target(context, key, i + 1, &render_targets[i], color_location); } static struct fbo_entry *context_create_fbo_entry(const struct wined3d_context *context, - struct wined3d_rendertarget_info *render_targets, struct wined3d_surface *depth_stencil, + const struct wined3d_rendertarget_info *render_targets, const struct wined3d_rendertarget_info *depth_stencil, DWORD color_location, DWORD ds_location) { const struct wined3d_gl_info *gl_info = context->gl_info; @@ -519,11 +512,11 @@ static struct fbo_entry *context_create_fbo_entry(const struct wined3d_context * memset(&entry->key, 0, FIELD_OFFSET(struct wined3d_fbo_entry_key, objects[object_count])); context_generate_fbo_key(context, &entry->key, render_targets, depth_stencil, color_location, ds_location); entry->flags = 0; - if (depth_stencil) + if (depth_stencil->resource) { - if (depth_stencil->container->resource.format_flags & WINED3DFMT_FLAG_DEPTH) + if (depth_stencil->resource->format_flags & WINED3DFMT_FLAG_DEPTH) entry->flags |= WINED3D_FBO_ENTRY_FLAG_DEPTH; - if (depth_stencil->container->resource.format_flags & WINED3DFMT_FLAG_STENCIL) + if (depth_stencil->resource->format_flags & WINED3DFMT_FLAG_STENCIL) entry->flags |= WINED3D_FBO_ENTRY_FLAG_STENCIL; } entry->rt_mask = context_generate_rt_mask(GL_COLOR_ATTACHMENT0); @@ -536,7 +529,7 @@ static struct fbo_entry *context_create_fbo_entry(const struct wined3d_context * /* Context activation is done by the caller. */ static void context_reuse_fbo_entry(struct wined3d_context *context, GLenum target, - struct wined3d_rendertarget_info *render_targets, struct wined3d_surface *depth_stencil, + const struct wined3d_rendertarget_info *render_targets, const struct wined3d_rendertarget_info *depth_stencil, DWORD color_location, DWORD ds_location, struct fbo_entry *entry) { const struct wined3d_gl_info *gl_info = context->gl_info; @@ -546,11 +539,11 @@ static void context_reuse_fbo_entry(struct wined3d_context *context, GLenum targ context_generate_fbo_key(context, &entry->key, render_targets, depth_stencil, color_location, ds_location); entry->flags = 0; - if (depth_stencil) + if (depth_stencil->resource) { - if (depth_stencil->container->resource.format_flags & WINED3DFMT_FLAG_DEPTH) + if (depth_stencil->resource->format_flags & WINED3DFMT_FLAG_DEPTH) entry->flags |= WINED3D_FBO_ENTRY_FLAG_DEPTH; - if (depth_stencil->container->resource.format_flags & WINED3DFMT_FLAG_STENCIL) + if (depth_stencil->resource->format_flags & WINED3DFMT_FLAG_STENCIL) entry->flags |= WINED3D_FBO_ENTRY_FLAG_STENCIL; } } @@ -570,28 +563,31 @@ static void context_destroy_fbo_entry(struct wined3d_context *context, struct fb /* Context activation is done by the caller. */ static struct fbo_entry *context_find_fbo_entry(struct wined3d_context *context, GLenum target, - struct wined3d_rendertarget_info *render_targets, struct wined3d_surface *depth_stencil, + const struct wined3d_rendertarget_info *render_targets, const struct wined3d_rendertarget_info *depth_stencil, DWORD color_location, DWORD ds_location) { + static const struct wined3d_rendertarget_info ds_null = {{0}}; const struct wined3d_gl_info *gl_info = context->gl_info; unsigned int object_count = gl_info->limits.buffers + 1; struct wined3d_texture *rt_texture, *ds_texture; + unsigned int i, ds_level, rt_level; struct fbo_entry *entry; - unsigned int i, level; - if (depth_stencil && render_targets[0].resource && render_targets[0].resource->type != WINED3D_RTYPE_BUFFER) + if (depth_stencil->resource && depth_stencil->resource->type != WINED3D_RTYPE_BUFFER + && render_targets[0].resource && render_targets[0].resource->type != WINED3D_RTYPE_BUFFER) { rt_texture = wined3d_texture_from_resource(render_targets[0].resource); - level = render_targets[0].sub_resource_idx % rt_texture->level_count; - ds_texture = depth_stencil->container; - - if (wined3d_texture_get_level_width(ds_texture, depth_stencil->texture_level) - < wined3d_texture_get_level_width(rt_texture, level) - || wined3d_texture_get_level_height(ds_texture, depth_stencil->texture_level) - < wined3d_texture_get_level_height(rt_texture, level)) + rt_level = render_targets[0].sub_resource_idx % rt_texture->level_count; + ds_texture = wined3d_texture_from_resource(depth_stencil->resource); + ds_level = depth_stencil->sub_resource_idx % ds_texture->level_count; + + if (wined3d_texture_get_level_width(ds_texture, ds_level) + < wined3d_texture_get_level_width(rt_texture, rt_level) + || wined3d_texture_get_level_height(ds_texture, ds_level) + < wined3d_texture_get_level_height(rt_texture, rt_level)) { WARN("Depth stencil is smaller than the primary color buffer, disabling.\n"); - depth_stencil = NULL; + depth_stencil = &ds_null; } else if (ds_texture->resource.multisample_type != rt_texture->resource.multisample_type || ds_texture->resource.multisample_quality != rt_texture->resource.multisample_quality) @@ -599,10 +595,15 @@ static struct fbo_entry *context_find_fbo_entry(struct wined3d_context *context, WARN("Color multisample type %u and quality %u, depth stencil has %u and %u, disabling ds buffer.\n", rt_texture->resource.multisample_type, rt_texture->resource.multisample_quality, ds_texture->resource.multisample_type, ds_texture->resource.multisample_quality); - depth_stencil = NULL; + depth_stencil = &ds_null; + } + else if (depth_stencil->resource->type == WINED3D_RTYPE_TEXTURE_2D) + { + struct wined3d_surface *surface; + + surface = ds_texture->sub_resources[depth_stencil->sub_resource_idx].u.surface; + surface_set_compatible_renderbuffer(surface, &render_targets[0]); } - else - surface_set_compatible_renderbuffer(depth_stencil, &render_targets[0]); } context_generate_fbo_key(context, context->fbo_key, render_targets, depth_stencil, color_location, @@ -610,15 +611,15 @@ static struct fbo_entry *context_find_fbo_entry(struct wined3d_context *context, if (TRACE_ON(d3d)) { + struct wined3d_resource *resource; + unsigned int width, height; + const char *resource_type; + TRACE("Dumping FBO attachments:\n"); for (i = 0; i < gl_info->limits.buffers; ++i) { - struct wined3d_resource *resource; if ((resource = render_targets[i].resource)) { - unsigned int width, height; - const char *resource_type; - if (resource->type == WINED3D_RTYPE_BUFFER) { width = resource->size; @@ -628,9 +629,9 @@ static struct fbo_entry *context_find_fbo_entry(struct wined3d_context *context, else { rt_texture = wined3d_texture_from_resource(resource); - level = render_targets[i].sub_resource_idx % rt_texture->level_count; - width = wined3d_texture_get_level_pow2_width(rt_texture, level); - height = wined3d_texture_get_level_pow2_height(rt_texture, level); + rt_level = render_targets[i].sub_resource_idx % rt_texture->level_count; + width = wined3d_texture_get_level_pow2_width(rt_texture, rt_level); + height = wined3d_texture_get_level_pow2_height(rt_texture, rt_level); resource_type = "texture"; } @@ -640,16 +641,27 @@ static struct fbo_entry *context_find_fbo_entry(struct wined3d_context *context, context->fbo_key->objects[i + 1].object, width, height, resource->multisample_type); } } - if (depth_stencil) + if ((resource = depth_stencil->resource)) { - ds_texture = depth_stencil->container; - TRACE(" Depth attachment: %p format %s, %s %u, %ux%u, %u samples.\n", - depth_stencil, debug_d3dformat(ds_texture->resource.format->id), - context->fbo_key->rb_namespace & (1 << 0) ? "renderbuffer" : "texture", - context->fbo_key->objects[0].object, - wined3d_texture_get_level_pow2_width(ds_texture, depth_stencil->texture_level), - wined3d_texture_get_level_pow2_height(ds_texture, depth_stencil->texture_level), - ds_texture->resource.multisample_type); + if (resource->type == WINED3D_RTYPE_BUFFER) + { + width = resource->size; + height = 1; + resource_type = "buffer"; + } + else + { + ds_texture = wined3d_texture_from_resource(resource); + ds_level = depth_stencil->sub_resource_idx % ds_texture->level_count; + width = wined3d_texture_get_level_pow2_width(ds_texture, ds_level); + height = wined3d_texture_get_level_pow2_height(ds_texture, ds_level); + resource_type = "texture"; + } + + TRACE(" Depth attachment: %p, %u format %s, %s %u, %ux%u, %u samples.\n", + resource, depth_stencil->sub_resource_idx, debug_d3dformat(resource->format->id), + context->fbo_key->rb_namespace & (1 << 0) ? "renderbuffer" : resource_type, + context->fbo_key->objects[0].object, width, height, resource->multisample_type); } } @@ -747,8 +759,16 @@ static void context_apply_fbo_state(struct wined3d_context *context, GLenum targ } else { - context->current_fbo = context_find_fbo_entry(context, target, render_targets, depth_stencil, - color_location, ds_location); + struct wined3d_rendertarget_info ds = {{0}}; + + if (depth_stencil) + { + ds.resource = &depth_stencil->container->resource; + ds.sub_resource_idx = surface_get_sub_resource_idx(depth_stencil); + ds.layer_count = 1; + } + context->current_fbo = context_find_fbo_entry(context, target, + render_targets, &ds, color_location, ds_location); context_apply_fbo_entry(context, target, context->current_fbo); } } -- 2.1.4

1 0

Wine staging update.
by Alistair Leslie-Hughes Feb. 26, 2018

Feb. 26, 2018

Hi Everyone, Big thanks for Sebastian and Michael for all the work they have done over the years to make wine-staging such a success. The new repository is at https://github.com/wine-staging/wine-staging I've taken the challenge of maintaining wine-staging with the help of Thomas Crider Zebediah Figura We are in the middle of rebasing all the patches, which is a considerable task. After two and a half months of nothing being done, there are huge amount of conflicts. People who have never done this, please understand it is time consuming process. We are about 90% of our way through them. (Advise on how to speed up this process are welcome.) I am working on getting access to the build infrastructure to make wine-staging packages but everything takes time. In the short term our goals are 1. Rebase to a state that it compiles 2. Rework any patch to restore functionally. 3. Make an initial release. Longer term Goals 1. Reduce the amount of patches in staging (Currently sitting just over 1100) 2. Improve links between a patch and application/bug report. Once the initial release, we'll start accepting new patches. Thanks everyone for the support and encouragement for this project to succeed. Best Regards Alistair Leslie-Hughes

6 10

[PATCH] testbot: Allow explicitly specifying a Step's dependency.
by Francois Gouget Feb. 26, 2018

Feb. 26, 2018

This allows the TestBot to start running the 64 bit tests even if there are still some 32 bit tests to run. It also lets the TestBot more freely schedule the tasks when a job involves many test units (and thus many steps). Signed-off-by: Francois Gouget <fgouget(a)codeweavers.com> --- This patch requires updating the database schema. So: * Apply testbot/ddl/update31.sql * Restart Engine.pl * Restart the web server. testbot/ddl/update31.sql | 10 +++++ testbot/ddl/winetestbot.sql | 4 +- testbot/doc/winetestbot-schema.dia | 87 +++++++++++++++++++++++++++++++++++++- testbot/lib/WineTestBot/Jobs.pm | 82 +++++++++++++++++++++++++++-------- testbot/lib/WineTestBot/Patches.pm | 13 +++++- testbot/lib/WineTestBot/Steps.pm | 29 +++++++++++++ testbot/web/Submit.pl | 13 +++++- 7 files changed, 214 insertions(+), 24 deletions(-) create mode 100644 testbot/ddl/update31.sql diff --git a/testbot/ddl/update31.sql b/testbot/ddl/update31.sql new file mode 100644 index 000000000..b0d1a0456 --- /dev/null +++ b/testbot/ddl/update31.sql @@ -0,0 +1,10 @@ +USE winetestbot; + +ALTER TABLE Steps + ADD PreviousNo INT(2) NULL + AFTER No, + ADD FOREIGN KEY (JobId, PreviousNo) REFERENCES Steps(JobId, No); + +UPDATE Steps + SET PreviousNo = 1 + WHERE No > 1; diff --git a/testbot/ddl/winetestbot.sql b/testbot/ddl/winetestbot.sql index cebc26556..41740159b 100644 --- a/testbot/ddl/winetestbot.sql +++ b/testbot/ddl/winetestbot.sql @@ -129,6 +129,7 @@ CREATE TABLE Steps ( JobId INT(5) NOT NULL, No INT(2) NOT NULL, + PreviousNo INT(2) NULL, Type ENUM('suite', 'single', 'build', 'reconfig') NOT NULL, Status ENUM('queued', 'running', 'completed', 'badpatch', 'badbuild', 'boterror', 'canceled', 'skipped') NOT NULL, FileName VARCHAR(100) NOT NULL, @@ -137,7 +138,8 @@ CREATE TABLE Steps DebugLevel INT(2) NOT NULL, ReportSuccessfulTests ENUM('Y', 'N') NOT NULL, PRIMARY KEY (JobId, No), - FOREIGN KEY (JobId) REFERENCES Jobs(Id) + FOREIGN KEY (JobId) REFERENCES Jobs(Id), + FOREIGN KEY (JobId, PreviousNo) REFERENCES Steps(JobId, No) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; diff --git a/testbot/doc/winetestbot-schema.dia b/testbot/doc/winetestbot-schema.dia index 7a9483465..7d18b7271 100644 --- a/testbot/doc/winetestbot-schema.dia +++ b/testbot/doc/winetestbot-schema.dia @@ -1704,7 +1704,7 @@ <dia:point val="8.24583,-14.8834"/> </dia:attribute> <dia:attribute name="obj_bb"> - <dia:rectangle val="8.24583,-14.8834;22.2508,-6.2834"/> + <dia:rectangle val="8.24583,-14.8834;22.2508,-5.4834"/> </dia:attribute> <dia:attribute name="meta"> <dia:composite type="dict"/> @@ -1716,7 +1716,7 @@ <dia:real val="14.004999999999999"/> </dia:attribute> <dia:attribute name="elem_height"> - <dia:real val="8.5999999999999996"/> + <dia:real val="9.3999999999999986"/> </dia:attribute> <dia:attribute name="name"> <dia:string>#Steps#</dia:string> @@ -1783,6 +1783,29 @@ <dia:string>##</dia:string> </dia:attribute> </dia:composite> + <dia:composite type="table_attribute"> + <dia:attribute name="name"> + <dia:string>#Previous#</dia:string> + </dia:attribute> + <dia:attribute name="type"> + <dia:string>#INT(2)#</dia:string> + </dia:attribute> + <dia:attribute name="comment"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="primary_key"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="nullable"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="unique"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="default_value"> + <dia:string>##</dia:string> + </dia:attribute> + </dia:composite> <dia:composite type="table_attribute"> <dia:attribute name="name"> <dia:string>#Type#</dia:string> @@ -3556,5 +3579,65 @@ <dia:connection handle="1" to="O23" connection="12"/> </dia:connections> </dia:object> + <dia:object type="Database - Reference" version="0" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="8.24583,-12.3834"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.87763,-13.0334;8.29583,-11.5334"/> + </dia:attribute> + <dia:attribute name="meta"> + <dia:composite type="dict"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="8.24583,-12.3834"/> + <dia:point val="5.92763,-12.3834"/> + <dia:point val="5.92763,-11.5834"/> + <dia:point val="8.24583,-11.5834"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="0"/> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="orth_autoroute"> + <dia:boolean val="false"/> + </dia:attribute> + <dia:attribute name="text_colour"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="line_colour"> + <dia:color val="#000000ff"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.10000000000000001"/> + </dia:attribute> + <dia:attribute name="line_style"> + <dia:enum val="0"/> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="start_point_desc"> + <dia:string>#0..1#</dia:string> + </dia:attribute> + <dia:attribute name="end_point_desc"> + <dia:string>#0..n#</dia:string> + </dia:attribute> + <dia:attribute name="normal_font"> + <dia:font family="monospace" style="0" name="Courier"/> + </dia:attribute> + <dia:attribute name="normal_font_height"> + <dia:real val="0.59999999999999998"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O11" connection="14"/> + <dia:connection handle="1" to="O11" connection="16"/> + </dia:connections> + </dia:object> </dia:layer> </dia:diagram> diff --git a/testbot/lib/WineTestBot/Jobs.pm b/testbot/lib/WineTestBot/Jobs.pm index 3eba27bc2..eea16864e 100644 --- a/testbot/lib/WineTestBot/Jobs.pm +++ b/testbot/lib/WineTestBot/Jobs.pm @@ -84,6 +84,28 @@ sub InitializeNew($$) $self->SUPER::InitializeNew($Collection); } +=pod +=over 12 + +=item C<OnDelete()> + +Resets the Steps PreviousNo fields because the corresponding foreign key +references both this Job and the Steps, thus preventing their deletion. + +=back +=cut + +sub OnDelete($) +{ + my ($self) = @_; + + my $Steps = $self->Steps; + map { $_->PreviousNo(undef) } @{$Steps->GetItems()}; + my ($_ErrProperty, $ErrMessage) = $Steps->Save(); + + return $ErrMessage || $self->SUPER::OnDelete(); +} + sub GetDir($) { my ($self) = @_; @@ -843,20 +865,45 @@ sub _ScheduleTasks($) { $JobRank++; - # The list of VMs that should be getting ready to run + # The per-step lists of VMs that should be getting ready to run # before we prepare the next step - my $PreviousVMs = []; + my %StepVMs = ("" => []); # no dependency for the first step - my $StepRank = 0; + # Process the steps in increasing $Step->No order for the inter-step + # dependencies my $Steps = $Job->Steps; $Steps->AddFilter("Status", ["queued", "running"]); foreach my $Step (sort { $a->No <=> $b->No } @{$Steps->GetItems()}) { - # StepRank 0 contains the runnable tasks, 1 the 'may soon be runnable' - # ones, and 2 and greater tasks we don't care about yet - $Step->HandleStaging() if ($StepRank == 0 and $Step->Status eq "queued"); - - my $StepVMs = []; + my $StepRank; + my $Previous = ""; # Avoid undefined values for hash indices + if (!$Step->PreviousNo) + { + # The first step may need to get files from the staging area + $Step->HandleStaging() if ($Step->Status eq "queued"); + $StepRank = 0; + $StepVMs{$Step} = []; + } + else + { + $Previous = $Steps->GetItem($Step->PreviousNo); + if ($Previous->Status eq "completed") + { + # The previous step was successful so we can now run this one + $StepRank = 0; + $StepVMs{$Step} = []; + } + elsif ($StepVMs{$Previous}) + { + # The previous step is almost done. Prepare this one. + $StepRank = 1; + } + else + { + # The previous step is nowhere near done + $StepRank = 2; + } + } my $Tasks = $Step->Tasks; $Tasks->AddFilter("Status", ["queued"]); @@ -872,17 +919,18 @@ sub _ScheduleTasks($) $Host->{queued}++; $Sched->{queued}++; - if ($StepRank >= 2 or !$PreviousVMs) + if ($StepRank >= 2) { # The previous step is nowhere near done so skip this one for now next; } if ($StepRank == 1) { - # Passing $PreviousVMs ensures this VM will be reverted if and only - # if all of the previous step's tasks are about to run. + # Passing $StepVMs{$Previous} ensures this VM will be reverted + # if and only if all of the previous step's tasks are about to run. # See _HasMissingDependencies(). - _AddNeededVM($NeededVMs, $VM, $NEXT_BASE + $JobRank, $PreviousVMs); + _AddNeededVM($NeededVMs, $VM, $NEXT_BASE + $JobRank, + $StepVMs{$Previous}); next; } $Sched->{runnable}++; # $StepRank == 0 @@ -893,12 +941,12 @@ sub _ScheduleTasks($) # scheduled to be reverted for a task with a higher priority. # So this task won't be run before a while and thus there is # no point in preparing the next step. - $StepVMs = undef; + $StepVMs{$Step} = undef; next; } # It's not worth preparing the next step for tasks that take so long - $StepVMs = undef if ($Task->Timeout > $BuildTimeout); + $StepVMs{$Step} = undef if ($Task->Timeout > $BuildTimeout); my $VMKey = $VM->GetKey(); if ($VM->Status eq "idle") @@ -939,12 +987,10 @@ sub _ScheduleTasks($) { # We cannot use the VM because it is busy (running another task, # offline, etc.). So it is too early to prepare the next step. - $StepVMs = undef; + $StepVMs{$Step} = undef; } - push @$StepVMs, $VM if ($StepVMs); + push @{$StepVMs{$Step}}, $VM if ($StepVMs{$Step}); } - $PreviousVMs = $StepVMs; - $StepRank++; } } diff --git a/testbot/lib/WineTestBot/Patches.pm b/testbot/lib/WineTestBot/Patches.pm index 42118e614..d572ec470 100644 --- a/testbot/lib/WineTestBot/Patches.pm +++ b/testbot/lib/WineTestBot/Patches.pm @@ -281,7 +281,15 @@ sub Submit($$$) my $Task = $NewStep->Tasks->Add(); $Task->VM($BuildVM); $Task->Timeout($BuildTimeout); - + + # Save this step (&job+task) so the others can reference it + my ($ErrKey, $ErrProperty, $ErrMessage) = $Jobs->Save(); + if (defined($ErrMessage)) + { + $self->Disposition("Failed to submit build step"); + return $ErrMessage; + } + foreach my $Unit (keys %{$Modules{$Module}}) { # Add 32 and 64-bit tasks @@ -294,6 +302,7 @@ sub Submit($$$) { # Create the corresponding Step $NewStep = $Steps->Add(); + $NewStep->PreviousNo(1); my $FileName = $Module; $FileName .= ".exe" if ($Modules{$Module}{$Unit} eq "patchprograms"); $FileName .= "_test"; @@ -317,7 +326,7 @@ sub Submit($$$) } } - my ($ErrKey, $ErrProperty, $ErrMessage) = $Jobs->Save(); + ($ErrKey, $ErrProperty, $ErrMessage) = $Jobs->Save(); if (defined($ErrMessage)) { $self->Disposition("Failed to submit job"); diff --git a/testbot/lib/WineTestBot/Steps.pm b/testbot/lib/WineTestBot/Steps.pm index 95c0a45aa..45ec6359f 100644 --- a/testbot/lib/WineTestBot/Steps.pm +++ b/testbot/lib/WineTestBot/Steps.pm @@ -30,6 +30,10 @@ A Job is composed of multiple Steps that each do a specific operation: build the test executable, or run a given test, etc. A Step is in turn composed of a WineTestBot::Task object for each VM it should be run on. +Note that the PreviousNo relation will prevent the deletion of the target Step. +It is the responsibility of the caller to delete the Steps in a suitable order, +or to reset their PreviousNo fields beforehand. + =cut use File::Copy; @@ -61,6 +65,30 @@ sub InitializeNew($$) $self->SUPER::InitializeNew($Collection); } +=pod +=over 12 + +=item C<Validate()> + +Enforces strict ordering to avoid loops. + +Note that a side effect is that processing steps in increasing step number +order is sufficient to ensure the dependencies are processed first. + +=back +=cut + +sub Validate($) +{ + my ($self) = @_; + + if ($self->PreviousNo and $self->PreviousNo >= $self->No) + { + return ("PreviousNo", "The previous step number must be less than this one's."); + } + return $self->SUPER::Validate(); +} + sub GetDir($) { my ($self) = @_; @@ -178,6 +206,7 @@ BEGIN { @PropertyDescriptors = ( CreateBasicPropertyDescriptor("No", "Step no", 1, 1, "N", 2), + CreateBasicPropertyDescriptor("PreviousNo", "Previous step", !1, !1, "N", 2), CreateEnumPropertyDescriptor("Status", "Status", !1, 1, ['queued', 'running', 'completed', 'badpatch', 'badbuild', 'boterror', 'canceled', 'skipped']), CreateEnumPropertyDescriptor("Type", "Step type", !1, 1, ['suite', 'single', 'build', 'reconfig']), CreateBasicPropertyDescriptor("FileName", "File name", !1, 1, "A", 100), diff --git a/testbot/web/Submit.pl b/testbot/web/Submit.pl index 0272a283d..875384ea3 100644 --- a/testbot/web/Submit.pl +++ b/testbot/web/Submit.pl @@ -834,6 +834,7 @@ sub OnSubmit($) my $Steps = $NewJob->Steps; my $FileType = $self->GetParam("FileType"); + my $BuildStepNo; if ($FileType eq "patchdlls" || $FileType eq "patchprograms") { # This is a patch so add a build step... @@ -852,6 +853,15 @@ sub OnSubmit($) my $Task = $BuildStep->Tasks->Add(); $Task->VM($BuildVM); $Task->Timeout($BuildTimeout); + + # Save this step (&job+task) so the others can reference it + my ($ErrKey, $ErrProperty, $ErrMessage) = $Jobs->Save(); + if (defined($ErrMessage)) + { + $self->{ErrMessage} = $ErrMessage; + return !1; + } + $BuildStepNo = 1; } # Add steps and tasks for the 32 and 64-bit tests @@ -875,6 +885,7 @@ sub OnSubmit($) { # First create the test step my $TestStep = $Steps->Add(); + $TestStep->PreviousNo($BuildStepNo); if ($FileType eq "patchdlls" || $FileType eq "patchprograms") { my $FileName=$self->GetParam("TestExecutable"); @@ -905,7 +916,7 @@ sub OnSubmit($) } } - # Now save the whole thing + # Now save the whole thing (or whatever's left to save) my ($ErrKey, $ErrProperty, $ErrMessage) = $Jobs->Save(); if (defined($ErrMessage)) { -- 2.16.1

1 0

[PATCH] testbot: Remove a debug trace from the Munin plugin.
by Francois Gouget Feb. 26, 2018

Feb. 26, 2018

Signed-off-by: Francois Gouget <fgouget(a)codeweavers.com> --- testbot/web/Munin.pl | 1 - 1 file changed, 1 deletion(-) diff --git a/testbot/web/Munin.pl b/testbot/web/Munin.pl index 9e65421b1..91155b776 100755 --- a/testbot/web/Munin.pl +++ b/testbot/web/Munin.pl @@ -132,7 +132,6 @@ sub GetOutput($) my $Stats = GetStatistics($VMs, $Period * 60); my @SortedHosts = sort keys %{$Stats->{hosts}->{items}}; - print sorted keys %{$Stats->{hosts}->{items}}, "\n"; my %HostDisplayNames; foreach my $Host (@SortedHosts) { -- 2.16.1

1 0