http://bugs.winehq.org/show_bug.cgi?id=35576
Bug ID: 35576 Summary: WineTestBot leaks staging files Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs@winehq.org Reporter: fgouget@codeweavers.com Classification: Unclassified
Currently the tools/testbot/var/staging directory of my test environment has more than 10000 files which is way more than the number of current tests. There are probably two issues:
* The janitor script purges all tests that are more than a week old. Yet a lot of files are older than that. It's possible that the staging files should not be there any more once a test has been run and thus by the time the Janitor script purges it. However the Janitor script should probably still ensure that the staging files get purged too.
* It seems that whenever WinePatchesWebSubmit.pl is run it creates a set of staging files. However if the Engine is stopped those won't be processed immediately and thus the next time WinePatchesWebSubmit.pl runs it will create a new set of staging files, presumably for the same set of patches. It's likely that this is also causes a leak, probably exacerbated by the Janitor script. Fortunately the official WineTestBot is rarely offline though.
http://bugs.winehq.org/show_bug.cgi?id=35576
--- Comment #1 from François Gouget fgouget@codeweavers.com --- One case where staging files can be leaked is simply if one goes to the 'Submit Job' page, upload a patch, and simply close the web page. In the first 'Submit Job' step the patch is uploaded to the staging directory but that it is only put in place once the job is created in the last step.
Furthermore, opening two 'Submit Job' windows to submit the same patch twice results in a staging filename collision so that one of the jobs has a broken patch filename.
http://bugs.winehq.org/show_bug.cgi?id=35576
François Gouget fgouget@codeweavers.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |FIXED
--- Comment #2 from François Gouget fgouget@codeweavers.com --- The following commits fixed the leaks:
commit cc218ea2c267f2aee508b5d9ed26a9886b29302f Author: Francois Gouget fgouget@codeweavers.com Date: Wed May 21 16:21:36 2014 +0200
testbot/Janitor: Check and clean up the staging directory.
Purge unsubmitted jobs (web/Submit.pl) as soon as they are a day old. There should be no file older than $JobPurgeDays. Report them so they are investigated and delete them after a 7 days grace period. Detect unknown files and report them so their source can be investigated.
commit 79460be9dd65f78183c9291d574cac2313e798f5 Author: Francois Gouget fgouget@codeweavers.com Date: Wed May 21 16:21:15 2014 +0200
testbot/web: Make it easier to identify Submit's temporary staging files.
It also ensures that GetTmpStagingFullPath() won't get a collision between the session id and the random part of an existing staging file, no matter what filename the user picks. Finally, this will make it possible to better check the content of the staging directory.
commit ed692db13e0b97d4b535fadae700d51fe45aa695 Author: Francois Gouget fgouget@codeweavers.com Date: Wed May 21 16:19:57 2014 +0200
testbot/CheckForWinetestUpdate: Improve error handling and plug staging file leaks.
The leaks would happen whenever there was no non-base VM, no 64-bit VM, or CheckForWinetestUpdate would exit prematurely due to an error. Also let AddJob() handle the staging file. This greatly simplifies the calling code. Finally, fix the indentation in a couple of places.
commit 71b945c8e59daf1139241ffaf1b1882a7b0573ae Author: Francois Gouget fgouget@codeweavers.com Date: Fri May 16 15:50:09 2014 +0200
testbot/lib: Move staging files into place rather than copying them.
commit b14902e06b73d7f18adce0cd9d24852cd2ac3a8a Author: Francois Gouget fgouget@codeweavers.com Date: Fri May 16 15:49:58 2014 +0200
testbot: Make Steps use the real filename for staging files too.
commit eeed0b5f0f4af7d985e9e72135f6f75ffe1c1d82 Author: Francois Gouget fgouget@codeweavers.com Date: Thu May 15 15:51:04 2014 +0200
testbot/CheckWinetestUpdate: Let the Engine handle moving staging files to the jobs directory.
Otherwise there is a race condition between CheckWinetestUpdate and the Engine.
commit 7d4c644fccff381619dbc49a48ae644daf7b81b4 Author: Francois Gouget fgouget@codeweavers.com Date: Thu May 15 15:48:45 2014 +0200
testbot: Better document the use of the staging directory.
Also remove an unused variable in CheckForWinetestUpdate.
commit 3471277f654cb79ac2dc0c4393c37a4a3fab47d8 Author: Francois Gouget fgouget@codeweavers.com Date: Wed May 14 19:34:06 2014 +0200
testbot/web: Always use a random prefix for the staging filename.
Starting with the session id means we'll have to try again to find a new unique filename.
commit 82049e35353b4da6d074977cf1b05003c22283dd Author: Francois Gouget fgouget@codeweavers.com Date: Wed May 7 16:43:47 2014 +0200
testbot/WinePatchesWebSubmit: Improve robustness, particularly when the Engine is not running.
Now the Engine will automatically pick up patches from the patches website as part of its periodic 'SafetyNet' check. However WinePatchesWebSubmit can still be used to avoid the potential 10 minutes delay. Also, running WinePatchesWebSubmit when the Engine is stopped no longer has the side effect of leaking files in the staging directory.
commit 2cdc80043ed19e500164d4255a2551bb24089203 Author: Francois Gouget fgouget@codeweavers.com Date: Wed May 7 02:41:26 2014 +0200
testbot/WinePatchesMLSubmit: Better handle errors and times when the Engine is not running.
WinePatchesMLSubmit must return a non-zero exit code in case it could not deliver the message so the caller gets a chance to handle the situation gracefully. Have the Engine scan the staging directory for pending patches from the mailing list and automatically handle them, in particular on startup. This makes it possible for WinePatchesMLSubmit to deliver the patches even if the Engine is not running which is more robust. This also fixes a potential source of staging files leak.
https://bugs.winehq.org/show_bug.cgi?id=35576
Bruno Jesus 00cpxxx@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED
--- Comment #3 from Bruno Jesus 00cpxxx@gmail.com --- Closing other products fixed bugs.
-
wine-bugs＠winehq.org