[Bug 3817] InstallShield very slow when copying many small files

http://bugs.winehq.org/show_bug.cgi?id=3817

Joseph Miller  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #29 from Joseph Miller   2009-09-05 09:10:55 ---
> imho the best solution for this (except the locking-thing) would be to rely on
> a cached intermediate state. when looking up files in a certain directory, we
> read the entire directory into a cache-structure (including ordering Foo.bar
> and foo.Bar) and start monitoring this directory using inotify.
> one big advantage about inotify is the clarity of messages: when a new file is
> added to the directory in question, we get to know the file's name in advance
> and don't need to re-read the entire directory. consider a directory with 1.000
> files, adding 100 files would require reading the directory once, reading from
> the cache 100 times and adding to the cache 100 times.
> if inotify is not present, we can always fall back to the slower version
> already implemented.

This sounds like the best solution to me as well.  The only problem with it is
that it is unlikely one would want wine to monitor every single directory that
any program accesses.  Windows programs are notorious for opening many files in
many directories.

Perhaps the cache could be designed as a needs-based cache.  The cache could
keep a list of each directory accessed, the last time it was accessed, and a
count for how many times the directory has been accessed in the last 2-3
seconds.  Once the count rises above a threshold for a directory, caching of
the directory would begin.  Once a directory hasn't been accessed for a period
of time, it could be removed from monitoring and caching.  This would provide
for smart memory management and significant performance increases as well.

More comments are needed on this so a request can be submitted for the best way
to do this.  Wine's got a good history of rejecting patches and I'm not going
to be very excited about starting work only to have to redo everything once I
finish.

We also need a test program to demo this.  I will try to make a simple EXE to
demonstrate this do a simple benchmark so progress can be shown.