ntoskrnl status

[Mario Demontis]

Hello
It's my first post here...

Any news about ntoskrnl.exe support?
As far as I can understand it's not yet integrated...
Are there any new patches?

Thanks in advance
Mario

--

Email.it, the professional e-mail, gratis per te: http://www.email.it/f

Sponsor:

Grande Offerta di Benvenuto Cassine di Pietra:

* 36 bottiglie di ottimo vino veneto direttamente a casa tua!

*

Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=3923&d=15-6

[Brian Vincent]

On 6/15/06, Mario Demontis  wrote:
> Any news about ntoskrnl.exe support?
> As far as I can understand it's not yet integrated...
> Are there any new patches?

Read here first:
http://www.winehq.com/pipermail/wine-devel/2006-April/047262.html

Read here next:
http://www.winehq.com/pipermail/wine-devel/2006-June/048583.html

Vitaliy Margolen seems to be working on it a bit.

Vitaliy - in your email above you wrote, "Here is last instalment".
Did you mean, "Here is _the latest_ installment."?  I was going to
include that post in this week's WWN and just wanted to make it
correct.

-Brian

[Vitaliy Margolen]

Thursday, June 15, 2006, 10:01:03 AM, Brian Vincent wrote:
> On 6/15/06, Mario Demontis  wrote:
> > Any news about ntoskrnl.exe support?
> > As far as I can understand it's not yet integrated...
> > Are there any new patches?
> Read here first:
> http://www.winehq.com/pipermail/wine-devel/2006-April/047262.html
> Read here next:
> http://www.winehq.com/pipermail/wine-devel/2006-June/048583.html
> Vitaliy Margolen seems to be working on it a bit.
> Vitaliy - in your email above you wrote, "Here is last instalment".
> Did you mean, "Here is _the latest_ installment."?  I was going to
> include that post in this week's WWN and just wanted to make it
> correct.

Yes sorry. It was last at the time I sent it. Now I have one more (with
suggested fixes and against yesterday's git.

As far as state of it:
1. It runs again using new method of communications to ntoskrnl.
2. It's still more of the prove of concept stage then production quality code.
3. There are several hacks that needs to be dealt with.
4. Not yet agreed on what ntoskrnl.exe should be (a dll, or a separate process).

1) We used named pipe to talk to ntoskrnl directly from the user space. Now all
the communication is going through wineserver.

2) New ioctl code is highly experimental. wineserver needs to use already setup
objects and structures. To start with, I think user space code and ntoskrnl code
are good enough.
Driver loading looks fine, with exception of service query. We need some way to
check if driver is loaded and running (might need to add one more ioctl to the
ntoskrnl).

3) ntoskrnl startup is suboptimal. It should look like explorer's startup (there
are few problems and major differences however). One more hack is associated
with process startup sequence of events. Wine differs from native in such a way
that brakes safedisc.

4) Alexandre offered an idea to make ntoskrnl.exe a dll that could be loaded by
an existent explorer process. This is an interesting approach, but there could
be some complications associated with it.

So to sum it up: we have a working prototype that's been in this state for
almost a year now. It's big project with number of different parts. Each part
could be a big project on it's own if implemented all the way. So if anyone
wants to spend some time learning the subject and getting it into Wine, they are
very welcome to jump on board.

Vitaliy.

[Vijay Kiran Kamuju]

If Alexander agrees on what path we should take on this, we should be
having some real stuff getting in.
A'int it Vitaly.
Well i will look into the ntoskrnl stuff as soon as i get my laptop
I think should be like a seperate process as in windows.

Just my two cents.

On 6/15/06, Vitaliy Margolen  wrote:
> Thursday, June 15, 2006, 10:01:03 AM, Brian Vincent wrote:
> > On 6/15/06, Mario Demontis  wrote:
> > > Any news about ntoskrnl.exe support?
> > > As far as I can understand it's not yet integrated...
> > > Are there any new patches?
> > Read here first:
> > http://www.winehq.com/pipermail/wine-devel/2006-April/047262.html
> > Read here next:
> > http://www.winehq.com/pipermail/wine-devel/2006-June/048583.html
> > Vitaliy Margolen seems to be working on it a bit.
> > Vitaliy - in your email above you wrote, "Here is last instalment".
> > Did you mean, "Here is _the latest_ installment."?  I was going to
> > include that post in this week's WWN and just wanted to make it
> > correct.
>
> Yes sorry. It was last at the time I sent it. Now I have one more (with
> suggested fixes and against yesterday's git.
>
> As far as state of it:
> - It runs again using new method of communications to ntoskrnl.
> - It's still more of the prove of concept stage then production quality code.
> - There are several hacks that needs to be dealt with.
> - Not yet agreed on what ntoskrnl.exe should be (a dll, or a separate process).
> - We used named pipe to talk to ntoskrnl directly from the user space. Now all
>
> the communication is going through wineserver.
> - New ioctl code is highly experimental. wineserver needs to use already setup
>
> objects and structures. To start with, I think user space code and ntoskrnl code
> are good enough.
> Driver loading looks fine, with exception of service query. We need some way to
> check if driver is loaded and running (might need to add one more ioctl to the
> ntoskrnl).
> - ntoskrnl startup is suboptimal. It should look like explorer's startup (there
>
> are few problems and major differences however). One more hack is associated
> with process startup sequence of events. Wine differs from native in such a way
> that brakes safedisc.
> - Alexandre offered an idea to make ntoskrnl.exe a dll that could be loaded by
>
> an existent explorer process. This is an interesting approach, but there could
> be some complications associated with it.
>
> So to sum it up: we have a working prototype that's been in this state for
> almost a year now. It's big project with number of different parts. Each part
> could be a big project on it's own if implemented all the way. So if anyone
> wants to spend some time learning the subject and getting it into Wine, they are
> very welcome to jump on board.
>
> Vitaliy.

[Mario Demontis]

In http://www.winehq.com/pipermail/wine-devel/2006-June/048583.html
I read that the patch is Diffed against
1d40bf0141b7f67b1188555962698f5dab631bc3
What does it mean? How can I get my cvs directory to that state?

Thanks
Mario

On gio, 2006-06-15 at 20:48 -0600, Vitaliy Margolen wrote:
> Thursday, June 15, 2006, 10:01:03 AM, Brian Vincent wrote:
> > On 6/15/06, Mario Demontis  wrote:
> > > Any news about ntoskrnl.exe support?
> > > As far as I can understand it's not yet integrated...
> > > Are there any new patches?
> > Read here first:
> > http://www.winehq.com/pipermail/wine-devel/2006-April/047262.html
> > Read here next:
> > http://www.winehq.com/pipermail/wine-devel/2006-June/048583.html
> > Vitaliy Margolen seems to be working on it a bit.
> > Vitaliy - in your email above you wrote, "Here is last instalment".
> > Did you mean, "Here is _the latest_ installment."?  I was going to
> > include that post in this week's WWN and just wanted to make it
> > correct.
>
> Yes sorry. It was last at the time I sent it. Now I have one more (with
> suggested fixes and against yesterday's git.
>
> As far as state of it:
> - It runs again using new method of communications to ntoskrnl.
> - It's still more of the prove of concept stage then production quality code.
> - There are several hacks that needs to be dealt with.
> - Not yet agreed on what ntoskrnl.exe should be (a dll, or a separate process).
> - We used named pipe to talk to ntoskrnl directly from the user space. Now all
>
> the communication is going through wineserver.
> - New ioctl code is highly experimental. wineserver needs to use already setup
>
> objects and structures. To start with, I think user space code and ntoskrnl code
> are good enough.
> Driver loading looks fine, with exception of service query. We need some way to
> check if driver is loaded and running (might need to add one more ioctl to the
> ntoskrnl).
> - ntoskrnl startup is suboptimal. It should look like explorer's startup (there
>
> are few problems and major differences however). One more hack is associated
> with process startup sequence of events. Wine differs from native in such a way
> that brakes safedisc.
> - Alexandre offered an idea to make ntoskrnl.exe a dll that could be loaded by
>
> an existent explorer process. This is an interesting approach, but there could
> be some complications associated with it.
>
> So to sum it up: we have a working prototype that's been in this state for
> almost a year now. It's big project with number of different parts. Each part
> could be a big project on it's own if implemented all the way. So if anyone
> wants to spend some time learning the subject and getting it into Wine, they are
> very welcome to jump on board.
>
> Vitaliy.

--

Email.it, the professional e-mail, gratis per te: http://www.email.it/f

Sponsor:

Ascolta tutta la musica che vuoi gratis!

* Clicca su www.radiosnj.com

Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=5176&d=16-6

[Jonathan Ernst]

Le vendredi 16 juin 2006 à 11:37 +0200, Mario Demontis a écrit :
> In http://www.winehq.com/pipermail/wine-devel/2006-June/048583.html
> I read that the patch is Diffed against
> 1d40bf0141b7f67b1188555962698f5dab631bc3
> What does it mean? How can I get my cvs directory to that state?
>
> Thanks
> Mario

You can use Git: http://wiki.winehq.org/GitWine

Regards.

P.S. please don't top-post bottom-posted e-mails/threads

[Mike McCormack]

Mario Demontis wrote:
> I read that the patch is Diffed against
> 1d40bf0141b7f67b1188555962698f5dab631bc3
> What does it mean? How can I get my cvs directory to that state?

http://wiki.winehq.org/GitWine

Mike

[Jaap Stolk]

I also would like to try the ntoskrnl.exe patch.
I uninstalled and deleted my cvs wine, updated python and installed
git, and completed "Cloning the Wine GIT repository". I had a little
peek at the patch by renaming it to .bz2, but how do I add this patch
to to my local wine copy, using git ?
http://wiki.winehq.org/GitWine seems to be more oriented at making
patches, not importing a patch made by someone else.

[Saulius Krasuckas]

* On Tue, 20 Jun 2006, Jaap Stolk wrote:
> I uninstalled and deleted my cvs wine, updated python and installed
> git, and completed "Cloning the Wine GIT repository". I had a little
> peek at the patch by renaming it to .bz2, but how do I add this patch
> to to my local wine copy, using git ?

I'd make some experimental branch or "master" or defined commit, then 
patch it with given diff, and commit the changes under single name.
> http://wiki.winehq.org/GitWine seems to be more oriented at making
> patches, not importing a patch made by someone else.

$ bunzip2 --keep origin_sd1.diff.bz2
$ git branch ntoskrnl 1d40bf0141b7f67b1188555962698f5dab631bc3
$ git branch
$ git checkout ntoskrnl
$ git branch
$ cat origin_sd1.diff | patch -p1
$ tools/make_requests
$ git commit -a -m "ntoskrnl: Experimental implementation."
  # recompile Wine
$ ./configure && make depend && make && tools/wineprefixcreate --use-wine-tree .
  # below goes your experiments
$ ...
  # and here we go back to the normal tree
$ git checkout master
  # recompile Wine
$ ./configure && make depend && make && tools/wineprefixcreate --use-wine-tree .
  # and do our stuff
$ ...

[Mike McCormack]

Saulius Krasuckas wrote:
> > http://wiki.winehq.org/GitWine seems to be more oriented at making
> > patches, not importing a patch made by someone else.
> $ cat origin_sd1.diff | patch -p1
> $ tools/make_requests
> $ git commit -a -m "ntoskrnl: Experimental implementation."

If the patch is created with "git format-message", you can use "git am 
origin_sd1.diff" to add it to your tree.  'am' applies a mailbox, which 
is a series of patches and their commit info.

Mike

[Saulius Krasuckas]

* On Tue, 20 Jun 2006, Mike McCormack wrote:
> - Saulius Krasuckas wrote:
> > > http://wiki.winehq.org/GitWine seems to be more oriented at making
> > > patches, not importing a patch made by someone else.
> > $ cat origin_sd1.diff | patch -p1
> > $ tools/make_requests
> > $ git commit -a -m "ntoskrnl: Experimental implementation."
>
> If the patch is created with "git format-message", you can use "git am
> origin_sd1.diff" to add it to your tree.  'am' applies a mailbox, which is a
> series of patches and their commit info.

Nice, thanks Mike.

BTW, my "tutorial" won't work well, because after the "patch" cmd no one 
instructs git to add newly created files to a repository.  So my answer 
should be enchanced.  I've just accidentally found a command to replace 
"patch", it's "git-apply".  Unfortunately it doesn't add newly created 
files for me, thus I change my advise this way:

$ cat origin_sd1.diff | patch -p1 | awk '{print $3}' | xargs git-update-index --add
$ tools/make_requests
$ git commit -a -m "ntoskrnl: Experimental implementation."

[Jaap Stolk]

Thanks for the "tutorial" Saulius. (I'm new to git) It's all compiled now.

Just to summarize: (with your enhancements)

# (after completing: http://wiki.winehq.org/GitWine )
 # rename/extract patch:
$ cd wine-git
$ mv origin_sd1.diff-0001.obj origin_sd1.diff.bz2
$ bunzip2 --keep origin_sd1.diff.bz2
 # patch wine:
$ git branch ntoskrnl 1d40bf0141b7f67b1188555962698f5dab631bc3
$ git branch
$ git checkout ntoskrnl
$ git branch
$ cat origin_sd1.diff | patch -p1 | awk '{print $3}' | xargs
git-update-index --add
$ tools/make_requests
$ git commit -a -m "ntoskrnl: Experimental implementation."
 # recompile Wine:
$ ./configure && make depend && make && tools/wineprefixcreate --use-wine-tree .
 # below goes your experiments
$ ...
 # and here we go back to the normal tree
$ git checkout master
 # recompile Wine
$ ./configure && make depend && make && tools/wineprefixcreate --use-wine-tree .
 # and do our stuff
$ ...

I noticed that the typical "make install" is missing.
I ran ./wine-git/programs/winecfg/winecfg and it seems to work ok.
Found some more info about this in an old Wine newsletter:
http://www.winehq.com/?issue=269#Speeding%20Up%20Builds

[Jaap Stolk]

On 6/20/06, Jaap Stolk  wrote:
> <snip>
> I noticed that the typical "make install" is missing.
> I ran ./wine-git/programs/winecfg/winecfg and it seems to work ok.
> Found some more info about this in an old Wine newsletter:
> http://www.winehq.com/?issue=269#Speeding%20Up%20Builds

And a script to setup the correct wine variables without installing wine.
http://wiki.jswindle.com/index.php/General_Developer_Information#Running_Win...

ntoskrnl followup

[Frans Kool]

Hi Vitaliy,

If I read the thread concerning ntoskrnl insertion into Wine correctly, the
patch will have to be divided into several parts to be accepted by AJ.

For this, tests need probably be written to make sure each sub-patch is
performing as it should. I haven't written C-code for a while, but there are a
lot of tests available for me to have a crack at it.

As I see it, the following tests/patches are needed (in this order?)

1) Tests for talking to ntoskrnl through wineserver (setting up channel, testing
messages both correct and incorrect and closing channel again) in order to test
the new way of communication.

2) Tests to try the ioctl structures

3) loading a device driver, talking to it, unloading it (including negatives
like unloading a non-loaded driver etc).

4) Open issues like detecting if the driver is actually loaded instead of
waiting X ms....

The questions I have are:
a) Is the breakdown correct, or are there more atomic parts (easier to test/get
into wine). Do you already have a breakdown idea of the patch into smaller parts
 so I/we can focus on this?
b) Did I miss tests or do you have ideas for more?
c) Is the order of the patches correct? If not, please correct me so I can start
creating tests.
d) Can I somehow access the most recent version of the patch? I have several
versions now, but they keep on changing/improving(!). It would be nice if you
could publish it somewhere (nightly CVS tarball?), so more people can provide
feedback on it too.

As you can see, I'm just trying to see if I can contribute on such a big project.
Thanks for the feedback,

Frans.

ntoskrnl followup

[Vitaliy Margolen]

Tuesday, June 20, 2006, 5:39:26 AM, Frans Kool wrote:
> Hi Vitaliy,
> If I read the thread concerning ntoskrnl insertion into Wine correctly, the
> patch will have to be divided into several parts to be accepted by AJ.

That is not the main problem. It will have to be divided, but it would be few
pieces.
> For this, tests need probably be written to make sure each sub-patch is
> performing as it should. I haven't written C-code for a while, but there are a
> lot of tests available for me to have a crack at it.

I'm not so sure how much can you test here. Under "tests" here we mean
conformance tests that could be compiled as an exe and ran on native platforms.
To test ntoskrnl you one would need kernel driver to test against. But this is
not something that we can compile ourselves. It requires DDK.
Also the only part that could be tested is ioctls, not the driver
loading/unloading.
> As I see it, the following tests/patches are needed (in this order?)
> - Tests for talking to ntoskrnl through wineserver (setting up channel, testing
>
> messages both correct and incorrect and closing channel again) in order to test
> the new way of communication.

This part can be a stand alone test, not a part of the test suite. It would be
helpful of course but I'm not sure if it could be committed.
> - Tests to try the ioctl structures

I'm not sure what you mean here. There are lots of data types associated with
predefined ioctls. But that's not what we need here.
> - loading a device driver, talking to it, unloading it (including negatives
>
> like unloading a non-loaded driver etc).

This we can not really test with integrated tests. We need a driver and admin
privileges.
> - Open issues like detecting if the driver is actually loaded instead of
>
> waiting X ms....

That's more of the design question. There are several ways how this could be
done. It could be as simple as adding one more internal ioctl.
> The questions I have are:
> a) Is the breakdown correct, or are there more atomic parts (easier to test/get
> into wine). Do you already have a breakdown idea of the patch into smaller parts
>  so I/we can focus on this?

I would brake it down to the following pieces:
1. ioctls to the wineserver "devices" (named-pipe, mail-slot, etc). This is an
   atomic change.
2. Implementing ntoskrnl.exe. So far we have it as a separate program. It could
   be turned into a dll.
3. Other part of the ioctls (wineserver <-> ntoskrnl).
4. Pieces required to load/unload/query driver and processing ioctls.
5. Other pieces required for safedisc support (privileged instruction emulation,
   and process startup sequence).
> d) Can I somehow access the most recent version of the patch? I have several
> versions now, but they keep on changing/improving(!). It would be nice if you
> could publish it somewhere (nightly CVS tarball?), so more people can provide
> feedback on it too.

I could make a current diff of what I have. But I don't really have time to make
a daily snapshots.
> As you can see, I'm just trying to see if I can contribute on such a big project.
> Thanks for the feedback,

Thank you for offering support in this area. I really do appreciate it.

Vitaliy