Using compiler error information to check undocumented prototypes and structures?

[Qian Hong]

Hi,

On Thu, Mar 5, 2015 at 7:27 PM, Piotr Caban  wrote:
> I'm expecting it to be quite hard taking in account the functions are not
> documented and that you can't look on the headers while working on it.

Could we use compiler error information to check undocumented
prototypes and structures?

For example, assume we don't know the prototype of CreateWindowExW.
Let's write a test program test.c like below:

#include <windows.h>

void CreateWindowExW();

int main(void)
{
    CreateWindowExW();
    return 0;
}

Then we compile test.c on Windows using:

gcc -sysroot /path/to/MicrosoftSDK/include test.c

Then we will get some error message like:

In file included from /path/to/windows.h:72:0,
                 from test.c:1:
/path/to/winuser.h:1935:26: note: previous declaration of
‘CreateWindowExW’ was here
   WINUSERAPI HWND WINAPI CreateWindowExW(DWORD dwExStyle,LPCWSTR
lpClassName,LPCWSTR lpWindowName,DWORD dwStyle,int X,int Y,int
nWidth,int nHeight,HWND hWndParent,HMENU hMenu,HINSTANCE
hInstance,LPVOID lpParam);

Then we will know the correct prototype.

In theory VC++ and clang should do similar job as well.

Could that be a valid way to detect a function prototype and a data
structure when  re-implementing undocumented API?

If it's valid, we can consider setting up a AppVeyor CI account and
connect to a github account to provide a "MS type and structure query
service". AppVeyor CI provides a free (as beer) Windows environment
and VC++ compiler (third party compilers like gcc and clang can be
installed as well). By tweaking the setting, we can trigger AppVeyor
rebuild by pull request. In that case, whenever we need to query a
undocumented prototype or structure, we can just write a (wrong) test
case and submit a pull request, without reading MS headers by human.

Any thought is appreciated.

[Dmitry Timoshkov]

Qian Hong  wrote:
> Then we compile test.c on Windows using:
>
> gcc -sysroot /path/to/MicrosoftSDK/include test.c
>
> Then we will get some error message like:
>
> In file included from /path/to/windows.h:72:0,
>                  from test.c:1:
> /path/to/winuser.h:1935:26: note: previous declaration of
> ‘CreateWindowExW’ was here
>    WINUSERAPI HWND WINAPI CreateWindowExW(DWORD dwExStyle,LPCWSTR
> lpClassName,LPCWSTR lpWindowName,DWORD dwStyle,int X,int Y,int
> nWidth,int nHeight,HWND hWndParent,HMENU hMenu,HINSTANCE
> hInstance,LPVOID lpParam);
>
> Then we will know the correct prototype.

An undocumented API doesn't have a prototype in a header file, once
there's a prototype in SDK it's no longer undocumented :)

[Piotr Caban]

Hi,

On 03/05/15 13:44, Qian Hong wrote:
> Could we use compiler error information to check undocumented
> prototypes and structures?

It's not helpful to use compiler here.

The problem is that person that implements it in wine can't see native 
implementation. And parts of it are in the headers. In this case the 
names mangling gives us function prototype. The problem are class size, 
virtual functions table, public class members.

Someone else can look on headers and document it for person that writes 
the implementation.

Thanks,
Piotr

[Qian Hong]

Thanks Dmitry and Piotr for comments.

You are write, compiler is not enough in this case.

On Fri, Mar 6, 2015 at 1:04 AM, Piotr Caban  wrote:
> The problem is that person that implements it in wine can't see native
> implementation. And parts of it are in the headers. In this case the names
> mangling gives us function prototype. The problem are class size, virtual
> functions table, public class members.
>
> Someone else can look on headers and document it for person that writes the
> implementation.

I have feeling that using libclang to write a customer parsing tool
can solve the problem in machine way rather than in human way.

I wrote a proof of concept using libclang pythong binding, see get_class.py.

Example:

$ python get_class.py test.cpp Room
Translation unit: test.cpp
class Room {
public:
    void add_person(Person)
    {
    }
    Person get_persons()
    {
    }
    int index;
}

find_classdecl firstly search for the class match the name we input,
then strips all methods and fields except public method and public
field, then strips the implementation details of the method, prints
only stub implementation of public methods, also prints the
declaration of public fields.

[Vincent Povirk]

> find_classdecl firstly search for the class match the name we input,
> then strips all methods and fields except public method and public
> field, then strips the implementation details of the method, prints
> only stub implementation of public methods, also prints the
> declaration of public fields.

At least in MFC headers, public fields are not always documented, and
many could be considered implementation details. It's not obvious
where that line should be drawn. And sometimes the headers contain
inline function implementations, which are effectively part of the
ABI.

For projects where "public" means things that are intended to be part
of the API, maybe this could work, but I'm not aware of any that we
need to replace.

Is this about any library in particular?

[Qian Hong]

On Sat, Mar 7, 2015 at 8:20 AM, Vincent Povirk  wrote:
> At least in MFC headers, public fields are not always documented, and
> many could be considered implementation details. It's not obvious
> where that line should be drawn. And sometimes the headers contain
> inline function implementations, which are effectively part of the
> ABI.

Good point, thanks a lot for the information.
> For projects where "public" means things that are intended to be part
> of the API, maybe this could work, but I'm not aware of any that we
> need to replace.
>
> Is this about any library in particular?

No, it is not for any particular library, I'm interesting in a general
solution to solve all undocumented situation including mfc and msvcp
and any other libraries. But it is not an urgent task, it's just
something like brainstorm. I was also wondering if writing such a
general tool could be consider a GSoC idea.