i'd just like to mention a small concern i have. hiding symbols might well be a perfectly honest thing to want to do. i personally can't see why, since a list of symbol names doesn't provide much extra help to somebody who wants to disassemble and reverse engineer your code. but i can understand that there might be some circumstances where its desirable. the problem is while hiding symbols doesn't impede a disassembler/reverse engineering effort very much, it does impede attempts to demonstrate the inappropriate use of someone else's code. for this reason, i personally wouldn't feel comfortable asking in a public forum how to go about doing this. your mileage may and probably does vary, of course. --p
"Paul Davis" <paul(a)linuxaudiosystems.com> wrote:
i'd just like to mention a small concern i have.
hiding symbols might well be a perfectly honest thing to want to do. i personally can't see why, since a list of symbol names doesn't provide much extra help to somebody who wants to disassemble and reverse engineer your code. but i can understand that there might be some circumstances where its desirable.
the problem is while hiding symbols doesn't impede a disassembler/reverse engineering effort very much, it does impede attempts to demonstrate the inappropriate use of someone else's code.
for this reason, i personally wouldn't feel comfortable asking in a public forum how to go about doing this. your mileage may and probably does vary, of course.
Well, if I could clearly predict ELF linker behaviour with several different modules each having global variables/functions with the same names I would not bother too much, but I couldn't. Could you? -- Dmitry.
for this reason, i personally wouldn't feel comfortable asking in a public forum how to go about doing this. your mileage may and probably does vary, of course.
Well, if I could clearly predict ELF linker behaviour with several different modules each having global variables/functions with the same names I would not bother too much, but I couldn't. Could you?
No, but if that was the reason I had for wanting to hide symbols, I would probably say so up front, because its a pretty good one :) --p
"Paul Davis" <paul(a)linuxaudiosystems.com> wrote:
for this reason, i personally wouldn't feel comfortable asking in a public forum how to go about doing this. your mileage may and probably does vary, of course.
Well, if I could clearly predict ELF linker behaviour with several different modules each having global variables/functions with the same names I would not bother too much, but I couldn't. Could you?
No, but if that was the reason I had for wanting to hide symbols, I would probably say so up front, because its a pretty good one :)
Also, I personally feel a bit not comfortable when someone (for any reason) is looking into a binary and sees names of internal interfaces not appropriate for external learn/use. I find it ugly that I can't fully control such basic things in the ELF world. -- Dmitry.
On Fri, 14 May 2004 13:29:48 +0900, Dmitry Timoshkov wrote:
Also, I personally feel a bit not comfortable when someone (for any reason) is looking into a binary and sees names of internal interfaces not appropriate for external learn/use. I find it ugly that I can't fully control such basic things in the ELF world.
IMHO it's cleaner to do what the glibc team do and simply mark them as private using symbol versioning. Yes, they still appear in the symtab, but when you read it at least you see: __libc_foobar@@GLIBC_PRIVATE and there can be no doubt whatsoever that it's internal. That also lets you mark things as internal and still use them from other DSOs. ELF linker behaviour when there are multiple symbols with the same name is defined by the way, it's just a dumb behaviour :) Basically they all get linked to the first symbol that was loaded unless you have things like -Bsymbolic set (though that only works if the symbol being linked is in the same object). The correct solution to making ELF not suck is to implement support for the DT_1_GROUP flag so we can get grouped fixed, like on Solaris. This is closer to the Windows model that people intuitively expect. It should also help with some of the pathological slowness that prompted initiatives like prelink. thanks -mike
"Mike Hearn" <mh(a)codeweavers.com> wrote:
Also, I personally feel a bit not comfortable when someone (for any reason) is looking into a binary and sees names of internal interfaces not appropriate for external learn/use. I find it ugly that I can't fully control such basic things in the ELF world.
IMHO it's cleaner to do what the glibc team do and simply mark them as private using symbol versioning. Yes, they still appear in the symtab, but when you read it at least you see:
__libc_foobar@@GLIBC_PRIVATE
and there can be no doubt whatsoever that it's internal. That also lets you mark things as internal and still use them from other DSOs.
It's even uglier. Instead of resolving the problem that guys decided just to hide it.
ELF linker behaviour when there are multiple symbols with the same name is defined by the way, it's just a dumb behaviour :) Basically they all get linked to the first symbol that was loaded unless you have things like -Bsymbolic set (though that only works if the symbol being linked is in the same object).
The only thing I can say is that yes, it's really dumb.
The correct solution to making ELF not suck is to implement support for the DT_1_GROUP flag so we can get grouped fixed, like on Solaris. This is closer to the Windows model that people intuitively expect. It should also help with some of the pathological slowness that prompted initiatives like prelink.
Sigh, I still think that ELF is inherently sick, and inventing new (intrusive) workarounds makes it even worse. -- Dmitry.
On Fri, 2004-05-14 at 21:33 +0900, Dmitry Timoshkov wrote:
It's even uglier. Instead of resolving the problem that guys decided just to hide it.
It's not necessarily bad. The information being there may be ugly, but it *can* be useful. If we didn't have that, we'd be unable to use the pthread overrides trick for non-NPTL systems.
Sigh, I still think that ELF is inherently sick, and inventing new (intrusive) workarounds makes it even worse.
Maybe. But let's face it, the alternatives aren't much better. The only real competition is PE (Windows) which is worse than ELF in many ways, and Mach-O (MacOS/X) which is so appallingly primitive it just blows my mind that a shipping OS actually uses it. Given the alternatives, we could have a lot worse than ELF!
"Mike Hearn" <mh(a)codeweavers.com> wrote:
Sigh, I still think that ELF is inherently sick, and inventing new (intrusive) workarounds makes it even worse.
Maybe. But let's face it, the alternatives aren't much better. The only real competition is PE (Windows) which is worse than ELF in many ways,
Can you list at least some things you think are bad in PE in comparison with ELF? Another downside in ELF in addition to mentioned already is missing a resource section, which makes our life implementing Wine much harder and forces all application developers invent their own formats to store icon/bitmap/dialog info/strings/etc. information. -- Dmitry.
On Fri, 2004-05-14 at 21:53 +0900, Dmitry Timoshkov wrote:
Can you list at least some things you think are bad in PE in comparison with ELF?
Sure - ELF has decent versioning, which PE lacks and has caused massive pain due to conflicts etc. While you can rename DLLs on Windows it's not a part of the system so not many people do, and when they do there's no pattern to it. The ELF file format is fairly simple compared to PE where you have layers of wrappers and cruft due to its long history. ELF is well specified for multiple CPU architectures and standard between several different operating systems. I can get the spec in many formats. The PE spec in contrast comes in the form of a Word document, and I have to agree to a bigass EULA to read it: http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx ELF is pretty easy to extend with new sections and flags.
Another downside in ELF in addition to mentioned already is missing a resource section, which makes our life implementing Wine much harder and forces all application developers invent their own formats to store icon/bitmap/dialog info/strings/etc. information.
Actually I see that as a benefit: PE resource files are IMHO a rather dubious feature. Yes it makes implementing Wine harder, but any time I see a filing-system-in-a-file I get suspicious. Sticking files inside the EXE or DLL just makes them harder to get at, harder to replace or edit: we already have a working filing system so reinventing that in a limited fashion seems silly. I think keeping code and data physically separate is a much better plan. In the rare case that you need to store binaries inside ELF it's still possible: see GTK+ which does this for its stock artwork. Finally, the linking semantics of ELF are as much a blessing as a curse. While it can cause chaos sometimes, it also lets us do things like the pthread override trick. Any LD_PRELOAD hack also uses this "feature", though it could be done more cleanly with an extension. Modern (ie GNU) ELF also has various interesting features like symbol versioning, prelinking etc ...
"Mike Hearn" <mh(a)codeweavers.com> wrote:
Can you list at least some things you think are bad in PE in comparison with ELF?
Sure - ELF has decent versioning, which PE lacks and has caused massive pain due to conflicts etc. While you can rename DLLs on Windows it's not a part of the system so not many people do, and when they do there's no pattern to it.
ELF versioning is just another attempt to hide problems instead of fixing the source of it IMHO. Exactly for this reason even simple 'Hello World' program compiled under RH 9.0 will not run under RH 6.2.
The ELF file format is fairly simple compared to PE where you have layers of wrappers and cruft due to its long history.
Actually PE format hasn't changed since NT 3.1 introduction.
ELF is well specified for multiple CPU architectures and standard between several different operating systems. I can get the spec in many formats. The PE spec in contrast comes in the form of a Word document, and I have to agree to a bigass EULA to read it:
http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx
PE is documented very well, and there is no need to download its description from microsoft site, find it somewhere else.
ELF is pretty easy to extend with new sections and flags.
If you call such a basic thing as having separate debug info as a yet another GNU extension invented only very recently, more than 10 years later than in PE world, then I'd agree.
Another downside in ELF in addition to mentioned already is missing a resource section, which makes our life implementing Wine much harder and forces all application developers invent their own formats to store icon/bitmap/dialog info/strings/etc. information.
Actually I see that as a benefit: PE resource files are IMHO a rather dubious feature. Yes it makes implementing Wine harder, but any time I see a filing-system-in-a-file I get suspicious. Sticking files inside the EXE or DLL just makes them harder to get at, harder to replace or edit: we already have a working filing system so reinventing that in a limited fashion seems silly. I think keeping code and data physically separate is a much better plan.
In the rare case that you need to store binaries inside ELF it's still possible: see GTK+ which does this for its stock artwork.
Finally, the linking semantics of ELF are as much a blessing as a curse. While it can cause chaos sometimes, it also lets us do things like the pthread override trick. Any LD_PRELOAD hack also uses this "feature", though it could be done more cleanly with an extension.
Modern (ie GNU) ELF also has various interesting features like symbol versioning, prelinking etc ...
An executable file format is such a basic thing in which a developer would expect: 1. be relatively simple and straitforward 2. be well documented 3. be stable (i.e. for instance not change for more than 10 years as PE) 4. be well supported on the OS system level API (as you can do with PE: find a PE header in memory, find import/export table, intercept an API using simple code patch and guarantee that such a program will work on all next version of the OS) 5. let a developer create a custom section and easily (I say easily!) access it at run-time. I find nothing of the above is true for ELF, sorry. -- Dmitry.
On Fri, 2004-05-14 at 23:58 +0900, Dmitry Timoshkov wrote:
ELF versioning is just another attempt to hide problems instead of fixing the source of it IMHO. Exactly for this reason even simple 'Hello World' program compiled under RH 9.0 will not run under RH 6.2.
What exactly are the problems? As for running RH9 binaries on RH6.2, there are two issues: 1) Using newer libraries that aren't available on the older distro. You can statically link these, ship them with the app etc - exactly the same as on Windows. 2) glibc symbol versions. You can select the versions you want at compile time though most people don't realise it. For instance, the apbuild tool will do this for you. You miss out of a few features like thread-scoped locales, but really it's not a big deal for desktop apps. So yes you can do a simple hello world program that runs on 6.2, you just have to be careful about it.
Actually PE format hasn't changed since NT 3.1 introduction.
Unless you count .NET or the GetProcAddress PLT-style jumptables (i think they were introduced in later versions of Visual C++, not even an OS feature!) ...
4. be well supported on the OS system level API (as you can do with PE: find a PE header in memory, find import/export table, intercept an API using simple code patch and guarantee that such a program will work on all next version of the OS)
Well, this is certainly possible, you can traverse _DYNAMIC to find the headers of the file you want then patch the GOT, but to intercept APIs overriding them with LD_PRELOAD is usually easier. This part of ELF is stable and has not changed for a long time. The parts that have changed are mostly for new technologies like prelink, execshield etc ...
5. let a developer create a custom section and easily (I say easily!) access it at run-time.
OK, I'll give you that one, you have to use a binary array compiled in. But we don't seem to suffer from its lack in practice. I guess we'll have to agree to disagree on this one :)
Dmitry Timoshkov wrote:
Also, I personally feel a bit not comfortable when someone (for any reason) is looking into a binary and sees names of internal interfaces not appropriate for external learn/use. I find it ugly that I can't fully control such basic things in the ELF world.
What is wrong with seeing names of internal interfaces? If someone wants to use a lib you publish, they just look in the .h file to find out how to interface with it. They won't be looking in the binary file to find out how to use it. I expect that one of the reasons why one can't control "such basic things in the ELF world" is that no-one has really asked for that feature yet. Most software in the linux world is open source, so why would anyone ever want to hide stuff? If you don't want to port your application to the open source world, then, probably the best thing for you to do would be to make sure that your MS Windows application runs well in wine. Wine does not support all MS Windows features yet, and some it does support are slow. So, if you just check that your program does not use any of the un-implemented win32 functions, your application should work well. If you have to use some win32 feature not yet implemented in wine, then you might have to implement it in wine and submit a patch to wine. There are some well know applications out there that work on both windows and linux natively. E.g. The game Quake. They have never complained about the .ELF format. Cheers James
On Fri, 14 May 2004 14:24:51 +0100 James Courtier-Dutton <James(a)superbug.demon.co.uk> wrote:
What is wrong with seeing names of internal interfaces?
The main problem is the likelihood of symbol collisions between internal symbols which are the same in two different libraries. By restricting the exported symbols to just the public interfaces you reduce the chances of this.
If someone wants to use a lib you publish, they just look in the .h file to find out how to interface with it. They won't be looking in the binary file to find out how to use it.
I expect that one of the reasons why one can't control "such basic things in the ELF world"
Controlling which symbols are exported from a lib is just as easy with ELF as it is in windows. I do it with both of my two libraries: http://www.mega-nerd.com/libsndfile/ http://www.mega-nerd.com/SRC/ Exported symbols are limited on Linux, MacOSX and Win32. Erik -- +-----------------------------------------------------------+ Erik de Castro Lopo nospam(a)mega-nerd.com (Yes it's valid) +-----------------------------------------------------------+ A good debugger is no substitue for a good test suite. -- -- +-----------------------------------------------------------+ Erik de Castro Lopo nospam(a)mega-nerd.com (Yes it's valid) +-----------------------------------------------------------+ A good debugger is no substitue for a good test suite.
Erik de Castro Lopo wrote:
On Fri, 14 May 2004 14:24:51 +0100 James Courtier-Dutton <James(a)superbug.demon.co.uk> wrote:
What is wrong with seeing names of internal interfaces?
The main problem is the likelihood of symbol collisions between internal symbols which are the same in two different libraries.
By restricting the exported symbols to just the public interfaces you reduce the chances of this.
I would really like the linux program loader to at least warn us when duplicate symbols are being used. But currently, all we get is some strangely performing application with a very hard to track down bug.
If someone wants to use a lib you publish, they just look in the .h file to find out how to interface with it. They won't be looking in the binary file to find out how to use it.
I expect that one of the reasons why one can't control "such basic things in the ELF world"
Controlling which symbols are exported from a lib is just as easy with ELF as it is in windows. I do it with both of my two libraries:
I know that, I write libs for the open source community. I think what this person was saying is from the point of view of looking at the binary file itself. But even after filtering the symbols, they will then not be published externally, but can still be listed if one views the binary directly.
http://www.mega-nerd.com/libsndfile/ http://www.mega-nerd.com/SRC/
Exported symbols are limited on Linux, MacOSX and Win32.
Erik
participants (5)
-
Dmitry Timoshkov -
Erik de Castro Lopo -
James Courtier-Dutton -
Mike Hearn -
Paul Davis