How is this controlled in toolkits like GTK? If they don't need an
environment variable then neither should Wine.

> For apps that want to customize the UI with a custom hook, I would either have them fallback to the current UI, or ignore their requests for custom hooks.  It may still be a better experience to use native FilesApp even without the customizations.

Well, the problem with the hook function in particular is that
applications may break without it. For the common item dialog, it
depends entirely on what the custom options do, this may represent a
loss of important functionality.

I could see providing an override if users know that it works in their
application, but I don't think that should be done by default.