Talk: Minimal Protonification - Slides

[Arek Hiler]

Link: https://hiler.eu/presentations/minimal-protonification.txt

also inline below

▙▗▌▗    ▗        ▜
                              ▌▘▌▄ ▛▀▖▄ ▛▚▀▖▝▀▖▐
                              ▌ ▌▐ ▌ ▌▐ ▌▐ ▌▞▀▌▐
                              ▘ ▘▀▘▘ ▘▀▘▘▝ ▘▝▀▘ ▘
                   ▛▀▖      ▐        ▗ ▗▀▖▗       ▐  ▗
                   ▙▄▘▙▀▖▞▀▖▜▀ ▞▀▖▛▀▖▄ ▐  ▄ ▞▀▖▝▀▖▜▀ ▄ ▞▀▖▛▀▖
                   ▌  ▌  ▌ ▌▐ ▖▌ ▌▌ ▌▐ ▜▀ ▐ ▌ ▖▞▀▌▐ ▖▐ ▌ ▌▌ ▌
                   ▘  ▘  ▝▀  ▀ ▝▀ ▘ ▘▀▘▐  ▀▘▝▀ ▝▀▘ ▀ ▀▘▝▀ ▘ ▘

what makes Proton a Proton

by ivyl

# What Is Proton?

Proton is a downstream Wine distribution that includes a lot of extras:

- **dxvk** - alternative D3D8-D3D11 implementation to wined3d
 - **vkd3d-proton** - alternative D3D12 implementation to vkd3d
 - **dxvk-nvapi** - Nvidia's nvapi implementation
 - **fonts** patched for metric compatibility
 - **piper** - speech synthesis
 - **kaldi** - speech recognition
 - ...

And some custom bits:

- **steamapi bridge**
 - **steam.exe shim**
 - **proton** launch script
 - tones of **patches** for game compatibility

Proton is intended to run inside of the **Steam Runtime**.

# What Would A Minimal "Proton" Be?

- runs at least some games
 - access to Steam API
 - launchable via Steam
 - as close to upstream Wine
 - no Proton build system involved
 - no runtimes involved

Let start stripping things down!

# Start Fresh

We are dropping all the external libraries and just start with a clean
`wine-10.15` tree.

We are going to bring back only the bare mnimum.

# Steam.exe Shim?

- lives in `steam_helper/steam.c`
- built using modified `makedep`
- some games expect to be descendants of a `steam.exe`
- sets registry values relatied to currently running steam instance
- calls lsteamclient's `steamclient_init_registry()`
- setups vrclient
- waits to be ptraced when is set `PROTON_WAIT_ATTACH=1`

Most games don't need it, but significant number does.

You definitely don't need it when running software without Steam integration.

I was on the fence but **we are dropping it for our miniminal example.**

# lsteamclient.dll

- `lsteamclient/`
- built using modified `makedep`
- exposes win32 version of Steamworks API (`steamclient*.dll`)
- we have a Wine patch to set up trampolines from the original DLL
- calls into the `steamclient.so`
- code mostly autogenerated using Steamworks SDK headers + libclang
- does some extra work like converting paths unix <-> dos, etc.

You don't need it when running software without Steam integration.
Some games are also fail gracefully and are fine without it.

**We'll include this!** We can import it into `dlls/` and add the new directory
to the `configure.ac` file.

# Wine Patches

Wine has hundreds of patches that make games run or run better - adding hacks,
dll overrides, dlls that are part of driver installation on Windows, etc.

**Only 4 patches are needed** to have a working 'Wine tree'-based Proton build!

I've tried to group the patches in a way that makes for easy cherry-picking on
top of any Wine tree to get a rebase running quickly. They are directly on top
of the upstream `wine-10.0` tag.

I'll keep on iterating on the patch organization each rebase to make the changes
more self contained and make what's essential clearer.

# The Cherry Picks!

Necessary for building lsteamclient.dll:

1d400e3aea5c ("makedep: Allow building modules with C++ sources.")

Redirecting `steamapi*.dll`s to lsteamclient:

38e8f39bda73 ("HACK: steam: ntdll: Setup steamclient trampolines to
                   lsteamclient.")

Misc Steam ingegration:

912e1b4b9fe9 ("HACK: steam: kernelbase: Substitute the current pid for the
                   Steam client pid.")
    447ecf2a3333 ("HACK: steam: wine.inf: Add required Steam registry entries.")

# The `proton` Script

A complicated Python script that:

- `WINEPREFIX=$STEAM_COMPAT_DATA_PATH/pfx`
- reflinks/links/copies `default_pfx` to speed up the first start
- ... and help with per-game prefix space usage
- copies some dlls provided by Steam to correct directories
- contains special handling for prefix upgrades / downgrades
- copies some files provided by Steam that the games may expect
- applies workarounds based on `SteamAppId`
- starts the actual target via `steam.exe`
- redirects logging, etc.

**We'll be replacing it.**

# The Replacement

`proton waitforexitandrun regedit.exe`

```sh
#!/bin/sh

export WINEPREFIX="$STEAM_COMPAT_DATA_PATH/pfx"

mkdir -p "$STEAM_COMPAT_DATA_PATH/pfx/drive_c/Program Files (x86)/Steam/"
for file in steamclient.dll steamclient64.dll; do
    cp "$STEAM_COMPAT_CLIENT_INSTALL_PATH/$file" \
       "$STEAM_COMPAT_DATA_PATH/pfx/drive_c/Program Files (x86)/Steam/"
done

if [ "$1" != "waitforexitandrun" ]; then
    echo "Unknown verb $1."
    exit 1
fi
shift

dir="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"

"$dir/wine/wine" "$@"
```

# Steam Runtime

Proton is built to run inside of the Steam Runtime.

Steam uses `pressure-vessel` (modified bubblewrap + extra tooling) to create a
namespace for the runtime and run things inside of it - think Flatpak-lite.

Pressure-vessel also **imports hosts graphics stack** with dependencies and
checks all the imported host libraries to meet the **ABI guarantees**.

There's a community project to allow running Proton inside of the runtime using
other launchers: https://www.openwinecomponents.org/

Proton is built inside Proton SDK OCI image based on the Steam Runtime SDK:
https://github.com/ValveSoftware/Proton/tree/proton_10.0/docker

# Why A Runtime?

It's designed to benefit Steam, native games and anything else pre-compiled
people may want to redistribute on the platform.

Design docs, host distribution assumptions, debugging instructions, etc.:
https://gitlab.steamos.cloud/steamrt/steam-runtime-tools/-/tree/main/docs
https://github.com/ValveSoftware/steam-runtime/tree/HEAD/doc

**For Proton** the biggest benefit was running in a more predictable
environment.

Once we switched away from the old `LD_LIBRARY_PATH`-style a whole class of bugs
reports was gone - **ABI incompatibility** due to distribution patching
libraries, compiling with unexpected flags causing missing exports or files
being located in unexpected places.

Linux user-space is not really great at ABI compatibility and requires extra
steps added somewhere.

# Compatibility Tool Metadata And Layering

We'll need `toolmanifest.vdf` to tell Steam we are a Proton.

The default file looks like this:

```
"manifest"
{
  "version" "2"
  "commandline" "/proton %verb%"
  "require_tool_appid" "1628350"
  "use_sessions" "1"
  "compatmanager_layer_name" "proton"
}
```

Note the `require_tool_app_id`.

# Is Steam Runtime Mandatory?

For good experience and getting help? **Yes.**

For tinkering and trying things out? No.

Just remove `require_tool_app_id` and restart Steam.
Providing all the dependencies on the host is on you. Good luck!

# What About Escaping The Runtime Steam Itself Uses?

We really don't need to escape it, everything should run fine at this point.

You still can do this though:

`STEAM_RUNTIME=0 STEAM_RUNTIME_HEAVY=0 steam`

This requires much more 32 bit libs on host and may be broken in unexpected
ways. Extremely not supported. May disappear at any time.

# The Last Step

We also need to add a `compatibilitytool.vdf` next to our `proton` replacement.

```
"compatibilitytools"
{
  "compat_tools"
  {
    "miniproton"
    {
      "install_path" "."

"display_name" "miniproton"

"from_oslist"  "windows"
      "to_oslist"    "linux"
    }
  }
}
```

An put everything in `~/.steam/root/compatibilitytools.d/`

# By Those Powers Combined

wine-10.15 + a dozen or so patches + lstemclient dll included in the tree.

https://gitlab.winehq.org/ivyl/wine/-/tree/miniproton-wine-10.15

Check `miniproton/README` for instructions on how to run it.

**DEMO IF TIME ALLOWS**