[PATCH v4 5/6] server: Track unbound input signaled state based on its original console.