It looks like on CI test VMs process_vm_readv returns EPERM for some reason (indicating it is not passing security checks). Normally that should not happen, process_vm_readv / process_vm_writev is guarded by the same permission checks as ptrace() attach, so if the former doesn't work the latter is not supposed to (and there is already 'prctl( 0x59616d61 /* PR_SET_PTRACER */, server_pid )' in server_init_process(). Yet CI machine is setup in some way which I so far could not guess so that process_vm_readv fails while ptrace attach succeeds. This is not the case here locally.
In v4 I changed the patches so EPERM from process_vm_readv / process_vm_writev triggers fallback to ptrace() same way as it does for ENOSYS. This way related tests pass on CI.