The performance difference is larger in Horizon Zero Dawn.
I concluded it's unsafe to use tag values stored in each object because we could load a value which becomes stale before the swap. A single tag value is more likely to overflow in 32-bit, but 32-bit use is likely very uncommon and the issue is rare enough that it's unlikely to be coincident with an overflow.