I created a new list for the resolved and used topologies, when the media session shuts down it releases all of the topologies in there, when a topology is removed from the topologies queue it will just be removed from that list.
There is still a bug which is already in the tests, where the topology id of the resolved topology will not match the original topology (when not using the no resolve flag), this is because of a bug with the topo loader where it creates a new topology object with a different ID, in Windows apparently it creates a new topology object but it contains the same ID as the not resolved topology.