Why not leverage the pre-existing alternatives system that Ubuntu inherited from Debian?
It’s already a mature tool for selecting between different implementations of the same thing, and is already used by Ubuntu commands such as parallel which are present in both moreutils and parallel packages.
That way deleting the Rust package would automatically switch back to the GNU implementation without requiring the user to mess with backups, and making it impossible to leave the system without foundational tools at all as long as at least one implementation is installed.