fix: remove toolchain file to fix msrv-check on CI (#750)

Also, this makes it easier to let people build with their preferred version, now that the msrv
is registered in each crate rustc will refuse to build crates that are not compatible, I think.

Otherwise it will download a different version of the compiler right before building it