One of the downsides would be the use of "virtual torture". This would be millions of times worse than real-life torture.
What are you talking about?
Imagine a situation where you are plugged into VR and you just happen to check your messages in your virtual office - turns out that the message contains a very dangerous worm that instantly turns your office into a torture chamber where you are chained in a cold, metal chair. This same worm also subconsciously triggers your brain to prevent you from lifting your hands to take off the VR headset you're wearing. Also, it was unfortnate that you had to buy this expensive VR headset that allows you to feel pleasure or pain as you navigate the virtual scene.
And there is a shadowy figure in the chamber staring at a vast assortment of devices, trying to figure out which one to use on you that isn't even remotely possible in reality. Should I go on?
This situation will only happen if you want,you will be able to "sign out" of virtual reality any time you need or want,because the nanobots will plug out of your neurons only by your thoughts...