The downside to shooting in slow mo is that you always end up with more footage than you need. Six seconds of shooting produces one minute of video at 300fps, two minutes at 600fps. Some cameras let you trim your clip right there on the spot , as it frees up wasted memory.
Another issue about shooting in slow-motion is that to be able to get a camera that can shoot with a high frame rate can be quite expensive in some cases hard to get your hands on. If the camera that you are shooting with is not of a high enough quality you will end up with blurred photos and will then stubble to get good quality when ' time-streching' or ' cranking '.
One more problem with slow mo is sound, as in there is none. That's the reason you often hear music playing over clips. It's not necessary to add music, but if there is no sound it may be something will need to do. This will also then draw an issue to conversation. If conversation is filmed in slow motion it comes out really deep and then sounds bad and ruins the tone.
The final issue with using slow mo is gonna be filming using smooth movements and not have people looking jumpy , if the speed of movement is different through out this could effect the later viewing of the scene.