Vibration alerts when not wearing

Does anyone know if there is a setting to stop vibration alerts when you are not wearing your watch? If I take it off during the day and leave on my desk, or even whilst charging, it still alerts and is driving me mad! I know you can use do-not-disturb but I dont want to have to keep turning that on and off!

Thanks