At least on the Vivoactive 4S, since I wasn't able to test this out on other devices, there seems to be an undocumented / unwanted behavior that is not directly visible when testing in the emulator.
While developing Monky (https://github.com/vladmunteanu/monky), I had to adjust one of the minigames so that it checks for a key pressed event instead of a swipe right, because when the app ran on the actual watch, it would trigger the onKey event, instead of onSwipe. (https://github.com/vladmunteanu/monky/blob/37fae93329cd7875d9d3c05c9a64c6cffd4cee07/source/SwipeGame/SwipeGameDelegate.mc#L17)
If intended, the documentation should at least be changed to reflect this when implementing an InputDelegate.