Voice notifications only from BT

Nowadays if you enable voice notifications for workouts, they send to your BT headset. But if you haven't paired any BT headset and you have paired the phone, voice notifications are sent to the phone.

Any way to configure voice notifications only to the headset? Disabling when no headset is paired?

Thanks!