Totally bogus and inaccurate sleep scores!

Overall, with the Garmins I have owned, the sleep tracking (wake and sleep times) are pretty on point. However, for some reason with Polar it also gets messed up, even though Polar is an industry standard for sleeping. So, gadget reacts different to people and it might not be suited for everyone. One reason I like Garmin, is that the way it tracks sleep for me. However for you it might be hot garbage.

One thing to try if you haven't already:

What is your sleep schedule? Did you set it to 8pm bedtime? It might be getting confused with the early bedtime, altough I never had this issue.

Maybe your high stress, makes it think you are still awake? 

Does the issue come with an elevated heart rate? 

Also, you can manually correct the sleep after the fact. Idk, if the algorhythm picks up on it if you are changing a lot manually. 

The one thing I liked about Polar, is that if you turned off overnight heart rate it would just measure basic sleep and wake and sleep times. 

With Garmin can't do that AFAIK, and with so many metrics being tied into sleep, I can understand it is frustrating for you.

My normal sleep time is set for 9PM. My Heart Rate is not high, around 60 BPM which is excellent for a late 70 YO male. I'm not bothering to try and correct it as it's easier to just ignore what it says. Yes, Garmin claims I have high stress, which I don't. I've been retired for 22 years and am very happy and relaxed. Get lots of exercise too, over 1,000 minutes/week.

Interestingly before I switched to Garmin (Venu 3, now 4) my sleep scores from Fitbit and Samsung were all in the 80 range. Now with Garmin they're in the 40s!

I switched from Fitbit to Samsung when Google bought Fitbit and started sending it downhill.

I switched from Samsung to Garmin as it wasn't tracking my workouts properly. 

Except for the bogus sleep scores, I pretty happy with Garmin. 

Maybe try setting the sleep time to 7pm? then it will already start looking for it maybe around 8pm?

Where now it may be activating too late (if with the false stress readings for example it is already tricky)

Yeah, I feel the same about Polar lolz

Right now I am using Fitbit for Sleep and Garmin/Polar for sports

Also, you could try the Garmin Sleep Index monitor maybe, it sits on the upper arm. My experience has been good with it tracking wise. Altough it still has bugs and stuff, could help uou see if wearing on your upper arm helps. It uses the same V5 sensor as the Venu 4, so that can't make a lot of difference maybe.

Are you putting the watch face in sleep mode, and touch screen off?  (if possible) idk just thinking

Maybe put Do Not Disturb on too

Those steps might help convince the Venu 4 that you are sleeping. At the same time you might need something else to measure sleep (and then import it, in some neutral app, like Runalyze maybe)

Does that help a bit ?