This is an often requested enhancement in the Feature Requests forum. For the time being, your only option is to change the privacy settings for your activities to something other than Everyone.
I have my privacy set to only me but I would love to hide the start and finish point "drop blobs" in my own display because they are way too BIG and hide whatever is on the map around my start and finish point. Why do they have to be so big and is there any way to change the size of the green and red drops?
You can't hide them as far as I know but if you zoom in on the map then the "blobs" stay the same size but the map features get bigger so you can see the detail round the start and finish. Hope that helps