I get this, but personally doesn't factor into my voting. Well, it can help, but not to the extreme it sounds like it does for you.as a producer as well when judging one of my beats or anyone elses one of the main factors i use is does this beat make me want to start writing or freestyling to it. if it does it gains major point twords being a good beat in my mind.
Reason being, tonnes of my favourite beats I don't feel that vibe for, whereas some beats that i don't consider on the same level, i just hear it and my body from some divine energy just starts making me spit. It's such a profound experience. But doesn't necessarily happen with my fave or what i consider better beats.
But similarly if a beat just makes me move, that also helps.
As for the vocals, i feel I've got a good knack for vocals to click with a beat that the combination gives that kinda feeling.