A man might have one ‘mistress’ or even several. However, there is no male version of this word in English. The word ‘mistress’ sometimes carries negative connotations, but it also suggests a more formal or longterm relationship than a casual one. In any case, having a mistress implies a relationship with a woman, who is not married to the man.
So given that there is no male version of a mistress , how does a woman refer to a man with whom she is having a relationship out of wedlock?
The word ‘partner’ is sometimes used. So, is ‘boyfriend’, ‘lover’, ‘guy’, ‘common law husband’, or even ‘paramour’. None of these words carry exactly the same interpretations as ‘mistress’.
To me, boyfriend suggests young love. When I was practising as a dentist, some quite elderly women used to mention their ‘boyfriends’, who were often at least as old as they were. To my ears, ‘boyfriend’ sounded wrong in these cases. ‘Partner’ seemed a better choice of word.
Years ago, a friend of my father, the late Cyril Sofer, used to refer to girls’ close male friends as their ‘chaps’. Although I like this term, it is still not a male version of the word ‘mistress’ with all of its implications.
The Urban dictionary suggests ‘Manstress’, but that is not only an obviously made-up word it is also ugly,. And .linguistically nonsense. So you are right, Adam.