Was there every any occasion prior to recent history, when you considered that he might be gay? Does he have reason to hide the fact he's gay if in fact he is? (i.e. intensely religious, expresses any intolerant opinion on LGBT?) The reason I ask, is I think most women would develop a 6th sense about this long before 3 years of married life. What you describe could also be attributed to someone who is straight. (Concerned with his physical appearance, etc.) Could his behavior toward your gay friends in any way be interpreted as flirtatious? If the relationship feels like it's lacking in other areas besides sex, have you considered counseling? You might try this - there is a theory that everyone falls somewhere within the categories of 100% straight to 100% gay - with many falling somewhere in between the 2 extremes. You might consider posing that question to him and see what his response is. If I were in the same situation and genuinely had no clue - I'd have to eventually ask him directly in as gentle a way as possible.
You could set him up and see if he takes the bait.
Besides that your choice is to ask; but, even then he could lie.