Why can't physical attraction be a criterion for women seeking men the way it is for men seeking women?

Geez, the guy doesn't need to be drop dead gorgeous but as an attractive woman, I find physical attraction to a man to be a mandate for a successful relationship. Unfortunately, the feedback I receive from both men and women is that this is shallow EVEN though it seems to be the first criterion to meet for most guys seeking women. I know that men and women don't have equal odds in the looks department but why the backlash for wanting to date an attractive man?