I think vocal fry is pretty annoying period, but let's assume it is gender specific. That doesn't mean having poor inflection is a female only problem. Read any public speaking guide or speak to a speech coach, fixing rising inflection is one of the first recommended fixes.

If anyone is interested, I found this book very useful: http://www.amazon.com/Its-Way-You-Say-Well-spoken/dp/1609947...