There's many things to take into factor in ways such as handling the schedule, being good with your mouth   , taking the bumps, staying in shape, etc etc.

Which one do you think would be the hardest of them all?

I would say handling your schedule would be the hardest next to staying in shape.

The fact that the slightest mistaken bump could end your career and make you walk in a wheelchair for the rest of your life sends chills up and down my spine just thinking about it, I can't/don't wanna imagine how actual wrestlers feel about that.

Psychology

Getting over the fact that you're in one of the most shamed, made fun of professions there is. Making below minimum wage (maybe not even getting paid) while you put your body in extreme danger, and knowing that you probably won't get any fame or credit for it unless by some miracle you get signed by a top company.

#1.  The Travel.  When you're driving/flying all over the place and living out of hotels, it is absolutely miserable on the family and home life.  If you look at the rate of divorce, affairs, and drug addiction, it's not surprising.  Men who only get to see their wife for a couple of days a month can't lead to a very happy or healthy relationship.

#2.  The Risk.  Breaking it down to brass tacks, a pro-wrestler is essentially a high risk stunt-man, who gets to use no harnesses and little safety equipment, and must perform every stunt in front of a live audience and with only one try.  A single mistake can result in injury, and even without any "big" injuries (broken necks, backs, etc), even the little things take a toll on the body.  The reward for years of service is a fast-track to the unemployment line, with people even in their 30s being considered "old" by wrestling standards.

I once saw an x-ray where they showed Shawn Waltman (Syxx, X-Pac) in his neck.  Even before any big breaks or injuries, he was absolutely trashed with repeated accounts of whiplash just from being bounced agains the turnbuckles, or from jarring himself around while doing the bronco buster.  The sort of injuries we get from car accidents is the stuff that pro-wrestlers take on a daily basis from moves as simple as an Irish Whip.