The charity Mind says so and they want professional clubs, governing bodies and community organisations to do more. In recent years some high profile sportspeople have taken their lives, including Wales manager Gary Speed. A new report from Mind says elite athletes can be particularly at risk from severe anxiety and stress.
Former QPR footballer Clarke Carlisle knows all about it. He tried to take his own life when he was 21 when doctors told him he may never play football again.
"The margins between success and failure are so small, that's why elite sport dominates your life," he says.
"It becomes a person's identity. A person's work life and personal life merge into one and it's all based on their performance in their sport.
"Life was rosy, then I suffered a very serious knee injury.
"The prospect of having my entire career and livelihood taken away was the beginning of my downward spiral."
He adds: "I was housebound for two months, I was drinking heavily. I really could not process the prospect of a live without football.
"In my head, my self esteem, my worth, my value was linked to football."
He says he couldn't handle the thought of not returning to the pitch.
Although individual sports do offer support services, Mind are calling for a national network to be established to help all elite athletes.
Clarke, who is an ambassador for Mind, has a suggestion for how it should be funded.
"You take the money from the TV revenue, before it's given out to the football clubs," he says.
"You take out one section of money and that is solely for the mental health initiative that will cover everyone across all four divisions."
Mind says young athletes also suffer if they don't have the success they expected and have to find another job.
Clarke says although there is "a great appetite to address mental health issues within sport," the support on offer "is nowhere near adequate".
The Professional Cricketers Association (PCA) has told Newsbeat the number of cricketers seeking help for mental health related issues has "doubled year-on-year for the past three years".
It's not just the professionals either.
Paul Farmer, chief executive of Mind, tells Newsbeat: "So many of us want to achieve the best we possibly can.
"Maybe we've won a few competitions and someone says 'Why don't you have a try for your county?' and as that pressure builds, it can be really challenging on young people's mental health."
He says the pressure of exams only makes things worse.
Both Clarke Carlisle and Paul Farmer say coaches and managers need to take the issue much more seriously
"The best coaches are the ones who really understand the whole person that they're working with," says Farmer.
"If you are a great coach you can help them through the tough times to help them achieve great things."
Carlisle says in professional football "when it comes to an issue like mental health [players] don't want to jeopardise their opportunity to play at the weekend or get their contract extended."
He adds: "The perception of mental health is that it is a personal individual weakness and something that will make that person a liability. And that perception has to change."
"Until the managers have an awareness that a person's mental health is as important as their physical health nothing will change."
For advice and help regarding mental health you can contact Mind.