Over the years, many coaches have said the same thing.
When things are going well for their side, the players get the credit.
But when things are going wrong, the first people in the firing line are those in the management team.
So spare a thought for the rugby defence coach.
It is not a title that someone takes on lightly.
Because if an evisceration of the sort Saints suffered against Saracens on the opening day of the season occurs, one man will step firmly into the spotlight.
That man was Mark Hopley, the Saints defence coach, who saw his side ship seven first-half tries and two in the second period on the way to a 55-24 defeat.
It was a sobering start to the new season and had some supporters on social media questioning Saints' defensive structure - or lack of it.
But seven days later, Saints bounced back in style, ensuring Leicester Tigers barely got a sniff at Franklin's Gardens.
Only a moment of magic from Jonny May helped Tigers penetrate the home defence and even then it was only enough for a 24-11 defeat for the outfit from Welford Road.
Six days later, Bath didn't even manage one score as Saints shut the door to secure an impressive 24-6 success and end the opposition's winning start to the season.
The players were rightly praised for bringing added intensity and desire, and Hopley and his fellow coaches stepped back out of the spotlight.
Such is life at the helm of a top professional team.
But Hopley isn't complaining - instead he recognises it is all part of the job and continues to focus on his current role.
"Obviously it's hard sometimes not to take things personally," said the former back row forward, who played 50 games for Saints before joining the Academy coaching staff.
"The result against Saracens wasn't the best in terms of nine tries conceded but we've worked really hard over the last few weeks, as we have all pre-season and I think we are getting better and better as we go.
"Even in the victory against Bath last Friday, there's still always things we can work on.
"It's about picking out positives and the things that have been really positive for me have been the desire and work rate.
"We've highlighted work rate in team meetings and it's been massive.
"Our players have grown from making big hits in games and hopefully that goes into the way we've played."
Teams will only manage to meet expectations if their coaches manage to coax the necessary amount of determination out of the players.
Saints lacked that physical edge against Saracens, but they rediscovered it against Tigers and Bath.
And now they will aim to take it to the Madejski Stadium for Sunday's Aviva Premiership clash with London Irish.
"It's about a mindset and we found out in the first game that if you're slightly off, a quality team will punish you," Hopley said.
"It's about having that focus every time you come out.
"It's got to be non-negotiable and I think we've done that over the past couple of weeks, but going away to London Irish this weekend is going to be tough.
"They're a very physical team and they showed against Harlequins (Irish beat Quins on the opening day of the season) that on their day they have been very competitive.
"We need to be ready going to the Madejski Stadium.
"Those two games at home were brilliant but away from home we need to front up.
"Away from home, solid defence wins you games and that's what we need.
"We don't need to be too elaborate away from home. We need a very good kicking game on the back of a good defensive effort and hopefully that will be enough."
Hopley was promoted to the position of first-team defence coach last season, after cutting his teeth in an Academy role.
Alan Dickens switched to attack coach, with Saints reaping the rewards in that area of the game.
Hopley has taken time to put his stamp on things, but he has been working tirelessly to do so.
And Saints have reaped the benefits during the past two games.
"The Academy role was development-based but since I've moved up to the first-team role it's very much performance-based and results driven," Hopley explained.
"The most important thing for us is that we win at the weekend.
"In terms of hours, you work extremely hard in the Academy, extremely anti-social hours, the days are very long - you work evenings a few nights a week.
"I've always had responsibilities with the Wanderers so some weeks I'd be doing Under-18s, second team and first team, but now I've got the sole focus of working with the senior squad.
"It's a lot more focused, but not necessarily any easier."
The presence of former Academy mentor Hopley will provide reassurance for the youngsters who have made the step up into the first team in recent years.
And the 33-year-old is delighted to see the recent progress of players such as Tom Collins and Harry Mallinder.
"It's brilliant," Hopley said.
"My first year in the Academy someone like Tom Collins was 16 and I've seen him go each step of the way.
"It's really pleasing for me to see how he's developed over the years into a fine player.
"He's started the season on fire and for us as coaches it's about making sure he's got that support and challenging him to get better every week."
And it is not just the youngsters who are learning all the time.
"As a coach you're always looking to get better and to learn," Hopley said.
"We've had people in for me to learn from and someone like (England defence coach) Paul Gustard has been very good.
"But also going away in the summer and going to (rugby league side) Warrington Wolves was very good for me to get snippets here and there.
"What you do is you take the bits you like and you form that into your philosophy."
Saints will hope that philosophy pays off again at the Madejski Stadium on Sunday.