Assistant coaches Peter Horachek and Steve Spott will fill in temporarily, the team said.
Carlyle had gone 91-78-19 in three-plus seasons as the Leafs coach, including a play-off appearance in 2013, but was deservedly criticised for a series of late-season collapses and a style that led to his team getting drastically outshot on a regular basis.
That principle was in play again this season; the Leafs started the day in play-off position in the Eastern Conference but have gone 3-7 in their last 10 games.
"It’s never an easy decision to make when changing your leadership but our team was not trending in the right direction and we felt an immediate change was necessary," GM Dave Nonis said in a released statement.
Most importantly, the move gives Nonis, president Brendan Shanahan and the rest of Toronto's front office an opportunity to assess the players away from Carlyle.
Seven players, including Phil Kessel and Dion Phaneuf, will now play for their third coach.
Kessel is as productive as any winger in the league, so it's hard to view him as part of the problem, but guys like Phaneuf, Tyler Bozak and Joffrey Lupul are easier to imagine getting moved.
Nonis did not blame them for the decision in the ensuing press conference, but he certainly did not give a full endorsement, either.
“People think that the players are set in stone, I’ve said before — players are movable. None of them have full no-moves. That doesn’t change today," he said.
"It's not that we don't have players who can't do it. The game in Boston (a 4-3 win on Dec, 31) is a good example," Nonis said.
"It's that our consistency hasn't been there ... It's not that they're capable, because they are. It's not that they haven't done it, because they have."
The 64-year-old Carlyle won a Stanley Cup with the Ducks in 2007, which bolstered his reputation and bought him time, but Tuesday's news has felt inevitable.