MORE: Liverpool heading back to Oz next year? | Herd's hunger clinched Bolton loan deal
Brendan Rodgers' men, who have lost three of their opening five Premier League fixtures, were far from their best again as they drew 2-2 after extra time in a pulsating encounter at Anfield.
And the incredible action continued into the spot kicks, with some players required to take second penalties after 20 of the first 22 were converted.
A further seven would follow suit until Middlesbrough's Albert Adomah missed the target to secure the hosts' progress via a scarcely believable 14-13 scoreline.
It appeared a comfortable night was in the offing for Liverpool when teenage debutant Jordan Rossiter struck in the ninth minute.
But Middlesbrough were undeterred and enjoyed the better of proceedings thereafter, their reward coming midway through the second period when Adam Reach headed home to stun the home crowd.
Liverpool again took control during extra time, substitute Suso hitting his first goal of the club to seemingly seal their progression.
Patrick Bamford equalised from the spot in injury time of extra time following mistakes from Raheem Sterling and Kolo Toure, but Rodgers' side eventually booked their fourth-round berth in dramatic fashion.
Liverpool fielded something of an experimental starting XI, with the likes of Rossiter being complemented by Sterling, Adam Lallana and Lazar Markovic.
The hosts came out of the blocks with positivity, moving the ball quickly in the final third.
But it was Middlesbrough who had the first effort on target, ex-Liverpool defender Daniel Ayala heading straight at Simon Mignolet.
Any momentum gleaned from that was quickly extinguished when the hosts took the lead, however.
A dreadful mix-up between Kenneth Omeruo and debutant Jamal Blackman preceded the latter making a point-blank save to deny Rickie Lambert, who captained Liverpool on his first start for the club.
But Blackman's clearance fell to Rossiter, with the goalkeeper then somehow failing to keep out the 17-year-old's low 25-yard strike.
Middlesbrough responded well as Kike's goal-bound volley was deflected over by Lambert.
That heralded a spell of visiting pressure, Lee Tomlin forcing a solid save from Mignolet with a fizzing strike.
Picking up where they left off, Middlesbrough began strongly after the interval as Tomlin struck narrowly wide from distance.
Liverpool took control again well as they found a passing rhythm, although their final ball was leaving much to be desired.
Yet Aitor Karanka's team found a leveller in the 62nd minute, Reach capitalising on being unmarked to head Grant Leadbitter's free-kick home.
They continued to enjoy the better of the chances afterwards, too, with Mignolet having to be on high alert to keep out Adam Clayton's excellent strike.
And Clayton threatened again with five minutes remaining, hitting the outside of the post after being found by Tomlin's superb throughball.
An uneventful first half of extra time made way for chances afterwards, Blackman making solid saves to deny Jose Enrique and substitute Mario Balotelli in quick succession.
Suso appeared to have put Liverpool through, only for Bamford to score from the spot in the 123rd minute to force penalties.
He could not follow that up with Middlesbrough's first kick of the shootout, his saved strike one of only two missed in the first 27.
But despite Sterling missing the other when he had the chance to win it 5-4, Liverpool eventually sent their fans home happy.