Leicester's record signing headed home in the 22nd minute to immediately cancel out Alexis Sanchez's first league goal for Arsenal since joining from Barcelona.
Ulloa's intervention proved enough to earn a valuable point as the visitors failed to find a winner in an entertaining encounter at the King Power Stadium.
Arsenal headed into the game desperate to cap a good week, having qualified for the UEFA Champions League group phase for the 17th successive season on Wednesday.
Yet an inspired Ulloa proved a handful and, with Leicester also defending doggedly, Arsene Wenger's men had to settle for a share of the spoils.
Arsenal have now collected a win and two draws from the opening three league fixtures of 2014-15, with Leicester taking two points from the same number of games.
The home side began confidently, Ritchie De Laet sending in a teasing cross for Ulloa - whose attempt to control the ball flew straight to Wojciech Szczesny.
Arsenal found themselves briefly depleted when Laurent Koscielny was forced to go to the dressing room for stitches to a head wound following a nasty collision with Jeffrey Schlupp.
And Leicester almost made full use of their one-man advantage when Riyad Mahrez hit just wide following a fine one-two with Ulloa.
But, soon after Koscielny returned to the field, Arsenal took the lead through Sanchez after 20 minutes.
Santi Cazorla's wonderful lobbed pass freed Yaya Sanogo and, after the striker's woeful effort was blocked by Kasper Schmeichel, the Chilean was left with a simple finish.
Yet Leicester found an immediate riposte through Ulloa, who met Schlupp's stunning cross with a powerful header that gave Szczesny no chance.
Koscielny, still suffering from the effects of his earlier injury, was at fault for the goal and it came as no surprise when Calum Chambers replaced him.
Arsenal continued to dominate possession after half-time, but were fortunate not fall behind when Ulloa chipped into the side-netting after brilliantly turning Chambers.
The visitors fashioned a couple of chances of their own almost immediately after, Aaron Ramsey heading tamely at Schmeichel before Sanogo hit straight at the goalkeeper when clean through.
Leicester, though, were always on hand to fire warning shots of their own, David Nugent seeing a goal-bound effort blocked following good work from the outstanding Ulloa.
The game entered something of a lull thereafter as neat Arsenal football failed to bring chances.
That impasse ended when substitute Jamie Vardy came agonisingly close to grabbing a second Leicester goal, Szczesny producing a fine reactionary save.