Two goals in the opening 20 minutes by Lee Miller and former Bhoy Mark Burchill left the home side with an uphill battle, albeit though they had over 70 minutes to recover from their poor start.
Although Celtic squandered chances most notably by Craig Bellamy and substitute Ross Wallace, who had come on for the disappointing Stephane Henchoz after just 53 minutes, it was the magnificent defensive work by Craig Gordon, Steven Pressley and Andy Webster which defied everything that Celtic could throw at them.
The home team started in their usual fashion with Stan Varga glancing his header wide from Alan Thompson's free-kick after John Hartson had been fouled.
There seemed little danger when Henchoz conceded a free-kick midway in his own half after a foul on Jamie McAllister. The Lithuanian Marius Kizys curled in an in-swinging free-kick which Webster got his head to only to see David Marshall block his effort on the line, but Miller was on hand to head home the rebound.
Pressley was booked for a crude challenge on Bellamy with Robbie Nielsen quickly following him into the book for his lunge at the Welshman.
Henchoz was also at fault for Hearts' second goal. The Swiss defender headed clear straight to Miller, whose quick pass found Burchill in acres of space and the striker slid his left-foot shot low past the stranded Marshall.
Celtic were stunned and from a Thompson free-kick, Hartson bulleted his header home, but the assistant referee had already signalled that the Welsh striker was offside.
Hartson turned Webster on the six-yard line, but his effort was well blocked by Gordon as Celtic started to recover their poise after their shocking start.
Seldom have Bobo Balde, Jackie McNamara and Varga have played so badly at the back and they remained nervous throughout, with the crowd quick to criticise any slip.
Stilian Petrov's 25-yard strike was parried out by Gordon before the home team wasted a great chance to reduce the deficit just before half-time.
Hartson's header found Bellamy and he quickly transferred the ball to Wallace who was through on Gordon, but the winger pulled his shot agonisingly wide, when a goal at that time would surely have revitalised Celtic.
It was no surprise that the home team mounted a siege on the Hearts goal at the start of the second half. Bellamy just failed to get on the end of McNamara's cross and Webster threw himself to block Petrov's strike.
The visitors were always dangerous on the break and from Miller's pass, Burchill's shot just floated just wide of the far post.
Gordon did well to hold two powerful shots from Varga and Aiden McGeady, but the Hearts keeper did even better just moments later when he touched over Hartson's close-range shot from Thompson's cross.
Hearts wasted a golden chance to put the game beyond Celtic when Kizys skipped pass the ponderous Varga and advanced on Marshall, only to see his shot with the outside of his boot go wide when a pass inside to the unmarked Miller might have paid dividends.
Celtic finally knew it was not going to be their day two minutes from time when Balde, who had stayed up in attack, tucked the ball back only to see Bellamy blaze wildly over in front of goal from only six yards.
Man of the Match: Steven Pressley - The Scotland international centre-half was the inspiration for the Jambos backline, making sure that Hearts didn't let their early advantage slip away.