Livi looked worth a point until Bednar struck three minutes from time after some slack defensive play.
Hearts were under the cosh early on and Sam Morrow pulled a shot just wide in the opening minute.
Greg Strong should have opened the scoring in the third minute when he was unmarked at a corner but sent his effort over the top.
But Hearts took the lead against the run of play after 17 minutes when Livingston failed to clear a Calum Elliot cross and Bruno Aguiar's shot deflected off Richard Brittain and over Roddy McKenzie.
Chances were few and far between. Rudi Skacel shot wide at one end and then David McNamee was too high with a long-range effort at the other.
Allan Walker forced Craig Gordon to make his only first-half save seven minutes from the break with a fiercely-struck shot.
McKenzie had to be on his toes shortly afterwards to touch a 30-yard drive from Martin Petras over the top.
Livingston stunned Hearts with an equaliser in the 59th minute when Brittain beat Gordon with a free-kick that seemed to take a deflection.
Gordon then had to react swiftly to make a magnificent save from Walker when the Livingston man was only six yards out.
But Hearts went in front again when Edgaras Jankauskas headed in a cross from Deividas Cesnauskis.
Livingston refused to lie down and Dave Mackay volleyed in a terrific equaliser 13 minutes from time.
But Bednar had the final say when he put Hearts in front again three minutes from the end after a mistake by Jason Dair.