It is tough to stand and climb with a lot of weight on your bike – the bike is naturally going to swing back and forth and the extra momentum makes it feel sketchy. One more reason to have low gears.
Grant Petersen claims there was a study that concluded 12 pounds = 1 mph, which sounds about right to me. My bikes are all heavy as hell, not to mention the rider. I think you mostly notice a bike is lighter when you pick it up, not when you ride it.