what roads are you riding on? I know one of them is saltsburg... you weren't headed home from the intersection of universal and saltsburgh, were you? There are a few gyms there... Or the Y on Frankstown? Frankstown people are a little jerky, but Saltsburg's never given me trouble.
Near my house there IS a guy I call The Invisible Man. I swear he's out for insurance money, because he's a decent sized black man, dressed all in black, riding a matte black bicycle, at night, against traffic, without a helmet or lights or reflectors. I am not making this up. I barely see him (usually milliseconds before I veer out of his way) biking up and down Third in rosedale at night any time I'm out after dark much. Scary.
As for the aiming, you know how cars tend to magically drift in the direction the driver is looking? Maybe it's the "hey, is that a really big guy on a bright orange bike I see? oops..." drift? Or you just have the absolute worst luck in riding in PH?