If it shakes the car uncomfortably, it is probably none of the above although it is good to do all the things Paul suggests. The shaking can be eliminated by shimming the transmission's rear main shaft bearing housing.
Good luck getting the brake drum off, whatever you do, DO NOT REMOVE THE NUT ON THE BACK OF THE SHAFT while you try to remove the brake drum flange, it may come off the tapered shaft with explosive force.
John