I have several Willys Knights and have driven thousands of miles over the last 40 years, after cleaning the oil pan I use 10w40 detergent .
Clean the rectifier and look for vaccuum leaks in the suction system of the rectifier, the t-bolts thru the flat tube get plugged and the copper washers on both sides of th tube develop leaks and will not suck the excess oil from the sleeves.
Check the bimetal valve in the top of the rectifier this controls vaccuum to the rectifier in the later cars the left the port open so the rectifier worked regardless of engine temperature.
The club has the rectifier housing to manifold gasket made of copper