It's hard for me to imagine such a difference in valve lash going from hot to cold.
Let me ask you this, what carburetor are you running? The Marvel, with the air valve & dashpot adjustment? My '24 sometimes does EXACTLY as you describe when starting cold after having run well. Inside the Marvel carburetor, there is an air valve, in other words a flapper, that opens up to allow additional air into the carburetor for higher speeds. The air valve has a dashpot attached to it, (basically a small shock absorber), that keeps it from fluttering and reacting too rapidly. That dashpot can get gummy and will stick after everything cools down, gluing itself up solid. It's located inside the large brass knurled knob, sticking out of the carb at a 45 degree angle. Removing that knob will reveal the small piston that travels within it, as well as a very special, (as in don't loose it or mess with it), spring. Before you remove the knob, count the number of turns it takes to do so and after cleaning with carb cleaner, screw it back in the same number of turns.
Might also be a vacuum tank problem, but let's look here first.