There is something wrong if the headset works loose with the stem tight. The star nut / top cap is there to pre-tension the bearing until the stem collar is tightened.
1 The bearing cups were not snug in the frame and are working their way in (common)
2 The headset ran loose and is ovalised slightly, these can be horrors to keep tight.
For 1, retighten and tell off whoever installed the cups...
For 2, I have solved it by removing the brake and front wheel, tightening the headset up and then spinning the whole steering several revolutions. This seems to settle the bearings into their loosest position. You may then need to retighten.
Ride it or milk it!