Hi Nick
this would not be the first one i've heard of or seen where the rivets that hold the pulley to the center boss have worn/ come loose

. to try fix it take the pulley off, then clean all the grease and muck off.
Inspect the pulley and hub to check that the center hole that locates the pulley to the flange is not worn, and that the rivet holes are not way out too. if the pulley is a good fit and all rivets are there, support one side of the rivet( head) on a solid object, then beat the other side with a hammer to tighten the rivets again

.
If the boss/ pulley hole or rivet holes/ rivets are worn real bad or missing buy a good used pulley to fit your motor and replace it.
If its worn bad you won't get it to run true or balance might possibly be off when it turns after doing it

I have had one here on a super that was all bolted together before. The rivets completely failed but it was a right mess again as the bolts came loose, and is probably re cycled as part of a Toyota somewhere in the world now
Regards Robert
A Fordson is for life not just for Christmas !.