tightening a pickup jack

The pickup jack on my dulcimer is loose.   It rattles around and creates a buzz when I play.

Has anyone used one of these tighteners ?  Would it really work? 

The other option, I fear, would be to bring it in to a shop.