There doesn't appear to be distrust of Jordan, rather, it is the instability of old regimes in the Middle East that concerns others. The fear of nuclear materials falling into the hands of third parties is absolutely legitimate. As you said yourself, Jordan is capable of proceeding without consent, so the "control" is really just pressure.
Israeli ambassador to U.K.: Elements of left 'in denial' about 'disease' of anti-Semitism (Haaretz)
from the article: Who's afraid of the Jordanian atom?