Your argument presupposes that, if ever they were let into Israel, the refugees would automatically become voting Israeli citizens. However, this is not at all necessary: they could well be Palestinian citizens with Israeli residency permits. Their numbers would therefore not threaten Israel's existence as a Jewish state.
Lebanese protesters storm environment ministry amid trash crisis (AP)
from the article: Palestinian refugees are a threat to Israel's existence