If all the girls are not married there is a good chance of adultery, and a society in which adultery is allowed cannot be very peaceful or pure
"If you are miserly and do not give Me the offerings, every one of you will have an old husband with at least four co-wives."
In India in those days and even until fifty years ago, polygamy was freely allowed. Any man, especially of the higher castes—the brāhmaṇas, the vaiśyas and particularly the kṣatriyas—could marry more than one wife. In the Mahābhārata, or the old history of India, we see that kṣatriya kings especially used to marry many wives. According to Vedic civilization there was no restriction against this, and even a man more than fifty years old could marry. But to be married to a man who had many wives was not a very pleasing situation because the husband's love would be divided among his many wives. To punish the girls unwilling to offer Him the naivedya, Lord Śrī Caitanya Mahāprabhu apparently wanted to curse them to be married to men who had at least four wives.
The social structure allowing a man to marry more than one wife can be supported in this way. Generally in every society the female population is greater in number than the male population. Therefore if it is a principle in the society that all girls should be married, unless polygamy is allowed it will not be possible. If all the girls are not married there is a good chance of adultery, and a society in which adultery is allowed cannot be very peaceful or pure. In our Kṛṣṇa consciousness society we have restricted illicit sex. The practical difficulty is to find a husband for each and every girl. We are therefore in favor of polygamy, provided, of course, that the husband is able to maintain more than one wife.