Notice that Alma had to turn around to address the crowd of poor people. That is how we know they came up behind him. Notice that it was the priests who had excluded the poor from their religious worship. However, these were not the priests of Alma. Going clear back to the days of king Noah we have noticed that as soon as apostasy creeps into a body of the people they immediately release the priests of the true Church and appoint priests of their own.
These substitute priests invariably turn out to be proud and money minded. They dress in fine apparel and stoutly resist the attendance of the poor in their worship services. The poor complained to Alma that they had helped to build the synagogues and places of worship but are now driven out. Naturally this was music to the ears of Alma and the missionaries. We read:
“Therefore he did say no more to the other multitude; but he stretched forth his hand, and cried unto those whom he beheld, who were truly penitent, and said unto them: I behold that ye are lowly in heart; and if so, blessed are ye. Behold thy brother hath said, What shall we do? -- for we are cast out of our synagogues, that we cannot worship our God. Behold I say unto you, do ye suppose that ye cannot worship God save it be in your synagogues only? And moreover, I would ask, do ye suppose that ye must not worship God only once in a week?”