Nephi's charge against the churches reprises another theme from Isaiah. Isaiah similarly railed against those in power who gained at the expense of the poor:
Isa. 3:13-15
13 The LORD standeth up to plead, and standeth to judge the people.
14 The LORD will enter into judgment with the ancients of his people, and the princes thereof: for ye have eaten up the vineyard; the spoil of the poor is in your houses.
15 What mean ye that ye beat my people to pieces, and grind the faces of the poor? saith the Lord GOD of hosts.
In particular, note the accusation in verse 14 of the Isaiah text. Isaiah complains that the powerful have "eaten up the vineyard." The results of the labor in the vineyard have all gone to the powerful. The fruit of the vineyard is supposed to be "eaten up" in the sense that it is eventually consumed. The problem Isaiah has is that the entire fruit of the vineyard is consumed by the powerful, leaving nothing for those who created that "fruit."
When Isaiah says that the "spoils of the poor are in your houses," he indicates that the labor and results of the labor of the poor are benefiting the powerful, not those who created those "spoils."
In a very similar fashion, Nephi condemns those whose sanctuaries and clothing come at the expense of the poor. Nephi is not condemning fine buildings or clothing, he is condemning the methods that provide them inequitably.