The fact that the Nephites were fighting "for their houses, and their homes" (Mormon 2:23) and that the Nephites "did gather in our people" tends to suggest that the land of Shem was somewhat a center of Nephite population.
If we assume a hemispheric Book of Mormon geography model, after a very convincing and continuous retreat of possibly 3000-4000 miles from the land of Joshua "in the borders west by the seashore" to the land of Jashon (near the hill Shim, and somewhat near the hill Cumorah and an east sea), plus another retreat northward to the land of Shem, would the Nephites still be fighting for their "houses," and their "homes" (Mormon 2:23)? They certainly wouldn't have had time to build any in the previous 2 years of retreat, and given the odds and circumstances, would they even want to retrace their steps back to where there were?
On the other hand, assuming a Mesoamerican setting, the land of Shem could have been a familiar part of Nephite occupied land close to the narrow pass area (travel corridor through the Isthmus of Tehuantepec) for quite some time. [Alan C. Miner, Personal Notes]