This is a specific reference to the Lehites, who are represented by the branches planted in the choice land. Even in such choice circumstances and with the promise of profitable fruit at one time, the final result is the destruction of the good by the wild.
The symbolism is perhaps stronger than the historical case and it would be a mistake to read this allegory as a condemnation of the Lamanites as bad. The result of Lehite history is that the gospel did not continue, and that is the only reference here.