We may well ask, “Who is our brother?” It is anyone in the covenant community. If we are angry with a fellow covenant member, we are alienating ourselves from the community. In Matthew it next says, “If thou bring thy gift to the altar, and there rememberest that thy brother hath ought against thee,” then we must reconcile with the brother, and then we can come back to the altar and offer our gift. In 3 Nephi 12, it is a little different. “If ye shall come unto me, or shall desire to come unto me, and rememberest that thy brother hath aught against thee.” Whether we already find ourselves at the altar or are just wishing to come to the altar of Christ, we must be reconciled to our community.
To which brother was Jesus referring? To any brother, or sister. Both in the Bible and here in 3 Nephi, where Jesus taught in an explicit temple context, if you ever realize that somebody especially within the Church has a problem with you, you need to seek that person out and reconcile before going forward with your service or worship in the name of Christ.