The Book of Mormon text typically refers to “that great city”, twice in 1 Nephi and eight times elsewhere (in Helaman, 3 Nephi, and 4 Nephi). The two examples in 1 Nephi are:
But twice in 1 Nephi (and nowhere else) we have “the great city”, here in 1 Nephi 1:4 and also in 1 Nephi 11:13 (“I looked and beheld the great city Jerusalem”). Since that is highly expected in this phrase, Oliver Cowdery’s correction of that to the in 1 Nephi 1:4 probably represents his correcting the printer’s manuscript to agree with the original manuscript (no longer extant here). The level of ink flow is unchanged, so the correction seems to be immediate.
Summary: Oliver Cowdery’s immediate correction of that to the in “the great city Jerusalem” is most probably a correction to the reading of the original manuscript; either determiner is possible, although that clearly dominates in the text.