One of the primary reasons why we use an apple is because of its sweetness. Coupled with the added sweetness of the honey, it is symbolic of the ultra-sweet year we hope G‑d will grant us.
I suppose, however, that you are asking why the apple was specifically chosen from all other fruits that are also sweet—why not, say, a peach or a mango dipped in honey?
The apple symbolizes Gan Eden (the Garden of Eden), which according to the Midrash has the scent of an apple orchard, and in Kabbalah is called "the holy apple orchard." When Isaac commented regarding his son Jacob (Genesis 27:27): "Behold, the fragrance of my son is like the fragrance of a field, which the L-rd has blessed!", the biblical commentator Rashi explains that this refers to the scent of an apple orchard; the scent of Gan Eden.
Furthermore, when Solomon depicts the love G‑d harbors for His nation, he writes (Song of Songs 8:5): "Beneath the apple tree I aroused you[r love]." Eating an apple on Rosh Hashanah is an attempt to remind G‑d of our age-old love.
Rabbi Baruch S. Davidson