Scripture never states explicitly that the tribe of Levi was not enslaved. In Midrash Tanchuma,1 however, Rabbi Joshua ben Levi makes this assertion.

Rashi 2 sees it as implicit from the fact that Aaron was able to go and greet Moses, and they were both allowed to come and go as they please—and even meet Pharaoh. How would they be roaming so freely if their tribe was bound in labor? Rather, since the Tribe of Levi was a priestly class, even in Egypt, the Egyptians allowed them complete freedom.

Even back then, the tribe of Levi served as the spiritual compass of the Jewish people. There are a number of explanations of how this saved them from enslavement. Here are some:

A. Pharaoh was aware of the fact that he would one day be punished for enslaving the Jewish people. He hoped that by allowing the tribe of Levi to continue to study G‑d's teachings and serve Him, he would be spared from retribution.3

B. Even Pharaoh understood the need for every nation to have spiritual leaders and guides and therefore kept the tribe of Levi as the guardians of the tradition.4

C. Jacob had told his children that the tribe of Levi would one day merit to carry the Ark of the Covenant—and the rest of the traveling Tabernacle—in the desert. At first, when Pharaoh began inducing the people to work, the people came as volunteers. Only later did the work become mandatory. The Levites were aware of their special purpose and felt that shoulders reserved for the Ark of G‑d should not carry Pharaoh's bricks. Since they never began working for Pharaoh even in the volunteer stage, they were not subjected to the subsequent slavery.5

Please let me know if this helps.

Yours truly,

Rabbi Menachem Posner