Rabbi Elazar Ha'Muda'ee says: “Whoever embarrasses a person in public, has no share in the World to Come.” (Avot 3:11)
Rabbi Hoshayah took great care with the education of his son. He made sure that he would have a teacher who would not only teach him Torah, but would also be exemplary in his character and behavior.
The teacher Rabbi Hoshayah took for his son was blind. He was very learned, and he had great fear of Heaven. Rabbi Hoshayah admired the teacher very much. Every day he would invite the teacher to come and eat with him. He felt it was a big mitzvah to honor his son’s teacher, and he enjoyed discussing the Torah with him.
One day Rabbi Hoshayah found himself in a predicament. Guests arrived unexpectedly, and stayed a very long time. Rabbi Hoshayah was afraid of bringing his blind guest for dinner. Perhaps the guests would feel uncomfortable. Perhaps they might even say something which would embarrass the blind teacher. Reluctantly, Rabbi Hoshayah decided it would be better not to invite the teacher that day.
Meanwhile, when Rabbi Hoshayah did not show up as usual, the teacher wondered what had happened. “Perhaps he no longer finds it pleasant to have me dine with him,” the teacher thought.
The guests stayed many hours. As soon as they left, Rabbi Hoshayah hurried over to the house of the teacher.
“Good evening, Rabbi,” said Rabbi Hoshayah. “Please forgive me. I feel so terrible. I had unexpected guests, and I was afraid that if I brought you to dine with us, they might insult you or embarrass you in some way, so I did not invite you to the meal. Please forgive me. Please don’t be angry.”
The blind teacher was very relieved. He thought Rabbi Hoshayah had forgotten him. Now he realized that Rabbi Hoshayah had wanted only to protect his feelings, and had even come to beg forgiveness.
The blind teacher said, “I cannot see, though everyone can see me — and you ask my forgiveness. May the Almighty G‑d who sees all, but cannot be seen, look kindly upon you.”
“Amen,” said Rabbi Hoshayah, grateful for the blind man’s blessing.

Invitation to Dinner Nosson makes a good point about sensitivity. Perhaps my initial remarks were too short and subtle, so let me be more explicit:
1. To not invite the blind teacher because of someone potentially making an insensitive remark is patronizing. it is wrong to go through life trying to avoid potential remarks or incidents that may or may not happen.
2. The presumption itself that someone will make an insensitive remark or someone might feel uncomfortable is also wrong.
3. To presume that the teacher with all of his experience and wisdom could not handle or address such a remark or be unable to have normal relationships at dinner is an insult to the teacher.
4. What does this story say about the Rabbi's friends/guests? Does he have so little faith in them, or have they already proven themselves to be intolerant, insensitive and unworthy?
If so, the proper course of action is for the Rabbi to have dinner with the teacher and disinvite the other guests.