Rabbi Israel Baal Shem Tov once said to his disciples:
“There once lived two neighbors, a Torah scholar and an impoverished laborer. The scholar would wake before dawn, rush to the study hall and study for several hours. He would then pray at length and with great devotion, hurry home for a quick bite of breakfast, and return to the study hall for more hours of study. After the noon meal he would go to the market and engage in some minimal dealing—just enough to earn him his basic needs—then back to the study hall. After evening prayers and the evening meal, he would again sit over the sacred books till late into the night.
“His neighbor would also wake early, but his situation did not allow for much Torah study: no matter how hard he struggled to earn a living, he barely succeeded in putting bread on the table. He would pray quickly with the first minyan at daybreak, and then his labor would consume his entire day and the greater part of his night. On Shabbat, when he finally had the opportunity to take a book in his hands, he would soon drop off from exhaustion.
“When the two neighbors would pass each other in the yard, the scholar would throw the crass materialist a look of contempt and hurry on to his holy pursuits. The poor laborer would sigh and think to himself: How unfortunate is my lot, and how fortunate is his. We’re both hurrying—but he’s rushing to the study hall, while I’m off to my mundane burdens.
“Then, it came to pass that the two men concluded their sojourn on earth, and their souls stood before the heavenly court, where the life of every man is weighed upon the balance scales of divine judgment. An advocate-angel placed the scholar’s many virtues in the right cup of the balance scales: his many hours of Torah study, his meditative prayers, his frugality and honesty. But then came the prosecuting angel, who placed a single object on the other side of the scales—the look of contempt that the scholar would occasionally send his neighbor’s way. Slowly, the left side of the scales began to dip, until it equaled, and then exceeded, the formidable load on the right.
“When the poor laborer came before the heavenly court, the prosecutor loaded his miserable, spiritually void life on the left scales. The advocating angel had but one weight to offer—the sorrowful, covetous sigh the laborer would emit when he encountered his learned neighbor. But when placed on the right side of the scales, the sigh counterweighted everything on the negative side, lifting and validating every moment of hardship and misery in the laborer’s life.”

I think this story is also saying that it is not only important that we study Torah or even how much we study, but with what intent. Judging from his contemptuous attitude toward his neighbor, the "scholar" seemed to be using all those hours of study to elevate his ego rather than to connect with Hashem or with his own better nature.
Good point about all the hours the “scholar” made of study seemed to be elevating his own ego rather than to connect with Hashem or his own better nature. However, it goes to show that no matter the efforts made in Torah study it may not change a person into a more right thinking and acting or “kinder person. But who knows where each person starts spiritually, and what they individually have to overcome in their life. Can we really judge properly, assess correctly, and assume we understand fully based on the appearances of things?