It would seem that some prayers are not answered. This does not mean, however, that they were in vain or not effective.
First of all, the principal objective of prayer is not that it be answered according to wish. Thus we are taught that he who sets his mind in prayer on the anticipation of seeing it fulfilled (iyun tefilah), will suffer heartache, as it is written, "Hope deferred makes the heart ache" (Proverbs 13:12).
The ultimate goal, therefore, is not the actual fulfillment of the request submitted, but the awareness "that in the whole universe there is none to whom it is fitting to pray other than G-d," and the recognition that man is altogether deficient "and only G-d can provide whatever he lacks."
The mental, emotional and spiritual results of prayer, as defined above, are more than sufficient to render tefilah worthwhile and effective.
To be sure, this does not mean that one should ignore the literal or common meaning of prayer, to petition G-d for all and any needs.
On the contrary: when one sees that he prayed and was not answered, he should pray again and again, as it is said, "Hope in G-d, be strong and let your heart be valiant, and hope in G-d." Nonetheless, one must not lose sight of the essence and underlying premises of the principle of tefilah.
Secondly: Some objectives, as, for example, the Messianic redemption, require multiple prayers - both in terms of the prayers articulated as well as in terms of petitioners. Though the literal results of these prayers are not perceived at the time, each of them is and remains significant: each of these prayers is effective, albeit partially, insofar that each contributes to the necessary sum-total, the ultimate whole.
Moreover, these individual prayers are not only part of a whole, which takes time to complete, but they effect partial or 'miniature' responses of the very genus of the request submitted to G-d. For throughout the period of the galut there are many forms of 'miniature' salvations and redemptions.
Thirdly, and most importantly: Every single prayer is effective and answered, though not necessarily on the level of the petitioner.
In the words of the Baal Shem Tov:
One must believe that as soon as the prayer has been uttered, one is answered for what has been requested. It may be asked, that at times the fulfillment of the request is not perceived. In fact, however, (the prayer has been answered, except that) it is in a manner hidden from the petitioner.
For example, one may have prayed specifically for the removal of his distress, and this request was granted in terms of the world in general. (The petitioner's personal anguish may remain, but) that itself is actually for his own good, or to expiate some sin, and the like.
When man's mind is set on awaiting the actual fulfillment on the specific, personal level, he brings materialism into the prayer, which in fact should be completely spiritual, for the sake of the Shechinah and not for the sake of the mundane. (The ulterior motive, therefore,) becomes a separating barrier.
In another version, recorded by the Baal Shem Tov's grandson and disciple R. Mosheh Chaim Ephrayim of Sudylkov:
My master and grandfather said that all prayers are effective in the upper worlds, and sometimes in other parts of the earth. (Sometimes one may ask for one thing, and he is given something else; and sometimes the prayer's effects are limited to the upper worlds.) He based this on the verse, "When the exalted things are debased among the children of man" (Psalms 12:9) - i.e., "the things that stand in the pinnacle of the universe, yet people debase them."
This refers to prayer which effects awesome things in the highest places of the worlds, yet people think that their prayers are not accepted and therefore treat them lightly.
For sometimes the effect of prayer is in the upper realms of the universe, and not below, and people, therefore, think that their prayer was, Heaven forbid, in vain. In truth, however, this is not the case. All prayers are accepted, but their effect is according to what omniscient G-d determines to be for the best interests of man and the world.
Excessive self-deprecation on the part of man, thinking his prayers to be of no avail, is in effect false humility, and may lead him astray. False humility causes man to think that his service of G-d, his prayers and Torah, is of no consequence. In truth, however, he must realize that he is a `ladder set on the earth, and its top reaches into heaven': all his motions, his speech, his conduct and involvements, leave impressions in the uppermost realms.
By thinking to himself, `who am I that I could blemish or correct anything above or below, that my doings will leave a mark,' he will be led to follow the inclinations of his heart, imagining that he has nothing to worry about.
Thus we are taught, "Da mah lema'alah mimach - know, that whatever is Above - it is all from and through you yourself!" All of man's actions are of cosmic significance. All of man's actions elicit commensurate reactions.
The needs of Your people Israel are many, and their wit is scant.
Yehi ratzon milfanecha, O G-d, our G-d, to give to each and everyone what is sufficient for his maintenance, and to each body sufficient for what it lacks; and do what is good in Your eyes. Blessed are You, G-d, who hears prayer.