The efficacy of prayer is not something which can be established empirically, because there is always another explanation for anything – in extremis anything, no matter how apparently miraculous, can be explained-away by mental illness, mass delusion or lying.
But the validity of prayer does need to be confirmed inwardly and personally by each person – and this is typically by a prayer being grated in such a way that we are convinced (even if, as is usual, we would not be able to convince others – that does not matter, because the prayer was of the nature of a private communication between the prayer and God).
However, not all prayers are granted. Why is this?
In a bigger picture, prayer is one way that we are helped in this life – by the interventions of divine powers. And asking for, and getting, such divine help is one of the important rocks of faith. But why do we not receive more help, since life is so difficult?
Such a question can only be answered if we understand the basic purposes of mortal living, and in particular, why life is some kind of struggle for most people most of the time. If we are able to appreciate that life is a kind of educational process, then we may realize why help is provided, and not provided, in the way it is.
When someone is learning there is a time for help – that is what teachers are supposed to do – and there is a time for solitary struggle – for practice, for grappling with problems, for try, try, try again.
So, one important reason that prayers are not answered will certainly be that we are being required to do something from our own resources, because that is the only way we can learn. In effect, in the unanswered prayer, we were asking the divine powers to ‘do our homework’ – when this homework was vital to our learning.
Now, this is not the only reason, nor is it always the reason, prayers are unanswered – but it is surely one of the reasons, and perhaps a neglected reason: prayers are not answered, help is not given, when it is important that we do something without help, for our own good: when we have asked for help instead of participating in the learning process that is life.
This does not mean we should not have asked for help in the first place, because we can’t always know the nature of the situation (if in doubt: ask); but it does mean that when we have asked for help and received a negative answer, we then acknowledge the validity of that reply, and act accordingly.
Note: I feel that the refusing of help to someone who has asked for it, must be one of the most difficult and painful things for the loving divine powers – especially when their refusal to help is misunderstood as indifference or dislike. Parents will know the situation in which they are asked for help by a child, and they could help their child, but the parent knows that such ‘help’ would actually harm the child’s development, and that it is necessary (or, at least, better) that the child is not helped. However, often the child cannot understand this, but feels only a sense of betrayal at being refused help that the parent could have given; and the child reacts with anger and resentment against the parent – may lapse into hatred which may be extreme and lasting. In the end, the parent may simply have to tolerate the situation; in hope that, at some point in the future, the child will recognize that the refusal of help was motivated by love.