What are we actually trying to accomplish? We want to live with our desires and fears under control. What does that mean? It means not being disappointed when you want something, and not falling into what you're trying to avoid. That's what your practice should focus on. You can't achieve unshakeable desire and fear-free avoidance without serious, constant practice. Here's the key: if you let your desires and fears focus on things outside your control, you'll fail on both counts. Your desires won't be satisfied, and you won't avoid what you fear.
And what is that which is proposed to us as a thing to be worked out? To live with desire and aversion (avoidance of certain things) free from restraint. And what is this? Neither to be disappointed in that which you desire, nor to fall into anything which you would avoid. Towards this object then exercise (practice) ought to tend. For since it is not possible to have your desire not disappointed and your aversion free from falling into that which you would avoid, without great and constant practice, you must know that if you allow your desire and aversion to turn to things which are not within the power of the will, you will neither have your desire capable of attaining your object, nor your aversion free from the power of avoiding that which you would avoid.