While booting up the game? or the system...
And is there any audible confirmation that you have done it successfully? Some sort of "you've got 30 lives" sound bit?
Yes, you're supposed to do it while booting up the game. As soon as I push Play Game or otherwise, I start holding down the buttons and don't let go until the title screen shows.
There's no confirmation that I'm aware of. I've never seen or heard one to be honest whenever I've used it. I got the instructions from a post by a Bethesda person from this thread: http://www.gamesas.com/index.php?/topic/1120449-please-read-before-posting/page__view__findpost__p__16522616
It would be better if there was something that acknowledges the cache being cleared, but all we can do is assume it's working as it's supposed to. :/