Confidence, for me, comes when I've taken steps to put myself at the lowest risk of getting hurt. This check-list from Schneider Saddlery overviews the equipment you should consider to keep you safe off and on your horse.
Felt Stick Horse Tutorial [I believe children need rocking horse s but have made my share of hobby horses when needed. Never used felt, boot socks as far as I recall. Anyway, these look pretty snazzy, don't you think?