I've been playing for a good 8 hours and have only come across them once, in a mine.

I can't buy them at merchants, or find them in barrels at the entrance to caves, or anywhere you'd expect torches to be in an Elder Scrolls game. Where is a surefire place to get a torch in game?

As far as i know you can get them off from the wall. But for your info, there is a spell called candlelight. Its a mana friendly spell and works great.

They're found by the truckload in most mines or caves - you can pick them off the wall.

Most guards also carries one

Or pickpocket guards.

Edit: Fleapants beat me to it

They do seem pretty rare... I've played for about 20 hours and have 2 torches... (In Oblivion, after 20 hours I must have had at least 20 torches)
How often do you actually use torches? I'm a sneaky thief/assassin archer, so I've never used a torch (too much light).
And do they burn out after a certain period of use? In Oblivion I would periodically unequip my torch then re-equip and it never burned out.

I've managed to find mine from different kinds of dungeons, you can pick them off the walls but the object has to be named "Torch Scounce". And regular Torches are sometimes found burning in the ground, you can pick those up too. I don't think I've bought them anywhere or found them in containers, but I've managed to gather 12 of them without really having to look for them.

Maybe it's the dungeon type, so once you find one with Scounces, you can expect to find a good bunch of them.

EDIT: Someone asked if they burn out - yes, they do, but they last a good while.

I wondered this at first, but then realised that the dungeons are alot brighter in this game compared to oblivion. I turned down the brightness quite a bit for my game, just to give off that scary, darker dungeon feeling. So I find myself using candlelight (spell) alot more then I use to !

I'm a thief too, but just for illuminating dark areas in dungeons searching for loot. I'd rather not have to use spells or staves, as I'm a more mundane character.