Knock the baskets over, they sometimes hide stuff under them. lol
Yes, do this. I have found that you have to search
everything in any cave, dungeon, etc. There is often good loot hidden in baskets, under bowls, in bowls up high on shelves, in cast iron pots, I could go on and on, but you get the picture. I have sometimes even found spell books hidden this way. Since I am a hoarder anyway, I sometimes just keep the basket or bowl to take home to decorate with.