I am talking about stuff like garnet, flawless garnet and other jewels. Do they serve any use in the game other than being converted via merchant into gold coin?

making jewelry

u can combine gems with silver or gold bars into very expensive jewelry

you can make jewelry then enchant the jewelry. i dunno if the quality of your bling affects how much +enchant you can do though.

ah, thanks for the info. Jewelry making. I have been ignoring smithing so had no idea.

I have several miscellaneous objectives where people have asked me to bring them various gems, so I now tend to hang onto mine. It's not like there's a shortage of money to be made by other means in this game, so I'm not crippling myself financially by not selling them!

keep the unusual gems though, they have a purpose (quest)

if you find a flawless ruby, for the love of talos keep it!!! You need it for something, (no spoilers) and they are stupidly hard to find when you actually need them