I'm thinking of starting a character who will only use and wear what he makes himself, and only make them from things he has found, as in he can't purchase or sell anything, though he would have no qualms about stealing the odd item. In my current game I have noticed that I can occasionally find straw, but have not come across glass, except via merchants.

Granted I will have to make a few exceptions, such as scraping together enough 'found' gold to buy the land in the first place.

So does anyone know if it is possible to find glass out in the 'wild', so to speak, and perhaps a few reliably places for straw too?

Glass does occasionally turn up as random loot or when raiding the homes of others, you can find some. I have not noticed any reliable sources for either, and glass is a lot harder to find than straw. I think you will be waiting a very long time to finish furnishing your house if you expect to find enough glass for it.

You cannot cut your own wood without first selling some firewood

You could maybe get the 5000 doing quests. You will also be waiting a while to find 5000 in loose coin.

Maybe by the time you find all that Bethesda would have finally found a way to get the game to run as it should.