Yes. I think hundreds of people have done this.
Physically walking to locations that harbor logs, clay, and/or quarried stone?
Well where else are you supposed to get it from?
You can only get logs from the Mills, and stores don't sell clay/stone. (except maybe the occasional scrap)
The amount of iron you need is so huge, that you can't mine it all. You gotta get some from the stores.
Gathered all the detailed supplies such as goat horns, glass, fur, etc... to build furnishings?
Pretty easy. Travel to the stores, and hunt some animals to get the rest.
Seems like a lot of people on the forum have completely finished their house and all the furniture. (some people even did all three houses)