Although you cannot usually remove furnishings, you can remove those ones by changing the small house to an entryway, I think it was the drafting table you used for that, or it could have been the capenter's workbench. You should have changed it before asking the steward to furnish it, now you will probably have to pay to have it furnished again.