At least for objects that are not usable in some way. When you get quests such as collecting unusual gems and other high weight quests (at least where the weight is not part of the challenge of the quest), due to length of quest or due to the possibility that you might complete a quest (e.g. return the flute quest) and still be stuck with a quest object these need to be weightless so you can enjoy quests without having the quests make you unable to carry much loot. Just my opinion.

Many quest objects were weightless in Oblivion which led to some exploits with weapons and the like. So I would make this request only for those objects you can't use yourself.

Or the option to remove quest objects as long as they are put in a (player) owned container.

Quest objects are in fact weightless. They just have a weight noted but in fact don't weigh anything. Try removing ALL your items and gear except for the obvious irremovable quest stuff. You will see you have exactly 0 weight in your backpack.
My example : I have a couple of those unusual gems and that flute as you described in my inventory, they all mention a certain weight, but when I drop everything I carry then my inventory screen will tell me I have 0 weight on me.

No different than Oblivion... I verified this by adding 10K quest objects with 0.5 weight through the console and could still run around normally whereas would have had 5K encumbrance otherwise.

Pretty much this.