I married Mjoll the Lioness but I don't like the fact that at home she's dressed in plate armor with that axe on her back. How can I get rid of her and marry another woman?

Nope. The game doesn't have a divorce or re-marriage option. You're stuck with whomever you married first.

You can mess around with her clothing and weapons, however, by giving her better stuff, buy her default armor and then exchange the better armor for clothes or whatever, just make sure you don't take it all, let her keep her boots for instance (if you take everything, she'll revert back to default). She might walk around in her underwear when she's not following you, however.

This is if you're on console, of course. On PC, you can both change her clothes and remarry. But you'll have to ask the PC folks about that.

I'm hoping Solstheim will let us have a second wife in a second home.

So much nicer than all that messy divorce. Marriage should be for life. If problems occur then just take a second wife (or husband). Competition is healthy