You are not forced to be part of a faction, you get a choice of 3 factions. In Skyrim you only get a choice of 2, did you complain how that broke the immersion?
No, because I was allowed to stay neutral.
I'd be more lenient with the whole 'this race can only join that faction' if staying neutral meant I got to explore the continent with one character.
It depends how it's implemented. Guard towers, walls etc and getting killed if you tried to enter would actually increase immersion. If it's an invisible wall that says you're the wrong faction and doesn't even let you try cross, it's going to svck.
I'm leaning more toward there being "natural" hindrances such as mountain, canyons, impassable forests, etc, with the occasional border guard.
It'd still svck not being able to venture further but everything's better than invisible borders, yeah.