Maybe an on going use of xp after the cap in some form, such as one off temporary expenditures. Abilities or something.
And/or maybe skills that slowly degenerates somewhat after the cap, like aging. Extra XP could then be used to maintain yourself and/or on one off abilities. You could start specialising in something else but then due to neglect you would loose other skills. Eventually you'd just get too old to keep up and face the bitter end.
maybe.
Or somethun

But yeah, having companions and leet words makes you lvl up quite fast, maybe don't do those things

On a side note, FO3 was much, much, much easier to lvl up so I think they really did try to make it last.