Deleted the DLC, and the game was still playable, with the exception of all DG related items (obviously).
So for one I am confirming this is doable.
Secondly I am wondering if this can cause serious problems in the future.
I have deleted DG before (although did not do the MQ quest for it) and noticed an interesting glitch where my PC was able to leave the boundaries of the gaming world. Made it to Vvardenfell.
*this was on 360*
Eh, I personally wouldn't want to risk anything.
Well, if any Dawnguard scripts were registered for updating and were never unregistered, then they will continue to attempt to update and will eventually bloat your save file to massive proportions.
In addition, many factions now have relationship data with invalid factions and form lists contain invalid records.
Not to mention that your save is missing hundreds of scripts that were attached to it; while removed scripts are of debatable concern, it's generally been agreed that users are "married" to their mods once a save has been made with mod content loaded -- and that includes DLC.
Basically, your save is a ticking time bomb, even if you don't see errors now, you're likely to encounter them later.