I've tried using the console command player.removespell
and get some kind of error. I can use this command
to remove spells I've added myself from skill books but the starting spells
for whatever reason don't respond to that command.

Also, from my poking around the CK I haven't found a straight forward way
to remove starting spells.

Any suggestions?

There is no way currently.
Wrye Bash is possibly the best bet for this happening in the future as a delete spell function has existed in the Oblivion version for a considerable time.
The Skyrim version is still in the early stages of development, but knowing the Bash Team it will happen eventually.

In the editor I noticed that there's a tick mark for Spells called something to the effect of Player Starts with or Starter Spell (Yeah, I'm too lazy to start it and find out...long night). Try going into the editor for those spells and unticking them. Save the mod, enable the mod, and then see if you can remove them.

-MM

Checked that already. The Heal and Flame spells aren't ticked as starting spells, which you would think it'd be that simple but it's apparently
dependent on something else.

http://tesalliance.org/forums/index.php?/files/file/1102-no-starting-spells/

but as mentioned, it only affects new characters.

I assume you can do it by altering the Spell List of the Player Actor though I haven't tried it?

That was big help and a good start. Now to find a way to remove the racial spells of bretons, dunmer, etc...

Hi, I wrote the above mod (glad it helped you!).

To remove the racial starting spells in the Creation Kit is very simple, what you need to do is:

Select Race under Characters in the Object Window.

Right-click each playable race (ArgonianRace, BretonRace, DarkElfRace, etc...) and select edit.

There is then a table of spells labelled Specials containing that races abilities.

Right-click the ones you want to be rid off and select delete.

Then save your new plugin!

If you get stuck with that and would like further advice or if editing in the Creation Kit makes you uncomfortable, send me a PM and I'll see what I can do. Best of luck!

Thank you. I can remember doing something like this in oblivion but things have changed in the kit since then.

That's awesome. I assume it's the same to add spells/perks to races? Then my only incompatibility will be with new race mods.