I have completed all of the City Influences Quests, but when I visit Solitude, I can not find Gulum-Ei. Where does he hang out, and for that matter, where are the other fences normally hanging out?

He's in the Winking Skeever - to the left in the first alcove as you walk in. The rest I have no idea about. (Oh, hmm. I do remember that some people have an issue with Gulum-Ei.... he may NOT be there, he may be in the East Empire warehouse - and unresponsive.)

Whiterun: Mallus Maccius - Honningbrew Meadery

Markarth: Endon - usually walking around outside Silver-Blood Inn or on the way up to Vlindrel. His house is basically under yours and one to the left facing Vlindrel. Wikias say he hangs out in the Silver-Blood Inn, but I've never found him in there. I'll zone into the Inn and back out and he's usu walking around nearby.

Riften: Tonilia - Ragged Flagon

Winterhold: Enthir - Frozen Hearth Inn or Hall of Attainment (he won't buy everything though for some reason)

Solitude: what Serethil said. Always found him there.

Edited for formatting.

Gulum-Ei and Mallus Maccius aren't unlocked through the Special Jobs. You unlock them through the Thieves' Guild main storyline.

...but do they act as Fences before the Influence Special Jobs?

Yes, but I don't think they have as much money. You also get Enthir from the TG's main quest. I think only Niranye and Endon come from the special jobs.

There's also a job you can do for Tonilia that will give you a fence in the Khajiit caravans.

I have yet to do any "special jobs". How do I get this job to get the Khajiit fence? I have done the the TG's questline.

To do the special jobs, keep after Vex and Delvin for work. Once you've done five of any combination of jobs for either of them in any one city, Delvin will have a special job to do there. In fact, if you speak to Vex at that point, she'll send you to him. I think you need to establish the connections in four cities before Tonilia offers the caravan one. So that's Solitude, Whiterun, Markarth and Windhelm. These also add the four vendors to the Ragged Flagon.

Wikia says you have to complete at least 3 of the 4 Special Jobs. It goes on to say that if you don't do Tonilia's quest "it will be difficult to acquire the Small Jobs for the last city you need".

Thanks!