I am not sure how to explain it very well so here is my best.
When ever I use the look stick the movement starts out slow but then speeds up to the right speed the longer I hold down a direction.
Is this just how the game is on a controller or is it a setting I can adjust. I remember looking to see if I could change it in the options but I couldn't find anything. Would getting a mouse style controller change this?

Its just something that has been bugging me for a long time and I am finally getting tiered of it.

I haven't has this. You might check the controller sensitivity in the options just to see if it helps.

I'm on 360 and I noticed that it was that way when I had the sensitivity pretty low. Like if you start rotating to the left or right (or look anywhere really) it would start slow, then speed up. I upped the sensitivity a bit and now it's a lot better. To me at least.

And if that doesn't work you'll probably need a new controller. Does this happen on every game?

No just when I play Skyrim. I will change it then and see how that goes. It just throws me off because it only starts slow then speeds up.

Okay. Tell if there's any differance

It's look acceleration and it's intended. (On PC you can disable it)

On console you can lessen the effect by turning up the look sensitivity but you'll get to a point of diminishing returns and it's never truly gone anyhow.

I like it. I think it makes finer grain movements easier, without making all movements slow.

Just my humble opinion.