Hi! I'm pretty much going to be in the same situation as you in a couple of years, so here's what I've found;
There are many jobs in the field of video game design. The main ones can be split into three fields; art, design and programming.
Now, you've already said that you aren't so good at math or physics, so perhaps programming isn't for you, but art and design are still open paths!
I believe you said you weren't too good at drawing and such, but have no fear, for you can get better at that sort of thing by practicing. Take it from me, I could just about draw stick figures when I started doodling in classes, but now I'm (if I do say so myself) a reasonably competent artist with my own style.
Also, not all jobs in game art require such fine drawing skills. If you happen to have a knack in 3D modeling, that's classed as art too. You don't even need that much art skill to do it, but you do need a little math skill so...maybe?
It's a little harder for me to describe design, as it branches out in so many different ways...level designers, scene editors...loads of things!
I'm not sure what I'd recommend for you to do to get into the field. Generally, you would learn how to do your preferred job of choice, create something (a large scale mod is generally a good one, like Oscuro's Oblivion Overhaul) that would catch a companies eye.
As for learning the ropes themselves, you could start right now if you wanted! Obviously, the art side can be practised by drawing a lot, and if you wanted to go into programming then there are simple programming languages you can pick up with how to guides that could get you into it. If you wanted to be a 3D model designer, you could start up now with Blender (which is free) and get some online tutorials (I know there are a few about).
For me, I plan to go to Futureworks, which is a media college which does a game development diploma that I plan to take. However, after all my exams are done and dusted, I would like to learn a bit of basic programming and perhaps a mod or two for Skyrim along the way?
Regards, Soulwithlife.
P.S - If you would like to have a look at Futureworks, you can visit their website http://www.futureworks.co.uk/index.html.