Well last time I had lag I noticed it was because of one person not having a good connection, so I voted to boot him. After we kicked him everything was fine. I also noticed that there were three people spectating. Honestly the real problem was probably the 3 people spectating, but there is no way to check their connection, and I don't know how having spectators affect the overall connection speed. Either way after we got rid of him another person joined and we still had no problems. So you probably are just getting stuck with a teammate that has a connection that blows.