Perhaps a small piece of tape covering the light up? (black electrical tape, if that matches the speakers better?)
edit: hmm. And either the thinner kind of masking tape, or a small pinhole poked near the edge, could still let out a bit of light. So that you can tell that they're on or not. Just a thought.