If you delete it, you will be able to download it again from Live. Once you purchase any content, for any game, you can re-download it again for no charge.
Configuring a USB stick for storage for the Xbox: http://support.xbox.com/en-US/xbox-360/accessories/usb-flash-drive
Moving content from the Xbox to USB stick: http://support.xbox.com/en-US/games/xbox-games/manage-games-data
Edit: Grammar correction.