Of course, if you just delete the directory and the shortcuts and whatever, you'll still have registry entries for the game. These will still be parsed (as will every registry key) when windows starts. The more you have, the longer startup will take. So try to uninstall from "add/remove programs" in your control panel or use some kind of uninstaller. In the long run orphaned registry entries will slow down your machine.

Of course you could go into the registry and manually remove the entries yourself. But I don't suggest it. You could make your computer un-bootable if you delete the wrong thing.

BTW, there are programs that will clean orphaned entries from your registry, but they can be problematic. They can either be badly written and hose your registry. Or, if you have removable hard drives and swap them out, they can delete registry entries that are actually needed.

Good luck,