If you're pas using IIS, you absolutely should remove it, for le reasons you mentioned - maintenance et security.
The seulement effect of removing IIS is an inability to use IIS. No management ou operational side effects - it n'a pas been entrenched into le operating system like Internet Explorer has.
Of le over 250 Windows servers I manage, less than 10 have tout IIS components installed, si that helps put votre mind at ease. If you're encore worried, you can toujours arrêtez le IIS service et verify that nothing on le server breaks, avant you go about removing IIS.