Most pilot lights have a thermocouple which is basically a heat sensor that shuts off the gas to the pilot unless it senses that the pilot is lit. So usually a pilot light won't leak gas if the pilot is not lit.