First, pure alcohol (ethanol) is tasteless. However, the process of making and distilling alcohol includes congeners - other chemicals that are the result of the fermentation process and then selected (and in some cases modified) during distillation.

The congeners depend on the raw material (cane sugar, beet syrup, barley, malted barley, grain, corn, or potato) and the yeast strain that feeds on the sugars to produce alcohol.

Distillation then extracts the alcohol from the water in the ferment. Along with the alcohol comes the methanol (the heads and tails, which are discarded), some water, and some of the congeners. The heat of distillation also chemically modifies the congeners.

The temperature, type and material of the distillation still further modifies the congeners - copper stills react with and remove sulphur-containing congeners, for example. Modern column and vacuum stills can have very selective distillation. Cycling the distillate through multiple distillation and filtering stages allows for highly selective congener selection and end-product.

However, better source sugars, distillation stills and multiple distillation stages result in more loss and wastage. This is expensive. So cheap spirits use cheaper raw materials and cheaper distillation methods, and their products have more congeners that generally result in a harsher taste. Better starting materials and distillation/filtering smooths out those congeners for a better raw spirit.

Then comes the flavouring process (for gins) or cask aging (for bourbons and whisky/whiskey), which have been discussed by others. Again - more expensive starting materials at this stage produce better results. Proper barrels vs wood chips, burnt casks vs charcoal filters, more cheap vs fewer expensive botanicals - it all adds up to better flavour in the end result.

However, as also pointed out, selection should be based on blind tasting and not on price alone - the human brain is hard-wired to appreciate more expensive tastes.