The fact that we eat peppers today is due to none other than Christopher Columbus, who brought them back to Europe from his travels in Central and South America. Interestingly, in botanical terms, peppers are actually berries.
The colour of a pepper reflects how ripe it is - which means all peppers start off green, and then turn either red, yellow or orange depending on the variety. The riper they are, the sweeter they become. Green peppers are picked when still unripe, whereas yellow and red peppers are harvested after they have taken on the colour of their variety. The taste of peppers is mainly due to an essential oil, the mild spiciness is due to capsaicin.