Table from U.S. Geological Survey's "Geologic Map of the United States," 1974
The traditional colors used on geologic maps are actually set by agreement among the world's geoscientists. In the United States, the U.S. Geological Survey set a color standard for its first national geologic map in 1881, just before the European geologic community established its own standard (more on that story here). As a result, there are two major color systems in use for geologic maps (see the current USGS and International standards).
In general, these colors apply only to sedimentary rocks, and plain colors apply only to marine rocks. Sedimentary rocks that form on land are depicted with these colors plus patterns, such as dots or stripes.
The other major rock classesigneous and metamorphichave their own color traditions. Reds and oranges are reserved for the igneous rocks: volcanics take orange while plutonics take red, and in both of these older rocks are shown with darker colors. Metamorphic rocks tend toward browns, olives, and other "complex" colors. All of them use patterns freely as well.