This question has an extremely variable response from person to person. Generally you want to apply a fresh coat of paint after you begin to see heavy color fading and light cracking. You want to get the most out of your paint, but you also want to prevent damaging your home's integrity. There are two major components of paint in relation to longevity of paint and that is pigment and resin. Resin is the armor in paint that acts as a shield to the elements. A good indication as to when the resin in your paint is beginning to fail is when the pigment starts to fade. When your gray siding turns to a white-ish blue or dark green turns to a very light pea green, it may be time to consider painting. You can gather from that information that the resin is failing and therefore affecting the innermost part of your coating, the pigment. It will be soon after that the paint begins to peel.
Ideally, you want to paint or stain before the bare surface begins to show. It does not take long for bare wood, metal, or stucco to fail and that is the last thing you want. When your surface begins to rot or fall apart, repairs and replacements can get extremely expensive, very quick. Don't wait until all of the paint is falling off your home to repaint, get it done before the bare surface begins showing, allowing the elements to rot and decay it. Make sure to use a high quality product to ensure longevity. The more glossy the paint, the longer it will last as they tend to have higher amounts of resin in them.