What Criteria must a Chronograph meet?
And indeed, a proper chronograph must meet two criteria: it must keep time (act like your typical wristwatch or pocket watch) and it must serve as a stopwatch. This requirement applies regardless of whether the device is digital or analogue. A fully digital chronograph naturally has an advantage, in that it may have a multi-purpose display that, with the touch of a button, can be repurposed as a dedicated stopwatch display or timekeeping display. An analogue device, on the other hand, may require two different displays. If the analogue design is clever enough, though, the same display can be used for both functions, which the user can toggle just as he or she would when using a digital device.
The True, Modern Chronograph
Some people, though, just use the word "chronograph" as a fancy word for a watch. It may or may not be to your taste, but, given that many modern wristwatches, at least those of the digital variety, include stopwatch functionality as something of a standard feature, the term is often used with inadvertent accuracy. The "elite" of the chronographic world, though, are those classic analogue models, wherein the stopwatch functionality isn't an easily-added standard feature but rather the result of special planning, attention to detail and craftsmanship on the part of the watchmakers.
There's an even more classic version of the chronograph than the analogue wristwatch variety. As you might expect from the "graph" suffix, the true origins of the chronograph lie in an analogue device with can actually mark down the moment at which an event occurs. Many advances in timekeeping were driven by the needs of navigators and astronomers, and the chronometer is just one of the many such devices that grew out of the old age of discovery. With a pen or stylus and rotating cylinder (powered by clockwork) on which marks can be made, astronomers could mark down the precise moment that celestial phenomena occurred, which could then be used to calculate the positions of other celestial bodies, which could then be used for navigating.
The Evolution of the Chronograph
How did we get from a rotating pen-and-cylinder on a desk in an observatory to a fancy wristwatch with pushbutton modality? People like (and need) portability. A stopwatch has many uses in everyday life and there's no reason to consign it to those with the money and space for a desktop device. Pocket watch chronometers first appeared in the 1700s but it took a while before personal taste and mechanical refinement converged in the 1820s, when such timepieces actually became popular. Chronometers can also have additional measuring functions added to their faces, tracking different increments of time, or for use as tachometers, etc.