You hear the word "exponential" tossed around a lot to mean "fast". That's not quite what it means, and the difference is important.
By definition, an exponential function is one whose growth is proportional to its magnitude. Double the starting amount, double the rate of growth. (I'll get to specific examples below, and there are a few in the references.) An exponential curve starts off (or can start off) at a small value, with slow, barely-noticeable growth. As it gets bigger, though, it grows faster. (Engineers call this positive feedback.)
Exponential functions in physical systems are often associated with instability: Whatever you planned on doing, if you've got an exponential in there which you're not tightly controlling, everything will go along fine until suddenly it blows up.
As a species, humans tend to have a poor feel for exponentials. We tend to shape our gut feelings and top-of-the-head estimates based on linear extrapolation. If we base our understanding of a system on its behavior when the exponential growth is small, we tend to assume the growth will _always_ be small. If, however, it's actually a small exponential, we'll underestimate the eventual growth - and possibly underestimate it by catastrophic amounts. This leads us to the next phenomenon associated with exponentials: To actually turn off the exponential, you need a huge change to the system, a change that makes whatever was driving the exponential growth irrelevant or self-sabotaging. (It's also possible to balance one exponential with another, but that's generally tricky.)
Near the system’s limit for the function, one of two things usually happens. The usually-preferred result is a smooth leveling-off of the growth into a logistic curve. The alternative is overshoot and possibly large, chaotic oscillations about the nominal limit. This chaos results as different portions of the system are in “growth” and “collapse” modes depending on detailed local conditions. Overshoot may also result in lower long-term capacity than if the limit was approached smoothly.
To summarize:
Exponentials start small and grow slowly, but pick up speed as they go.
It's easy to underestimate the effect exponential growth will have.
Having an exponential around which you can't control is dangerous.
Exponentials have the potential to radically change the system you're dealing with.
If an exponential is present, it is generally useful to engineer a controlled approach to the system’s nominal limit.
Examples:
Interest on a savings account or on a debt, including the national debt, is an exponential. Initially-small loans can result in large debts if left unattended for too long. When debts are small, a larger portion of payments goes to paying down the principal and less is spent paying off interest. As debts grow, it becomes harder to pay them off. For this reason, it is usually advisable to make more than the minimum payment on loans - especially high-interest or high-principal loans. For national economies, debt should be paid down when the economy is growing because it will be more difficult to do so when the economy slows. Debts that grow beyond the debtors’ ability to pay can cause bankruptcy for them or financial collapse for the lenders.
Given sufficient resources, all organisms’ populations grow at exponential rates as adults generate children, grandchildren, great-grandchildren, and so on - more in each generation. As population approaches the environment’s carrying capacity, limits appear in the form of disease, starvation, and other resource constraints which tend to result in mass die-offs. Interestingly, for human populations, increasing wealth and economic security decreases the population growth rate. In particular, more educated women have lower fertility as professional ambitions and knowledge of birth control options limit the number of children women choose to have. This introduces a humane option for controlling human population growth: Improve economic conditions and educational opportunities in poor areas. Doing so will decrease the pressures on resources that lead to war, starvation, and environmental disasters.
