The End of Economic Growth? Unintended Consequences of a Declining Population

In many models, economic growth is driven by people discovering new ideas. These models typically assume either a constant or a growing population. However, in high income countries today, fertility is already below its replacement rate: women are having fewer than two children on average. It is a distinct possibility - highlighted in the recent book, "Empty Planet" - that global population will decline rather than stabilize in the long run. What happens to economic growth when population growth turns negative?