Emergent Abilities

Emergent abilities are capabilities that appear abruptly in models above certain scale thresholds rather than improving gradually. In-context learning, chain-of-thought reasoning, and arithmetic ability have all shown roughly step-function improvements at critical parameter counts or training compute levels.

The mechanism is debated: some researchers argue emergence is a measurement artifact (metrics that show sudden jumps actually reflect smooth underlying improvements), while others argue genuinely qualitative transitions occur.

Emergence is one reason AI progress is hard to predict, capabilities that seem absent can appear rapidly as training scale increases, without anyone having explicitly trained for them.