flowchart TD
A[Starting Point] --> B[Dead End]
A --> C[Dead End]
A --> D[Stepping Stone 1]
D --> E[Dead End]
D --> F[Stepping Stone 2]
F --> G[Dead End]
F --> H[Stepping Stone 3]
H --> I[Masterpiece/Breakthrough]
H --> J[New Stepping Stone]
J --> K[Unknown Possibilities...]
style D fill:#90EE90
style F fill:#90EE90
style H fill:#90EE90
style J fill:#90EE90
style I fill:#FFD700
style B fill:#FFB6C1
style C fill:#FFB6C1
style E fill:#FFB6C1
style G fill:#FFB6C1
Why This Book Changed My Perspective
As an AI scientist, I found this book particularly striking because its core insights emerged directly from AI research. The authors, Kenneth Stanley and Joel Lehman, discovered through experiments with evolutionary algorithms that novelty search - exploring what’s different rather than optimizing toward a goal - often outperforms objective-driven approaches, even at achieving the very objectives it ignores.
This challenged how I think about both research and career decisions. In AI & ML, we’re conditioned to define clear objectives and optimize toward them. But the book argues that ambitious goals can be deceptive - they blind us to the stepping stones that would actually get us there. The path to breakthroughs often looks nothing like the destination.
For me, this offered a liberating new compass: instead of asking “what should I achieve?”, ask “what’s interesting from where I am now?” Follow the gradient of curiosity, collect stepping stones, and trust that valuable destinations will emerge from the journey itself.
The Core Thesis
The book argues that ambitious objectives are often deceptive because they distract us from the very stepping stones that would lead us to greatness. Instead of fixating on a distant goal, we should embrace exploration, follow our curiosity, and let the process of discovery guide us to unexpected breakthroughs.
Key Concepts
The Search Space and Stepping Stones
Think of all the possibilities for our pursuits as a “search space” - a vast room of potential outcomes. Within this space:
- Most possibilities are useless and don’t lead anywhere meaningful
- A few are “stepping stones” - critical achievements that reveal new directions and serve as foundations for what comes next
- A handful are masterpieces that lead to major inventions and discoveries
The crucial insight is that stepping stones often don’t look like the final destination. First computers were made of tube vacuums, but vacuums were never invented for that purpose - they were merely stepping stones that happened to enable something greater.
Why Ambitious Goals Deceive
Ambitious goals are deceptive because they distract us from their own stepping stones. If we fixate on an ambitious goal like “invent the computer,” we would never be able to figure it out directly. The path to greatness is rarely a straight line.
flowchart LR
A[Glass Blowing] --> B[Light Bulbs]
B --> C[Vacuum Tubes]
C --> D[First Computers]
D --> E[Modern Computing]
style A fill:#E6E6FA
style B fill:#E6E6FA
style C fill:#90EE90
style D fill:#FFD700
style E fill:#FFD700
Example: If your goal is to become a millionaire, taking an unpaid internship seems like a bad decision when measured against that objective. Yet it could be the stepping stone that helps you acquire skills and build a network that leads to unexpected opportunities.
Objectives as a Wrong Compass
Objective functions are a flawed compass that often lead to deception. Stepping stones typically score low on objective metrics because they don’t resemble the final ambitious goal - yet they remain critical steps toward it.
flowchart TD
subgraph ObjectiveView["Objective-Based View"]
direction TB
OBJ[Goal: Become Millionaire]
O1[Unpaid Internship<br/>Score: 0/10]
O2[Entry-Level Job<br/>Score: 3/10]
O3[Senior Role<br/>Score: 5/10]
end
subgraph RealityView["Reality"]
direction TB
R1[Unpaid Internship<br/>Skills + Network] --> R2[Opportunity<br/>Discovered]
R2 --> R3[Startup<br/>Founded]
R3 --> R4[Success<br/>Achieved]
end
style O1 fill:#FFB6C1
style R1 fill:#90EE90
style R4 fill:#FFD700
When crossing a lake using stepping stones in dense fog, we have no idea which direction is correct. The solution is to follow the gradient of interestingness and remain open to any possibility that might lead us to the next stepping stone.
Two Types of Search
flowchart TD
subgraph OBS["Objective-Based Search"]
direction TB
O_Start[Start] --> O_A[Path A]
O_Start --> O_B[Path B - Looks Promising]
O_B --> O_C[Closer to Goal]
O_C --> O_D[Even Closer]
O_D --> O_Goal[Goal Achieved]
O_A -.->|Abandoned: Low Score| O_X[Unexplored<br/>Could have been great]
end
subgraph NBS["Novelty-Based Search"]
direction TB
N_Start[Start] --> N_A[Novel Path A]
N_Start --> N_B[Novel Path B]
N_Start --> N_C[Novel Path C]
N_A --> N_D[New Discovery]
N_B --> N_E[Another Discovery]
N_C --> N_F[Unexpected<br/>Breakthrough!]
end
style O_Goal fill:#90EE90
style O_X fill:#FFB6C1
style N_F fill:#FFD700
style N_D fill:#90EE90
style N_E fill:#90EE90
Objective-Based Search
- Always converges toward the objective using past errors to follow the gradient
- Explores very few paths because many look unpromising, even if they might lead to significant stepping stones later
- Progress is measured as movement from bad to good based on the gradient
- Once you achieve the goal, you stay in that state
Novelty-Based Search
- Explores many more paths as long as they are novel
- Divergent approach that checks new possibilities
- Only concerned with whether a new state is novel - NOT how close it is to an objective
- Once you explore a state, it loses its novelty over time
- Constrained by environment, which defines the search space of what’s possible
- Moves from simple to complex behaviors as simple paths are exhausted
- Uses insights from previous experiences to discover new paths
Novelty-based search often solves harder problems faster than objective-based search, though it fails at some complex problems. The No Free Lunch theorem reminds us that no algorithm achieves best results across all domains - novelty search won’t guarantee getting us where we want to go.
The Compass of Interestingness
Novelty and interestingness should be our compass for uncovering future stepping stones. Instead of comparing where we are with an objective/destination, we compare our current state with the past. This isn’t deceptive because we already know the past and can see if our current path led to something novel.
flowchart LR
subgraph Wrong["Wrong Compass"]
W_Now[Where I Am] -.->|Compare| W_Goal[Distant Goal<br/>Unknown Path]
end
subgraph Right["Right Compass"]
R_Past[Where I Was] -->|Compare| R_Now[Where I Am Now]
R_Now -->|Is it Novel?| R_Next[Explore Next]
end
style W_Goal fill:#FFB6C1
style R_Now fill:#90EE90
style R_Next fill:#90EE90
The practical approach:
- Be open-minded and follow your gut-feeling about what looks interesting
- Branch out from there and be willing to abandon paths if experiments aren’t useful
- Remember that serendipity favors the prepared mind - invest in your knowledge and experiment without a destination in mind
- It’s a matter of time before you uncover stepping stones along the way
The Problem with Metrics
When people are measured or incentivized to improve a metric, the metric becomes the target - even though it was created to be an indicator of progress.
Example: GDP can be high in the short-run while the economy is unhealthy and heading toward disaster in the long-run. The metric deceives us about the true state of affairs.
flowchart LR
A[Current State] --> B[Temporary Decline]
B --> C[Learning/Growth]
C --> D[Breakthrough]
A -.->|Metric says NO| B
style B fill:#FFB6C1
style D fill:#FFD700
Key insight: Sometimes you need to go down to go up. A metric that only rewards upward movement will prevent us from taking necessary backward steps that enable future progress.
The Treasure Hunt Approach
When we don’t know what we’re looking for, we can adopt the concept of a “treasure hunt”. We look to collect as many stepping stones (treasures) along the way, which opens tons of possibilities without necessarily knowing the grand/ambitious goal.
flowchart TD
Start[You Are Here] --> S1[Treasure 1<br/>New Skill]
Start --> S2[Treasure 2<br/>New Connection]
S1 --> S3[Treasure 3<br/>Insight]
S2 --> S3
S3 --> S4[Treasure 4<br/>Opportunity]
S3 --> S5[Treasure 5<br/>Discovery]
S4 --> Q1[? Unknown<br/>Possibilities]
S5 --> Q2[? Unknown<br/>Possibilities]
style S1 fill:#90EE90
style S2 fill:#90EE90
style S3 fill:#90EE90
style S4 fill:#90EE90
style S5 fill:#90EE90
style Q1 fill:#E6E6FA
style Q2 fill:#E6E6FA
Note: Combining objective-based search with novelty search (keeping few paths alive to try when things don’t work) isn’t always a great idea when we don’t know what we’re looking for and when stepping stones may look bad according to an objective compass.
Innovation and Research
If we only fund research based on objectives:
- Ideas will converge to few paths
- Nobody will try to explore the unknowns
- Human knowledge will stagnate
The critical aspect of innovation is the accumulation of knowledge along the way from exploring stepping stones. Together, these accumulated discoveries form revolutions and breakthroughs.
Applying This to Personal Life
To apply novelty-based search at the personal level, use clues other than pure novelty:
- Potential of stepping stones - which ones might lead to other great stepping stones and unlock more?
- Interestingness - the most important clue, though subjective to the person’s instincts, knowledge, and experiences
It’s hard to argue why a given idea is interesting - that’s the point. Use your gut feelings and instinct as clues for what to explore next, given what you have available at your disposal.
Given current technology, innovations, resources, and what interests you + potential of ideas → pursue and explore new ideas to see where they take you. You will accumulate information along the way and more often than not arrive at a great stepping stone that unlocks new innovations.
Key Takeaways
Decide where to go based on where you are - this is often better than deciding based on where you want to be, because we have control to transform present into the future, NOT to transform future to present
When there is no destination, there is no right path - multiple paths can still lead to great places
Be open and flexible when new opportunities show up. What you’ve chosen as a pursuit may only be a stepping stone towards something completely different
Pick it up when you feel it’s right - It’s important to keep moving and opportunities will find you
Let the exploration/creation process guide you - along with following gradients of interestingness, this leads to masterpieces with stepping stones along the way