From Execution to Problem Definition — The Evolution of AI Autonomy

This post focuses on mapping the five levels of Autonomous Driving (AD) onto AI coding and software development, and digs into the fundamental differences between the two, the limitations of AI, and the "moat" that remains irreplaceably human in the AI era.

I. Autonomous Driving vs. AI Coding: A Five-Level Mapping Framework¶

The industry and academia are increasingly applying the autonomous driving L1–L5 framework directly to software development, 1:1.

II. The Fundamental Difference: Physical Boundaries vs. Logical Infinity¶

While the architectures look similar, the source material points out a key "harsh reality" difference between the two:

1. Environmental stability:

2. Autonomous driving (L4): operates within "bounded scenarios" (e.g., fixed routes), where physical rules are relatively stable.

3. Software development: there's never a fixed scenario. Environments drift, APIs get updated, data pipelines break — and when AI handles long workflows, success rates tend to drop exponentially as the number of steps increases.

4. Clarity of the endpoint:

5. Autonomous driving (L5): the goal is crystal clear and has an endpoint — "safely get a person from point A to point B."

6. Software development: there is no endpoint. Requirements stem from the constantly shifting desires and pain points of human society. If AI starts defining its own requirements, it risks becoming "self-indulgent" and disconnected from actual human needs.

III. AI's Blind Spots and Limitations: The Constraints of argmax¶

AI's underlying logic is mathematical optimization, which leads to the following blind spots:

• The local-optimum trap: AI excels at finding the maximum probability or minimum loss — argmax — within a given "sandbox" or problem space. If the objective function is set wrong, AI will produce an answer that's logically sound but commercially disastrous.

• Lack of business intuition ("taste"): AI can produce ten dashboards, but it has no idea which one management will actually consider to have the right "taste" or sense of direction.

• Inability to handle ambiguous decisions: AI excels in quantified worlds, but when faced with real-world business decisions involving incomplete data and shifting goals — like how to trade off precision against recall — it can't give a "correct answer," because that involves taking on risk.

IV. The Human Moat: From "Doing" to "Defining"¶

As AI makes execution cheap, human value shifts to a higher dimension:

1. Problem framing: AI knows how to solve a problem, but not which problems are worth solving. Humans have to draw the walls of the racetrack — the track itself and the finish line.
2. System design: weighing long-term maintainability and cross-system trade-offs — something AI is currently bad at when it comes to global design.
3. Defining KPIs and objective functions: the real value lies in defining what to optimize for. For example, in defect detection, are you chasing 99% accuracy, or prioritizing catching critical failures? That requires real domain knowledge.
4. Governance and risk control: once AI reaches L4/L5 capability, mistakes get amplified. Humans must decide what AI is allowed to do, and perform the final audit and oversight.

💡 Summary¶

Over the next 2-3 years, the value of people who purely write code or tune parameters will depreciate. Your professional value doesn't lie in trying to out-compute AI — it lies in "defining the problem space." "Asking the right question" and "business impact mapping" will become the most valuable skills of the AI era.