🤖 Why Robotics Is Still Not Solved — And What's Actually Working Now

📌 The Perennial Promise of 'Next Year in Robotics'

For over a decade, the refrain has been consistent: next year, robotics will be solved. From the emergence of AlphaGo and Mujoko's rapid simulation training to the hype around teleoperation data collection via systems like Aloha, the field has continuously moved the goalposts. Even in 2026—now halfway through—commercial humanoid robots remain elusive. While pre-orders for systems like Neo 1X are open, truly deployable home or general-purpose robots are not yet on store shelves.

Despite this, 2026 has undeniably become the year of the demos. Policies can fold clothes, make sandwiches, and perform dexterous manipulations—but only in controlled, short-duration tasks, typically under two minutes. The question remains: why hasn't the leap to long-horizon, real-world deployment happened yet?

"If we're relying on teleoperation data and need to scale that up like crazy, we're kind of doomed."

🧱 The Four Walls Blocking Real-World Robotics

According to Stanford researchers and practitioners, there are four core obstacles that continue to limit embodied AI:

• Physical Real-World Modeling: Current video-based world models do not respect physics. When used to simulate driving or manipulation, they produce hallucinated transitions—drive into a grocery store, and the scene magically becomes a highway. The sim-to-real gap remains unsolved.
• Deformable Objects & Action Conditioning: Predicting state transitions conditioned on actions (not just passive observation) requires vastly more data and fails on non-rigid objects. Representation learning for action spaces remains an open problem.
• Sensory-Motor Limitations: Humans possess rich tactile feedback—detecting normal force, tangent force, moisture, temperature, vibration, and friction across the entire body. Robots, by contrast, often have one force-torque sensor per fingertip and a wrist camera. Neuroscientists emphasize humans can build world models without vision (e.g., finding a charger in a backpack by touch alone). Robots lack this "epidermis."
• Embodiment Drift: Actuators degrade over time due to dust, corrosion, and wear. The amount of force applied for a given command shifts unpredictably, rendering teleop data stale and requiring constant retraining. This is a silent killer of long-term deployments.

These challenges are not theoretical. They are encountered daily by teams deploying robots in real environments—from self-driving cars to manipulation systems.

🧠 Breakthrough: Adding Memory to Vision-Action Models

Presenter: Marcel (PhD, Stanford; Physical Intelligence)
Work: Multiscale Embodied Memory (MAM)

Most state-of-the-art policies—Pi0.5, Groot, and others—lack memory entirely. At every timestep, the robot processes a new observation with zero context about what happened before. This leads to pathological behaviors:

• A robot washing dishes forever, with no sense of task completion.
• A grilled cheese left on the stove until it burns, with no temporal awareness.

MAM addresses this by decomposing robot policies into two components:

1. High-Level Policy: Predicts the next step in language, maintaining long-term memory via compressed textual representations.
2. Low-Level Policy: Executes dexterous manipulation using short-term dense visual memory via a new ViT-based encoder with temporal attention layers.

This architecture enables:

• Cooking a grilled cheese without burning it (waiting as long as needed)
• Unloading groceries from a bag (remembering item locations seen only intermittently)
• Cleaning windows (knowing when to stop rather than wiping indefinitely)

Perhaps most critically, memory enables in-context adaptation—a property sorely lacking in current policies. Robots with memory can recognize and correct mistakes in real time, such as switching hands to open a fridge or adjusting grip height to pick up a dropped object.

"This is something that is very lacking now from robot policies and I'm super excited about the future with VLAs with memory."

Key Insight: The high-level policy uses supervised fine-tuning (SFT) with human annotations to determine what information to track. Future work may involve reinforcement learning to automatically discover optimal memory structures.

🔁 Self-Supervised Bootstrapping of Embodied Reasoning

Presenter: Milan (PhD, Stanford; Waymo)
Work: R&B ENCORE (Refine and Bootstrap Embodiment-Specific Chain-of-Thought)

Embodied reasoning—akin to chain-of-thought prompting for LLMs—has emerged as a powerful way to augment scarce robotics data. But what should robots reason about? And how verbose should that reasoning be?

R&B ENCORE treats reasoning as an unobserved latent variable and uses variational inference to:

1. Propose various reasoning traces (e.g., visible objects + move reasoning, subtask planning + gripper position)
2. Validate them based on three criteria:
   • Concision: Keep reasoning short to reduce latency
   • Non-triviality: Encourage generalization
   • Action-Predictiveness: Ensure reasoning is grounded in the embodiment
3. Resample to create a refined dataset of action-aligned reasoning

Findings across embodiments:

• Manipulation: Move reasoning + gripper position is useful; perceptual reasoning (listing all objects) is not—it introduces distracting information.
• Legged Locomotion: Structural affordances and movement reasoning matter; terrain and counterfactual reasoning is less useful.
• Self-Driving: Meta-action, visible objects, and perceptual reasoning are valuable; hallucinated experiences should be pruned.

Critically, the approach enables action-forcing at inference time—reasoning can be dropped to eliminate latency while still benefiting from its training-time signal.

"Selective reasoning is way more important than exhaustive reasoning. Even if reasoning is valid, it's not necessarily useful."

🎯 Sim-to-Real at Scale: Goal-Conditioned Dexterous Manipulation

Presenter: Tyler (PhD, Stanford)
Work: Sim Tool Real & Play to Perfect

Teleoperation for dexterous hands is slow, deliberate, and difficult to scale. Sim Tool Real takes a radically different approach: train entirely in GPU-accelerated simulation using reinforcement learning, then deploy zero-shot in the real world.

Key Innovation: Unify all dexterous tasks—grasping, reorientation, tool use—as goal-reaching. The policy doesn't need task labels ("sweeping," "hammering"). It only needs to move an object from its current pose to a desired pose.

Training: Procedurally generate primitive objects (cylinders, cuboids), sample random goals, and train with massively parallel RL.

Inference: Extract goal poses from a human video demonstration using foundation models (FoundationPose + SAM). The policy tracks these goals at 60 Hz, controlling a 22-DOF hand and 7-DOF arm simultaneously.

Results:

• A single frozen policy works zero-shot across 12 unseen tools and tasks
• Demonstrates extreme dexterity: spinning screwdrivers, sweeping with brushes, hammering
• Strong recovery behaviors: if the robot drops a hammer, it immediately regrasps and continues

Failure Modes: Over 60% of failures stem from pose tracking issues, particularly with small or heavily occluded objects. The policy itself is remarkably robust.

Follow-Up Work (Play to Perfect): Extends this to precise contact-rich assembly by first pre-training on task-agnostic "play," then fine-tuning on sparse-reward assembly tasks. This enables tight insertion and multi-part assembly—tasks requiring millimeter-level precision.

"A new task simply becomes a new sequence of goal poses rather than a new training run."

🏗️ The Rise of Robotics Application Companies

Presenter: Nico (CEO, Rerun)

A new category of robotics company is emerging—what some call "neo-integrators" or robotics application companies. These firms:

• Take ownership of a full business problem end-to-end
• Deploy in data centers, construction, warehouses, manufacturing
• Excel at operations, support, and deployment
• Build minimal custom hardware
• Start with teleoperation first, ensuring the business works before automating
• Fine-tune models incrementally rather than starting with foundation models

This approach mirrors the SaaS revolution—robotics application companies may become the new SaaS, transforming physical-world work at scale.

🛠️ A Practical Playbook

Step 1: Sell & Deploy Fast

• Solve a single customer problem someone will pay for
• Use teleoperation + off-the-shelf hardware
• Learn real business requirements (e.g., "need 1,000 perfect units/day," "onboarding new designs is the bottleneck")

Step 2: Set Up Learning Basics

• Fine-tune an open model (e.g., Pi) on a few hours of teleoperation data
• Build a replica environment in your office for testing (every successful robotics company has one)
• Create repeatable, domain-specific evals—encode what matters to this business, not generic benchmarks

Step 3: Collect Data That Improves the Model

• Train, evaluate, and debug constantly—do not collect all data upfront
• Iterate on data collection instructions based on model failures
• Use domain-specific tooling for multimodal, episodic, 3D-structured robotics data (traditional databases struggle here)

Step 4: Iterate & Scale

• Improve intelligence (scale data, refine algorithms, add modalities)
• Excel at sales, shipping, support, unboxing experience
• Build moats through operational excellence—not just modeling

"The market is enormous. The base models keep getting better. There is enough friction in the physical world to build real business moats. You will not need to raise a billion-dollar seed."

⚡ Making World-Action Models Run in Real Time

Presenters: Bill & Guangming (General Instinct, YC)
Work: Infrastructure for Real-Time Physical AI

World-action models—like Nvidia's DreamZero—outperform many VLAs by explicitly predicting future frames and kinematics jointly. However, they are prohibitively expensive: running DreamZero requires two GB200 GPUs (~$140K total).

General Instinct's approach:

• Distill the video encoder-decoder (VAE)
• Split the diffusion transformer (DiT) into two: one for video, one for actions, using cross-attention instead of full decoding
• Reduce sampling steps for flow matching from 50–100 steps down to 1–2 steps (50x speedup)
• Explore alternative modalities for world representation (e.g., masks, flow) instead of raw pixels

Result: World-action models running at 500 milliseconds per chunk (16 actions) on a single Jetson Orin—making real-time deployment economically viable.

"VLAs are not dead because they're smaller. But world-action models can be fast—if you build the right infrastructure."

🎯 Takeaways

1. Memory is essential for long-horizon tasks and enables in-context adaptation—a critical missing piece in current policies.
2. Embodied reasoning must be selective, not exhaustive—action-predictiveness matters more than verbosity.
3. Sim-to-real RL can achieve extreme dexterity at scale, unifying tasks as goal-reaching and bypassing teleoperation bottlenecks.
4. Robotics application companies are the future—start with teleoperation, own the full business problem, and fine-tune incrementally.
5. World-action models can be fast—with the right infrastructure, they can run in real time on edge hardware.

The path to deployable robotics is not about waiting for a single breakthrough. It's about ruthlessly iterating on the full stack—from data collection and memory architectures to domain-specific reasoning and operational excellence. The companies that master this will define the next wave of physical AI.