# S1: The GPT-3 Emergence Moment for Robots S1: The GPT-3 Emergence Moment for Robots Full article: https://haiguangboy.com/posts/skild_blog_introducing_s1_in_context_learning_20260817 ## Core Method ↳ Key comparison reported by the company: with pretraining scale of only 1,000 hours, in-context learning (ICL) actually loses to traditional language instructions + fine-tuning (43% vs 53%); when scaled to 100,000 hours, ICL overtakes to 66% vs 9%, a 7x gap. This is not a linear improvement with data volume; it is a curve that flips only after a certain scale is reached—the same kind of effect as the GPT-3 phenomenon in language models, where "no matter how a small model is tuned, it cannot learn strong in-context learning, but a large model suddenly can." ↳ What specifically drives this emergence is data hours, not architectural innovation—the blog does not mention any new module designed for ICL, and the model architecture is the same as before. The company's own explanation is: what pretraining truly learns is not the task itself, but the meta-ability of "how to learn from a few examples"; quantitative change only turns into qualitative change at a certain point. ## Key Results · The value of one in-context demonstration is officially estimated to be roughly equivalent to that of 380 traditional post-training demonstrations. · Real-robot demonstrations covered long-horizon unseen tasks such as planting flowers, making pancakes, and pour-over coffee, with the longest taking 10 minutes and requiring dozens of steps. · In perturbation tests, once a task requires fundamentally changing the execution plan (e.g., forced hand-switching), the performance drop of traditional language-prompted VLA models was up to 3 times that of ICL. ## Comparison of Related Routes - Opposite route · [Switching logic should not be learned by the policy](https://haiguangboy.com/posts/dr-lfd) `decompose_and_reorganize_planning_with_primitives_and_visuomotor_policies_learne_2026_08`: S1, a video-demonstration-driven in-context learning (ICL) robot manipulation foundation model, uses no language instructions and no fine-tuning, with weights unchanged throughout; this contradicts the core claim that switching logic should be handed to the planner rather than learned by the policy—a structural rebuttal to "end-to-end scaling". - Opposite route · [Robot GPT-3 moment: plenty of story, insufficient evidence](https://haiguangboy.com/posts/gen-1-5-one-shot) `generalist_blog_gen_1_5_20260818_2026_08`: the training objective itself is meta-learning: pretraining uses episodic data in which "tasks are specified only by in-context demonstrations" for the outer loop, and inference-time demonstrations drive the inner loop while weights remain unchanged—ICL is not accidentally acquired after training an ordinary policy; this contradicts the core narrative (company self-report, not third-party verified) that these capabilities emerge directly from scaled pretraining and require no specialized mechanism design. - Opposite route · [First human-to-robot zero-shot transfer scaling law](https://haiguangboy.com/posts/dyna-2-scaling-law) `dyna_blog_dyna_2_20260811_2026_08`: data-engine stance: no single data source can simultaneously dominate hardware proximity, diversity, and scalability, so multiple sources such as teleoperation and first-person video are scaled together; for every $1 spent on data collection, an additional $3 is spent on quality control; this contradicts the route bet: human first-person video, rather than teleoperation or specially collected data, is the correct scaling substrate for robot pretraining. - Same route · [1 day, 1,000 tasks, powered by inductive bias](https://haiguangboy.com/posts/mt3-thousand-tasks) `learning_a_thousand_tasks_in_a_day_2026_08`: company-reported estimate: one in-context demonstration is worth about 380 traditional post-training (fine-tuning) demonstrations; this validates the problem quantification: mainstream BC systems average 175–250 demonstrations per task, and bimanual tasks require about 8K demonstrations. - Same route · [An_Open_Foundation_Model_Towards](https://haiguangboy.com/posts/an_open_foundation_model_towards) `an_open_foundation_model_towards_2026_07`: data-engine stance: no single data source can simultaneously dominate hardware proximity, diversity, and scalability, so multiple sources such as teleoperation and first-person video are scaled together; every $1 spent on data collection incurs an additional $3 on quality control; this validates Ψ0's extreme data efficiency: 800h+30h surpasses the 10x-data baseline by 40%+. - Same route · [JEPA-WAM: An innovative approach to make world models better understand state changes](https://haiguangboy.com/posts/jepa-wam) `jepa_wam_learning_vision_language_action_policies_with_joint_embedding_world_modeling_2026_08`: company-reported internal experiments: under perturbations where action patterns fundamentally change (L5, such as forced hand-switching), language-prompted VLA performance drops by up to 3 times as much as ICL policies; this validates the problem decomposition: the latent WAM leaves only two design questions—what target to predict, and how supervision is connected to action generation. ## Limitations It does not mean these numbers have been independently verified—the 7x gap and the 380-demonstration equivalence all come from company-built benchmarks, marked as "needs independent verification"; nor does it mean scale is the only variable—in the low-data regime, traditional fine-tuning is actually better; emergence itself has a threshold, and it is not a case of "the sooner, the better"; in evaluation, policy failures can be helped by human intervention, and the reported metric is stepwise success rate, not the probability of completing the entire long-horizon task end-to-end in one attempt. ## Author's Judgment (not paper content; cross-paper synthesis) "Trade scale for emergence, without specialized mechanism design"—this technical judgment is almost identical to what another American company, Generalist AI, said in its Gen-1.5 blog. Both companies admit that they cannot explain why emergence happens, and offer only a language-model analogy as a hypothesis, with no empirical support. The two companies each independently put forward the same narrative; this looks more like a shared understanding taking shape in this field than like two independent discoveries. The real thing to watch is: will this route of "natural emergence when scale is reached" repeat the history of language models—once the emergence threshold is mapped out, the next step is a race over who can more cheaply stack data across that line.