论文

arxiv:2609.28654
复制 Markdown

在世界模型中训练物体恒存性

发布于 9月23日
·
提交者
Hokin Deng
于 9月25日

1 今日论文

作者：
Haotian Zhang
,
Fengyuan Yu
,
Dezhi Luo
,
Haoran Sun
,
Zehong Zhao
,
Qingying Gao
,
Yihan Li
,
Siyuan An
,
Huayi Qin
,
Yilan Zhang
,
Zhengze Jiang
,
Pinyuan Feng
,
Renrui Zhang
,
Ziyu Guo
,
Letian Wang
,
Mengyue Yang
,
Kangfu Mei
,
Maijunxian Wang
,
Ran Ji
,
Vikash Kumar
,
Freda Shi
,
Chandra Sripada
+9位作者

摘要

物体恒存性和实体性是人类认知先验的标志。最近的研究表明，视频生成模型作为当前世界模型的典范类别，已展现出涌现的推理能力，使其成为构建类人物理智能的理想候选者。视频模型是否具备了涌现的物体恒存性？如果没有，我们能否使用受核心认知启发的数据集对其进行训练？我们引入了 WROP（World Reasoning with Object Permanence，基于物体恒存性的世界推理），这是一个由150个手工设计的、受认知科学启发的任务组成的数据基础设施，分为六个认知类别。我们构建了 Blender 生成器，在保持每个任务认知结构的同时随机化速度、光照、相机角度及其他干扰参数，从而为每个任务生成超过10,000个样本。我们发布了一个包含150万个样本的训练语料库和一份300道题的考试。在此考试中，我们评估了14个视频模型：3个参考到视频（reference-to-video）模型、7个编辑（edit）模型和4个续写（continuation）模型，其中包括 PWM-WROP，即我们的16B参数世界模型。在一项盲测成对 Elo 研究中，PWM-WROP 在续写模型中排名第一，总体排名第三，仅次于两个参考到视频模型之间的统计平局。我们发布了数据、考试、模型答案、分数、权重以及 PWM，这是我们在 AWS Trainium2 上的原生 PyTorch 训练栈。

查看 arXiv 页面
查看 PDF
项目页面
GitHub
8
添加到合集
社区
Hokin
论文提交者
约20小时前

物体恒存性是人类认知的基础。在此，我们展示了一个非常完整的数据基础设施，由一组非常多样化的物体恒存性认知任务组成，并且对于每个任务，我们都拥有一个基于 Blender 的数据生成器，允许将每个任务扩展至至少10,000个多样化数据样本。我们已经证明了该数据基础设施在训练具有物体恒存性的视频模型和世界模型方面的有效性。

查看翻译
回复
编辑
预览
通过拖拽、粘贴或点击此处上传图片、音频和视频。
评论

· 注册或登录以发表评论

赞成票
155
+143

在你的智能体中获取此论文：

hf papers read 2609.28654
没有最新的 CLI？
引用此论文的模型
1
Hokin/PWM-WROP
视频到视频
•
15B
•
更新于约3小时前
•
11
引用此论文的数据集
2
Hokin/object-permanence
更新于约3小时前
•
207
•
12
Hokin/object-permanence-benchmark
更新于约3小时前
•
152
•
11
引用此论文的空间
1
🫥
hugging-apps/pwm-wrop-object-permanence
包含此论文的合集
0

无包含此论文的合集

将此论文添加到合集中以从本页链接它。

Papers
arxiv:2609.28654
Copy markdown
Training Object Permanence in World Models
Published on Sep 23
·
Submitted by
Hokin Deng
on Sep 25

1 Paper of the day

Authors:
Haotian Zhang
,
Fengyuan Yu
,
Dezhi Luo
,
Haoran Sun
,
Zehong Zhao
,
Qingying Gao
,
Yihan Li
,
Siyuan An
,
Huayi Qin
,
Yilan Zhang
,
Zhengze Jiang
,
Pinyuan Feng
,
Renrui Zhang
,
Ziyu Guo
,
Letian Wang
,
Mengyue Yang
,
Kangfu Mei
,
Maijunxian Wang
,
Ran Ji
,
Vikash Kumar
,
Freda Shi
,
Chandra Sripada
+9 authors
Abstract

Object permanence and solidity are hallmarks of human cognitive priors. Recent studies show that video generation models, a paradigmatic class of current world models, have begun to show emerged reasoning abilities, making them ideal candidates for building human-like physical intelligence. Do video models have emerged object permanence in them? If not, could we train them with a core-cognition inspired dataset? We introduce WROP (World Reasoning with Object Permanence), a data infrastructure of 150 hand-designed cognitive science inspired tasks, divided into six cognitive categories. We build Blender generators that randomize speed, lighting, camera angle, and other nuisance parameters while preserving each task's cognitive structure, yielding 10,000+ samples per task. We release a 1.5M-sample training corpus and a 300-question exam. On this exam we evaluate 14 video models: 3 reference-to-video, 7 edit, and 4 continuation, among which PWM-WROP, our 16B world model. In a blind pairwise Elo study, PWM-WROP ranks first among continuation models and third overall, behind only a statistical tie between two reference-to-video models. We release the data, exam, model answers, scores, weights, and PWM, our native-PyTorch training stack on AWS Trainium2.

View arXiv page
View PDF
Project page
GitHub
8
Add to collection
Community
Hokin
Paper submitter
about 20 hours ago

Object permanence is the foundation of human cognition. Here we present a very complete data infrastructure that's composed of a very diverse set of object permanence cognitive tasks, and with each task we have a Blender-based data generator that allows one to scale each task to at least 10,000 diverse data samples. We have shown the effectiveness of this data infrastructure in training video models and world models with object permanence.

See translation
Reply
Edit
Preview
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Comment

· Sign up or log in to comment

Upvote
155
+143

Get this paper in your agent:

hf papers read 2609.28654
Don't have the latest CLI?
Models citing this paper
1
Hokin/PWM-WROP
Video-to-Video
•
15B
•
Updated about 3 hours ago
•
11
Datasets citing this paper
2
Hokin/object-permanence
Updated about 3 hours ago
•
207
•
12
Hokin/object-permanence-benchmark
Updated about 3 hours ago
•
152
•
11
Spaces citing this paper
1
🫥
hugging-apps/pwm-wrop-object-permanence
Collections including this paper
0

No Collection including this paper

Add this paper to a collection to link it from this page.