refactor: reorganize research/ into topic subfolders (world-models, spatial-memory, crowdroom, plans)
Sync to site1 / sync (push) Has been cancelled
Sync to site1 / sync (push) Has been cancelled
This commit is contained in:
gaojie
@@ -0,0 +1,402 @@
|
||||
---
|
||||
title: "人类空间记忆：大脑如何构建、存储与更新空间信息"
|
||||
date: 2026-05-20
|
||||
draft: false
|
||||
tags: ["调研", "论文综述", "导航"]
|
||||
categories: ["research"]
|
||||
---
|
||||
|
||||
# 人类空间记忆：大脑如何构建、存储与更新空间信息
|
||||
|
||||
> 研究目标：系统梳理人类大脑构建空间认知地图的神经机制，提取可工程化的设计原则，指导 PRISM 从 v1.5 升级至 v2.0。
|
||||
>
|
||||
> 版本：v1.0 | 日期：2026-05-17
|
||||
|
||||
---
|
||||
|
||||
## 0. 一句话
|
||||
|
||||
> **大脑用「稀疏编码的位置信号 + 周期性网格度量 + 分层巩固」三件套，在几瓦功耗下实现了终身空间记忆。PRISM 2.0 要做的就是把这三件套翻译成工程架构。**
|
||||
|
||||
---
|
||||
|
||||
## 1. 核心神经系统：海马-内嗅皮层回路
|
||||
|
||||
人类空间记忆的核心硬件是**海马体 (Hippocampus)** 和**内嗅皮层 (Entorhinal Cortex)** 组成的回路。这个回路在哺乳动物中高度保守——老鼠、猴子、人类的导航系统原理相同。
|
||||
|
||||
```
|
||||
┌──────────────────────┐
|
||||
│ 内嗅皮层 (MEC) │
|
||||
│ ┌──────┬──────┬───┐ │
|
||||
│ │Grid │Head │ │ │
|
||||
│ │Cells │Dir. │… │ │
|
||||
│ └──────┴──────┴───┘ │
|
||||
└──────────┬───────────┘
|
||||
│ 输入
|
||||
┌──────────▼───────────┐
|
||||
│ 海马体 │
|
||||
│ DG → CA3 → CA1 │
|
||||
│ ┌──────┬──────┬───┐ │
|
||||
│ │Place │Time │… │ │
|
||||
│ │Cells │Cells │ │ │
|
||||
│ └──────┴──────┴───┘ │
|
||||
└──────────┬───────────┘
|
||||
│ 输出
|
||||
┌──────────▼───────────┐
|
||||
│ 新皮层 (PFC等) │
|
||||
│ 长期存储 │
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 五种基础空间细胞类型
|
||||
|
||||
### 2.1 位置细胞 (Place Cells) — 你在哪里
|
||||
|
||||
| 属性 | 说明 |
|
||||
|------|------|
|
||||
| **位置** | 海马体 CA1 / CA3 |
|
||||
| **行为** | 当动物进入特定位置时放电，每个细胞只对应一个或几个位置域 (place field) |
|
||||
| **编码** | **稀疏编码**——海马体中只有 ~2-5% 的神经元对当前位置活跃 |
|
||||
| **关键特性** | 位置域在进入新环境**几分钟内**形成，且一旦形成就相对稳定 |
|
||||
| **重映射 (Remapping)** | 环境改变后，同一群神经元会重新分配到新位置——这是"新地图"的神经信号 |
|
||||
|
||||
> **工程启发**：PRISM 应该为每个"位置"维护稀疏的视觉指纹（已有 clip_embedding），但需要支持**快速形成新位置表征 + 环境变化时触发重映射**。
|
||||
|
||||
### 2.2 网格细胞 (Grid Cells) — 走了多远、什么方向
|
||||
|
||||
| 属性 | 说明 |
|
||||
|------|------|
|
||||
| **位置** | 内侧内嗅皮层 (medial Entorhinal Cortex, MEC) |
|
||||
| **行为** | 多个空间周期性地放电，形成**正六边形网格** |
|
||||
| **编码** | 每个细胞有特定的间距 (spacing)、方向 (orientation) 和相位 (phase) |
|
||||
| **层级性** | 不同网格细胞的间距从 ~30 cm 到数米不等，**间距越大越靠 MEC 背侧** |
|
||||
| **通用性** | 网格表征**跨环境通用**——同一个网格细胞在不同房间里保持相同的间距和方向 |
|
||||
|
||||
> **工程启发**：这是 PRISM 当前**最缺的一层**。L2 的 OctoMap / TSDF 是纯笛卡尔坐标，没有网格细胞那种"泛化度量"。应该增加一个 **GridMetric 层**——由多个不同间距的周期编码叠加，实现跨房间的度量泛化。
|
||||
|
||||
### 2.3 头方向细胞 (Head Direction Cells) — 面向哪边
|
||||
|
||||
| 属性 | 说明 |
|
||||
|------|------|
|
||||
| **位置** | 多个脑区（前背侧丘脑、后下托、MEC 等） |
|
||||
| **行为** | 只对特定头部方向放电，与位置和速度无关 |
|
||||
| **编码** | 360° 环形吸引子网络，每个细胞调谐到特定角度 |
|
||||
|
||||
> **工程启发**：ZED VIO 已提供朝向，但缺少与空间记忆的直接耦合。PRISM 2.0 应该让朝向信号与位置表征**联合编码**（而非仅作为 Pose 的一个分量）。
|
||||
|
||||
### 2.4 边界细胞 (Boundary Cells) — 离墙多远
|
||||
|
||||
| 属性 | 说明 |
|
||||
|------|------|
|
||||
| **位置** | 下托 (Subiculum)、MEC |
|
||||
| **行为** | 在距环境边界特定距离处放电；部分编码**距最近墙的距离**，部分编码**特定方位的墙** |
|
||||
| **功能** | 定义环境的几何框架——没有它，位置细胞就无法稳定地重新映射 |
|
||||
|
||||
> **工程启发**：iPhone RoomPlan 输出的墙/隔断 = 天然边界。PRISM 2.0 应在 L2 初始化时显式计算每个位置的**边界距离特征**，作为位置细胞形成的基础脚手架。
|
||||
|
||||
### 2.5 速度细胞 (Speed Cells) — 移动快慢
|
||||
|
||||
| 属性 | 说明 |
|
||||
|------|------|
|
||||
| **位置** | MEC |
|
||||
| **行为** | 放电率线性正比于运行速度 |
|
||||
| **功能** | 驱动网格细胞的相位更新——没有速度信号，网格就会"静止" |
|
||||
|
||||
> **工程启发**：ZED VIO 的速度估计可直接作为网格更新的驱动力。与路径积分 (path integration) 直接对应。
|
||||
|
||||
---
|
||||
|
||||
## 3. 海马体的两个核心操作：模式分离与模式完成
|
||||
|
||||
### 3.1 模式分离 (Pattern Separation) — "这两个房间不一样"
|
||||
|
||||
```
|
||||
输入 ──→ 齿状回 (DG) ──→ CA3
|
||||
│
|
||||
大数量神经元 + 极稀疏活动
|
||||
→ 相似的输入被"正交化"为不重叠的表征
|
||||
```
|
||||
|
||||
| 场景 | 例子 |
|
||||
|------|------|
|
||||
| 两个布局相似的酒店房间 | 人不会搞混——DG 把相似的感知输入映射为不同的海马表征 |
|
||||
| 同一个房间，家具挪了 | CA3 仍然能认出（模式完成），但 DG 会标记"有变化" |
|
||||
|
||||
> **工程启发**：PRISM 需要 **DG 层**——当 ZED 进入一个与已有 L3 节点视觉相似但实际不同的房间时，DG 层应该强制新建节点而非错误地"认出"旧节点。当前只有 `clip_embedding` 做相似度匹配，缺少正交化步骤。
|
||||
|
||||
### 3.2 模式完成 (Pattern Completion) — "我只看到床角，但知道这是卧室"
|
||||
|
||||
```
|
||||
部分线索 ──→ CA3 (自联想网络) ──→ 完整记忆
|
||||
│
|
||||
CA3 有丰富的递归连接
|
||||
→ 部分输入即可激活整个记忆模式
|
||||
```
|
||||
|
||||
| 场景 | 例子 |
|
||||
|------|------|
|
||||
| 重定位 | 只看到房间一角，就能识别是哪个房间（PRISM Pipeline B 的核心需求） |
|
||||
| 弱光/遮挡 | 部分视野被遮挡，仍然能导航 |
|
||||
|
||||
> **工程启发**：PRISM 的重定位模块已经在做类似的事（CLIP embedding 匹配），但缺少 CA3 风格的**递归完成**——即用部分匹配激活的节点反过来"期望"看到哪些物品，再去 L4 验证，形成一个**双向确认**循环。
|
||||
|
||||
---
|
||||
|
||||
## 4. 记忆的三种时态：工作 → 短期 → 长期
|
||||
|
||||
人类的空间记忆不是"存进去就完事"，而是沿着**时间轴**经历了三种状态：
|
||||
|
||||
```
|
||||
感知输入
|
||||
│
|
||||
▼
|
||||
┌────────────────┐
|
||||
│ 工作记忆 (WM) │ ← 前额叶皮层 + 海马体
|
||||
│ 秒 ~ 分钟 │ 容量有限 (4±1 chunks)
|
||||
│ 当前任务上下文 │ 表征当前导航目标、最近经过的路标
|
||||
└────────┬───────┘
|
||||
│ 注意筛选 (只有"重要的"往下走)
|
||||
▼
|
||||
┌────────────────┐
|
||||
│ 短期记忆 (STM) │ ← 海马体依赖
|
||||
│ 小时 ~ 天 │ 突触可塑性 (LTP/LTD)
|
||||
│ 今天的经历 │ 快速编码，但不稳定
|
||||
└────────┬───────┘
|
||||
│ 睡眠巩固 (海马→皮层转移)
|
||||
│ 去重 + 与旧知识整合
|
||||
▼
|
||||
┌────────────────┐
|
||||
│ 长期记忆 (LTM) │ ← 新皮层（前额叶、颞叶等）
|
||||
│ 周 ~ 年 ~ 终身 │ 结构可塑性（树突棘稳定化）
|
||||
│ 稳定的知识 │ 半永久存储
|
||||
└────────────────┘
|
||||
```
|
||||
|
||||
### 三层的关键区别（PRISM 映射）
|
||||
|
||||
| 维度 | WM | STM | LTM | PRISM 当前对应 |
|
||||
|------|----|-----|-----|---------------|
|
||||
| **时间** | 秒~分钟 | 小时~天 | 周~终身 | L1/WM ✓ / L2-L4(部分) / 缺STM显式层 |
|
||||
| **容量** | 极小 | 有限 | 几乎无限 | — |
|
||||
| **可塑性** | 随时覆盖 | 可修改 | 难修改 | — |
|
||||
| **依赖** | 注意力 | 海马体 | 皮层 | — |
|
||||
| **遗忘** | 瞬时 | 干扰/衰减 | 极少 | — |
|
||||
|
||||
> **工程启发**：PRISM 目前只有 L1 (WM-like) 和"其他全算持久化"。缺失了中间的 **STM 层**——即"今天见到但还没巩固"的状态。这导致差异检测 (`delta/`) 直接写死而非可恢复的临时标记。
|
||||
|
||||
---
|
||||
|
||||
## 5. 系统巩固：睡眠中把记忆从海马搬到皮层
|
||||
|
||||
### 5.1 标准巩固模型 (Standard Consolidation Model)
|
||||
|
||||
```
|
||||
睡眠 / 安静清醒时：
|
||||
|
||||
海马体 ──Sharp-Wave Ripples (SWR)──→ 新皮层
|
||||
│ │
|
||||
以 10-20× 加速重播 缓慢调整突触权重
|
||||
白天的轨迹 把"昨天发生的事"
|
||||
变成"已知的事实"
|
||||
```
|
||||
|
||||
关键发现：
|
||||
- **海马重播 (Replay)**：清醒时经历的轨迹在 NREM 睡眠期间被压缩到 ~100 ms 的 SWR 事件中快速重放
|
||||
- **皮层巩固**：每次重播驱动皮层突触的微小变化，经多次睡眠周期后形成稳定表征
|
||||
- **去重与抽象**：大脑不是逐字记录——重播过程会**抽取共性、丢弃细节、与已有知识整合**
|
||||
|
||||
### 5.2 补充学习系统理论 (Complementary Learning Systems, CLS)
|
||||
|
||||
| | 海马体系统 | 皮层系统 |
|
||||
|---|-----------|---------|
|
||||
| **学习速度** | 快速（一次经历即可） | 慢速（需要多次重复） |
|
||||
| **表征** | 稀疏、分离（防止干扰） | 重叠、压缩（提取统计规律） |
|
||||
| **作用** | 记忆特定事件 | 学习通用知识 |
|
||||
| **类比** | 内存（快但容小） | 硬盘（慢但容大） |
|
||||
|
||||
> **工程启发**：Pipeline D（充电巩固）已经捕获了"睡眠"的直觉，但缺少两个核心操作：
|
||||
> 1. **显式重播**——不是简单地 "delta 变永久"，而是**重新走过白天的路径**，用重播后的表征更新各层
|
||||
> 2. **双系统学习**——新增物体先以"快速、稀疏、可能有噪声"的形式写入，Consolidator 再以"慢速、去噪、整合"的模式转入长期
|
||||
|
||||
---
|
||||
|
||||
## 6. 预测编码：大脑是预测机器
|
||||
|
||||
### 6.1 核心思想
|
||||
|
||||
```
|
||||
┌────── 自上而下的预测 ──────┐
|
||||
│ │
|
||||
高层表征 感知输入
|
||||
│ │
|
||||
└────── 自下而上的误差 ←──────┘
|
||||
|
||||
大脑不断生成对下一刻的预测 → 对比实际输入 → 只把"意外"（预测误差）向上传。
|
||||
```
|
||||
|
||||
### 6.2 空间导航中的预测
|
||||
|
||||
| 预测类型 | 例子 | 脑区 |
|
||||
|---------|------|------|
|
||||
| **感官预测** | "转过去应该看到沙发" | 感觉皮层 ↔ 海马体 |
|
||||
| **运动预测** | "走 10 步应该到门口" | 小脑 + MEC |
|
||||
| **物体预测** | "卧室里应该有一张床" | 前额叶 + 海马体 |
|
||||
|
||||
当预测失败（打开门发现房间被重新布置了）→ 预测误差信号 → 触发**注意 + 学习 + 记忆更新**。
|
||||
|
||||
> **工程启发**：PRISM 当前是**被动接受数据**——ZED 来了就写，差异检测只是"发现了"就记录。缺少**主动预测**层。PRISM 2.0 应该在每帧都生成"期望看到什么"，只有当偏差超过阈值才触发写入和更新。
|
||||
|
||||
---
|
||||
|
||||
## 7. 记忆更新：再巩固而非覆写
|
||||
|
||||
### 7.1 重新巩固 (Reconsolidation)
|
||||
|
||||
经典模型认为记忆"巩固后就稳定了"。2000 年后的研究发现：**已被巩固的记忆在被再次激活（回忆）时，会短暂回到不稳定状态，允许修改后再重新巩固。**
|
||||
|
||||
```
|
||||
稳定记忆 ──回忆触发──→ 不稳定状态 ──整合新信息──→ 再巩固（更新版）
|
||||
│
|
||||
└── 如果不重新巩固 → 记忆消退
|
||||
```
|
||||
|
||||
对 PRISM 的启示：
|
||||
- 当 ZED 重新观察到 L4 中的一个已知物品时 → 不应直接覆写，而是**先"解锁"该记忆，融合新观测，再"重新封印"**
|
||||
- 连续观测不一致 → 降低 confidence 而非直接删除
|
||||
- 已标记 `moved` 的物品 → 短暂进入"可修改窗口"，允许位置更新
|
||||
|
||||
### 7.2 去稳定化的条件
|
||||
|
||||
什么导致一个记忆变得可修改？
|
||||
|
||||
| 条件 | 神经机制 | PRISM 对应 |
|
||||
|------|---------|-----------|
|
||||
| **预测误差** | 打开门，床不在原位 → LC 释放去甲肾上腺素 | `delta/` 差异检测 |
|
||||
| **新奇性** | 出现不认识的物品 → 海马体 CA1 强响应 | 未匹配到 L4 节点的新检测 |
|
||||
| **上下文变化** | 同样的房间但灯光/时间不同 | 时间戳跨度大 + 场景视觉差异大 |
|
||||
|
||||
> **工程启发**：PRISM 需要显式的 **Reconsolidation Flag**——不是所有差异都立即写，而是先标记"待重新协商"，在 Consolidator 阶段才决定采纳/部分采纳/拒绝。
|
||||
|
||||
---
|
||||
|
||||
## 8. 双流视觉加工：Where 通路 vs What 通路
|
||||
|
||||
### 8.1 两条通路
|
||||
|
||||
| | 背侧通路 (Dorsal) | 腹侧通路 (Ventral) |
|
||||
|---|------------------|-------------------|
|
||||
| **俗称** | Where / How | What |
|
||||
| **路径** | 初级视皮层 → 后顶叶 | 初级视皮层 → 颞下回 |
|
||||
| **表征** | 以自我为中心 (egocentric) | 以物体为中心 (allocentric) |
|
||||
| **功能** | 引导动作、伸手抓取、避障 | 物体识别、语义理解 |
|
||||
| **速度** | 快、实时 | 慢、需要"辨认" |
|
||||
| **记忆** | 不持久 | 可持久 |
|
||||
|
||||
### 8.2 两条通路在海马体汇合
|
||||
|
||||
海马体是 where + what 的**汇合点**——它同时接收来自两条通路的信息，把它们绑定成一个"在某个地方有某个东西"的完整记忆（episodic memory 的核心）。
|
||||
|
||||
> **工程启发**：PRISM 的 L2 (度量/where) 和 L4 (语义/what) 已经体现了两条通路，但**缺少海马体式的汇合层**——即把一个物品和它的精确位置绑定为一个不可分割的 episode。当前物品的位置只是 L4Node 的一个 `pose` 字段，而非与 L2 几何深度耦合的绑定。
|
||||
|
||||
---
|
||||
|
||||
## 9. 空间注意力与显著性
|
||||
|
||||
### 9.1 大脑不记录所有东西
|
||||
|
||||
每秒钟视网膜输入 ~10^8 bit，但只有 ~10^1-10^2 bit 进入意识/记忆。大脑通过**显著性过滤器**决定什么值得记住：
|
||||
|
||||
| 过滤器 | 机制 | 例子 |
|
||||
|--------|------|------|
|
||||
| **空间新奇** | 新房间、新走廊 → 海马体强编码 | 第一次进酒店大堂 |
|
||||
| **物体新奇** | 没见过的东西 → 多巴胺释放 | 新放的装饰品 |
|
||||
| **变化检测** | 预期被违反 → 预测误差 | 椅子不在原来位置 |
|
||||
| **目标相关** | 当前任务相关的物品 → 前额叶偏置 | 找遥控器时注意茶几 |
|
||||
| **情感标记** | 情感事件 → 杏仁核调节记忆强度 | (机器人场景不适用) |
|
||||
|
||||
### 9.2 显著性 → 编码强度
|
||||
|
||||
不是所有的经历都以相同精度存储。高显著性事件 → 更强的突触可塑性 → 更持久、更精确的记忆。低显著性 → 只保留模糊的统计信息。
|
||||
|
||||
> **工程启发**：PRISM 需要一个 **Salience Gate**——不是 2 Hz 的 VLM 检测全部写入，而是根据"这个检测有多意外/多重要"动态决定**写入精度和持久性**。
|
||||
|
||||
---
|
||||
|
||||
## 10. 最新研究进展 (2020-2025)
|
||||
|
||||
### 10.1 继任表征 (Successor Representations)
|
||||
|
||||
内嗅皮层不只编码"当前位置"，还编码**从当前位置出发，未来可能访问的状态**——即对环境的"预测地图"。
|
||||
|
||||
- 一个位置的 SR 不只看那个位置本身，而是编码"从这里最容易到达哪里"
|
||||
- 这解释了为什么人能瞬间判断"从卧室到厨房要经过走廊"而不需要显式跑 A*
|
||||
|
||||
> **工程启发**：PRISM 的 L3 边目前只存 `cost` (距离/难度)。PRISM 2.0 应存储 **SR 向量**——从每个房间到所有其他房间的**期望到达频率**，让路径规划从 A* 降级为一次矩阵乘法。
|
||||
|
||||
### 10.2 物体向量细胞 (Object-Vector Cells)
|
||||
|
||||
MEC 中发现了一类新细胞：当动物处于某个物体（如障碍物）的特定方向和距离时放电。
|
||||
|
||||
- 与位置细胞不同——它不关心绝对位置，而是**相对于地标的位置**
|
||||
- 解释了为什么人能"走离沙发两米然后右转"而不需要看地图
|
||||
|
||||
> **工程启发**：PRISM 2.0 的 L2 应增加 **Landmark-Relative Coordinates**——不仅存全局位姿，还存相对于最近锚点物体 (L4 家具) 的位置。这在 GPS 不可用场景下比纯 VIO 更鲁棒。
|
||||
|
||||
### 10.3 社交位置细胞 (Social Place Cells)
|
||||
|
||||
海马体有专门编码"另一个人（或物体）所在位置"的神经元——**空间记忆天然是社会性的**。
|
||||
|
||||
> **工程启发**：多机器人场景下，PRISM 应为每个机器人/人维护一个独立的位置轨迹——"小明在厨房" 成为一个可查询的 L4 属性。
|
||||
|
||||
### 10.4 认知地图的泛化
|
||||
|
||||
2024-2025 年的研究表明，海马-内嗅系统的"认知地图"不仅编码物理空间，还编码**抽象空间**——概念距离、社交关系、甚至图表结构。同一套神经机制在不同领域重复使用。
|
||||
|
||||
> **工程启发**：PRISM 的架构可能不仅适用于物理空间——如果抽象得好，同一个 Schema 可以扩展为机器人对"任务流"、"时间线"、甚至"技能树"的记忆。
|
||||
|
||||
---
|
||||
|
||||
## 11. 总结：从大脑到 PRISM 2.0 的设计原则
|
||||
|
||||
| # | 神经机制 | 当前 PRISM 1.5 | → PRISM 2.0 | 优先级 |
|
||||
|---|---------|---------------|-------------|--------|
|
||||
| A | 网格细胞：跨环境的度量泛化 | ❌ 缺失 | 新增 GridMetric 层 | **P0** |
|
||||
| B | 模式分离 (DG)：区分相似场景 | ❌ 只有 CLIP 相似度 | 新增正交化模块 | **P0** |
|
||||
| C | 模式完成 (CA3)：部分线索→完整回忆 | ⚠️ 单向匹配 | 改为双向递归确认 | P1 |
|
||||
| D | 预测编码：预期→误差驱动更新 | ❌ 被动写入 | 新增预测层 | **P0** |
|
||||
| E | 系统巩固：海马重播 + CLS 双系统 | ⚠️ Pipeline D 太粗糙 | 显式重播 + 快慢双系统 | P1 |
|
||||
| F | 再巩固：解锁→更新→重新封印 | ❌ 直接覆写 | 引入 Reconsolidation 流程 | P1 |
|
||||
| G | 显著性门控：不是所有东西都该记 | ❌ 全量写入 | 新增 Salience Gate | P1 |
|
||||
| H | 继任表征：预测未来状态 | ❌ 只有 A* | L3 边存储 SR 向量 | P2 |
|
||||
| I | 物体向量细胞：地标相对定位 | ❌ 只有全局位姿 | L2 增加地标相对坐标 | P2 |
|
||||
| J | 边界细胞：以墙为骨架 | ⚠️ 房间 polygon | 显式计算边界距离特征 | P1 |
|
||||
| K | Where/What 汇合：位置-物品绑定 | ⚠️ 松散关联 | 紧耦合 episodic binding | P2 |
|
||||
|
||||
**P0 = 必须在 2.0 实现，P1 = 2.0 应包含，P2 = 2.1 或后续**
|
||||
|
||||
---
|
||||
|
||||
## 12. 关键参考
|
||||
|
||||
| 发现 | 年份 | 核心贡献 | 诺贝尔奖 |
|
||||
|------|------|---------|---------|
|
||||
| 位置细胞 (O'Keefe & Dostrovsky) | 1971 | 海马体编码空间位置 | 2014 |
|
||||
| 认知地图 (Tolman) | 1948 | 动物形成内部空间表征，不仅靠刺激-反应 | — |
|
||||
| 网格细胞 (Hafting, Fyhn, Molden, Moser & Moser) | 2005 | 内嗅皮层六边形网格编码 | 2014 |
|
||||
| 头方向细胞 (Taube, Muller, Ranck) | 1990 | 独立于位置的方向编码 | — |
|
||||
| 边界细胞 (Solstad et al. / Lever et al.) | 2008 | 距边界特定距离放电 | — |
|
||||
| 系统巩固 (McClelland, McNaughton, O'Reilly) | 1995 | 互补学习系统 (CLS) 理论 | — |
|
||||
| 海马重播 (Wilson & McNaughton / Skaggs & McNaughton) | 1994-1996 | 睡眠 SWR 中重播清醒轨迹 | — |
|
||||
| 再巩固 (Nader, Schafe, LeDoux) | 2000 | 已巩固记忆可被重新不稳定化 | — |
|
||||
| 预测编码 (Rao & Ballard / Friston) | 1999-2005 | 大脑通过预测误差驱动学习 | — |
|
||||
| 继任表征 (Stachenfeld, Botvinick, Gershman / Momennejad et al.) | 2017-2022 | 内嗅皮层编码预测性地图 | — |
|
||||
| 物体向量细胞 (Høydal, Skytøen, Andersson, Moser & Moser) | 2019 | 相对地标的方向和距离编码 | — |
|
||||
| Lyra 2.0 (Shen et al., NVIDIA) | 2026 | 几何只做路由不做合成 | — |
|
||||
|
||||
---
|
||||
|
||||
**文档版本**：v1.0
|
||||
**撰写日期**：2026-05-17
|
||||
**下一步**：将此文档中的 P0/P1 原则转化为 [`plans/PRISM/20_v2_upgrade.md`](../plans/PRISM/20_v2_upgrade.md) 的具体架构改动
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
title: "空间记忆与感知综述"
|
||||
date: 2026-05-20
|
||||
draft: false
|
||||
tags: ["spatial-memory", "ZED2i", "VIO", "综述"]
|
||||
categories: ["research"]
|
||||
description: "空间记忆与感知方向的论文综述，涵盖人类空间记忆机制、ZED 2i 数据管线相关 arXiv 论文。"
|
||||
---
|
||||
|
||||
## 空间记忆与感知综述
|
||||
|
||||
本目录收录空间记忆、视觉惯性里程计（VIO）与神经建图方向的论文综述。
|
||||
|
||||
| 文章 | 简介 |
|
||||
|------|------|
|
||||
| [人类空间记忆综述](human_spatial_memory/) | 神经科学视角的人类空间记忆机制综述 |
|
||||
| [ZED 2i 实时 arXiv 综述](zed2i_arxiv_live_review/) | SLAM、VIO 与神经建图最新进展追踪 |
|
||||
| [双目立体+VIO+神经建图综述](zed2i_stereo_vio_arxiv_review/) | 服务于 ZED 2i 数据管线的专项综述 |
|
||||
@@ -0,0 +1,750 @@
|
||||
---
|
||||
title: "ZED 2i 数据 Pipeline 实时 arXiv 综述（自动生成）"
|
||||
date: 2026-05-20
|
||||
draft: false
|
||||
tags: ["调研", "论文综述", "SLAM", "VIO", "Gaussian Splatting", "API"]
|
||||
categories: ["research"]
|
||||
---
|
||||
|
||||
# ZED 2i 数据 Pipeline 实时 arXiv 综述（自动生成）
|
||||
|
||||
> **数据来源**：[`research/search_results.json`](search_results.json) 由 [`research/search_info.py`](search_info.py) 通过 HTTP 代理 `127.0.0.1:6984` 拉取自 arxiv.org / api.github.com。
|
||||
> **生成时间**：2026-05-16T16:44:34+0800 (merged)
|
||||
> **检索代理**：http://127.0.0.1:6984
|
||||
> **每主题最多**：10 篇
|
||||
|
||||
## 0. 数据概览
|
||||
|
||||
- **arXiv 论文总数**：100
|
||||
- **arXiv 主题数**：10
|
||||
- **疑似国产团队论文**：1（占比 1%；🇨🇳 标记，启发式判断）
|
||||
- **GitHub 仓库总数**：50
|
||||
- **GitHub 主题数**：5
|
||||
|
||||
### 0.1 主类目分布（arXiv primary_category）
|
||||
|
||||
- `cs.CV`: 59
|
||||
- `cs.RO`: 31
|
||||
- `eess.SY`: 2
|
||||
- `cs.AI`: 2
|
||||
- `quant-ph`: 1
|
||||
- `cs.LG`: 1
|
||||
- `cs.MA`: 1
|
||||
- `eess.SP`: 1
|
||||
- `cs.HC`: 1
|
||||
- `cs.SD`: 1
|
||||
|
||||
### 0.2 与本项目框架的映射
|
||||
|
||||
| 项目阶段 | 主线主题 | 论文数 |
|
||||
|---|---|---|
|
||||
| M2-3 / M3-4 | A. 双目立体匹配（被动深度） | 10 |
|
||||
| M2-1 / M3-3 | B. 视觉惯性 SLAM / VIO | 10 |
|
||||
| M3-5 / M4 | C. 3D Gaussian Splatting SLAM（融合建图） | 10 |
|
||||
| M3-4 | D. 单目深度基础模型 | 10 |
|
||||
| M4-1 / M4-2 | E. 室内 RGB-D 数据集与重建 | 10 |
|
||||
| M4-4 | F. 视频世界模型（下游应用） | 10 |
|
||||
| 全周期 | G. ZED 相机相关应用工作 | 10 |
|
||||
| 国产化 / 替代硬件 | H. Orbbec / Femto / Azure Kinect 相关工作 | 10 |
|
||||
| M2-2 / M3-5 | I. RGB-D 室内重建 | 10 |
|
||||
| M3-4 | J. 神经立体深度（指定 RAFT/IGEV/Foundation 家族） | 10 |
|
||||
|
||||
---
|
||||
|
||||
# 第一部分 · arXiv 论文（按主题分组，时间新→旧）
|
||||
|
||||
## A. 双目立体匹配（被动深度）
|
||||
**项目阶段**: M2-3 / M3-4 | **论文数**: 10
|
||||
|
||||
对应 ZED 双目深度算法的替换/超越路线。关注零样本泛化、Transformer 架构、神经几何编码。
|
||||
|
||||
#### 1. [2605.14963](https://arxiv.org/abs/2605.14963) — H-OmniStereo: Zero-Shot Omnidirectional Stereo Matching with Heading-Aligned Normal Priors
|
||||
- **发表**: 2026-05-14 | **分类**: cs.CV
|
||||
- **作者**: Chenxing Jiang, Zhe Tong, Pusen Gao, Peize Liu 等 (8 人)
|
||||
- **摘要**: Stereo matching on top-bottom equirectangular images provides an effective framework for full-surround perception, as vertically aligned epipolar lines enable the use of advanced perspective stereo architectures that are largely driven by large-scale datasets and monocular priors. However, the performance of such adaptations is severely limited by the scarcity of omnidirectional stereo datasets an...
|
||||
|
||||
#### 2. [2605.08592](https://arxiv.org/abs/2605.08592) — Cross-Modal RGB-D Fusion Transformer for 6D Pose Estimation of Non-Cooperative Spacecraft with Stereo-Derived Depth
|
||||
- **发表**: 2026-05-09 | **分类**: cs.CV
|
||||
- **作者**: Yongliang Zhen, Bo LÜ, Hang Yang, Xiaotian WU
|
||||
- **摘要**: On-orbit servicing and active debris removal involving non-cooperative spacecraft require reliable pose estimation to supply accurate position and orientation data for autonomous visual navigation. Learning-based monocular methods have seen widespread adoption in spacecraft pose estimation, yet they suffer from an intrinsic depth ambiguity problem and tend to fail under the harsh illumination cond...
|
||||
|
||||
#### 3. [2604.20393](https://arxiv.org/abs/2604.20393) — MLG-Stereo: ViT Based Stereo Matching with Multi-Stage Local-Global Enhancement
|
||||
- **发表**: 2026-04-22 | **分类**: cs.CV
|
||||
- **作者**: Haoyu Zhang, Jingyi Zhou, Peng Ye, Jiakang Yuan 等 (7 人)
|
||||
- **摘要**: With the development of deep learning, ViT-based stereo matching methods have made significant progress due to their remarkable robustness and zero-shot ability. However, due to the limitations of ViTs in handling resolution sensitivity and their relative neglect of local information, the ability of ViT-based methods to predict details and handle arbitrary-resolution images is still weaker than th...
|
||||
|
||||
#### 4. [2604.10218](https://arxiv.org/abs/2604.10218) — SMFormer: Empowering Self-supervised Stereo Matching via Foundation Models and Data Augmentation
|
||||
- **发表**: 2026-04-11 | **分类**: cs.CV
|
||||
- **作者**: Yun Wang, Zhengjie Yang, Jiahao Zheng, Zhanjie Zhang 等 (6 人)
|
||||
- **摘要**: Recent self-supervised stereo matching methods have made significant progress. They typically rely on the photometric consistency assumption, which presumes corresponding points across views share the same appearance. However, this assumption could be compromised by real-world disturbances, resulting in invalid supervisory signals and a significant accuracy gap compared to supervised methods. To a...
|
||||
|
||||
#### 5. [2604.09142](https://arxiv.org/abs/2604.09142) — Geometry Reinforced Efficient Attention Tuning Equipped with Normals for Robust Stereo Matching
|
||||
- **发表**: 2026-04-10 | **分类**: cs.CV
|
||||
- **作者**: Jiahao Li, Xinhong Chen, Zhengmin Jiang, Cheng Huang 等 (6 人)
|
||||
- **摘要**: Despite remarkable advances in image-driven stereo matching over the past decade, Synthetic-to-Realistic Zero-Shot (Syn-to-Real) generalization remains an open challenge. This suboptimal generalization performance mainly stems from cross-domain shifts and ill-posed ambiguities inherent in image textures, particularly in occluded, textureless, repetitive, and non-Lambertian (specular/transparent) r...
|
||||
|
||||
#### 6. [2603.29368](https://arxiv.org/abs/2603.29368) — StereoVGGT: A Training-Free Visual Geometry Transformer for Stereo Vision
|
||||
- **发表**: 2026-03-31 | **分类**: cs.CV
|
||||
- **作者**: Ziyang Chen, Yansong Qu, You Shen, Xuan Cheng 等 (5 人)
|
||||
- **摘要**: Driven by the advancement of 3D devices, stereo vision tasks including stereo matching and stereo conversion have emerged as a critical research frontier. Contemporary stereo vision backbones typically rely on either monocular depth estimation (MDE) models or visual foundation models (VFMs). Crucially, these models are predominantly pretrained without explicit supervision of camera poses. Given th...
|
||||
|
||||
#### 7. [2603.24836](https://arxiv.org/abs/2603.24836) — WAFT-Stereo: Warping-Alone Field Transforms for Stereo Matching
|
||||
- **发表**: 2026-03-25 | **分类**: cs.CV
|
||||
- **作者**: Yihan Wang, Jia Deng
|
||||
- **摘要**: We introduce WAFT-Stereo, a simple and effective warping-based method for stereo matching. WAFT-Stereo demonstrates that cost volumes, a common design used in many leading methods, are not necessary for strong performance and can be replaced by warping with improved efficiency. WAFT-Stereo ranks first on ETH3D (BP-0.5), Middlebury (RMSE), and KITTI (all metrics), reducing the zero-shot error by 81...
|
||||
|
||||
#### 8. [2603.21882](https://arxiv.org/abs/2603.21882) — Deep S2P: Integrating Learning Based Stereo Matching Into the Satellite Stereo Pipeline
|
||||
- **发表**: 2026-03-23 | **分类**: cs.CV
|
||||
- **作者**: Elías Masquil, Thibaud Ehret, Pablo Musé, Gabriele Facciolo
|
||||
- **摘要**: Digital Surface Model generation from satellite imagery is a core task in Earth observation and is commonly addressed using classical stereoscopic matching algorithms in satellite pipelines as in the Satellite Stereo Pipeline (S2P). While recent learning-based stereo matchers achieve state-of-the-art performance on standard benchmarks, their integration into operational satellite pipelines remains...
|
||||
|
||||
#### 9. [2603.15019](https://arxiv.org/abs/2603.15019) — Reference-Free Omnidirectional Stereo Matching via Multi-View Consistency Maximization
|
||||
- **发表**: 2026-03-16 | **分类**: cs.CV
|
||||
- **作者**: Lehuai Xu, Weiming Zhang, Yang Li, Sidan Du 等 (5 人)
|
||||
- **摘要**: Reliable omnidirectional depth estimation from multi-fisheye stereo matching is pivotal to many applications, such as embodied robotics. Existing approaches either rely on spherical sweeping with heuristic fusion strategies to build the cost columns or perform reference-centric stereo matching based on rectified views. However, these methods fail to explicitly exploit geometric relationships betwe...
|
||||
|
||||
#### 10. [2603.01650](https://arxiv.org/abs/2603.01650) — PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts
|
||||
- **发表**: 2026-03-02 | **分类**: cs.CV
|
||||
- **作者**: Xianqi Wang, Hao Yang, Hangtian Wang, Junda Cheng 等 (7 人)
|
||||
- **摘要**: Modern stereo matching methods have leveraged monocular depth foundation models to achieve superior zero-shot generalization performance. However, most existing methods primarily focus on extracting robust features for cost volume construction or disparity initialization. At the same time, the iterative refinement stage, which is also crucial for zero-shot generalization, remains underexplored. So...
|
||||
|
||||
---
|
||||
|
||||
## B. 视觉惯性 SLAM / VIO
|
||||
**项目阶段**: M2-1 / M3-3 | **论文数**: 10
|
||||
|
||||
对标 ZED 内建 VIO 的替代方案。关注 IMU 融合、长时鲁棒、动态环境。
|
||||
|
||||
#### 1. [2605.07552](https://arxiv.org/abs/2605.07552) — VIMCAN: Visual-Inertial 3D Human Pose Estimation with Hybrid Mamba-Cross-Attention Network
|
||||
- **发表**: 2026-05-08 | **分类**: cs.CV
|
||||
- **作者**: Zepeng Yang, Junxuan Bai, Hao Li, Ju Dai 等 (7 人)
|
||||
- **摘要**: The rapid advances in deep learning have significantly enhanced the accuracy of multimodal 3D human pose estimation (HPE). However, the state-of-the-art (SOTA) HPE pipelines still rely on Transformers, whose quadratic complexity makes real-time processing for long sequences impractical. Mamba addresses this issue through selective state-space modeling, enabling efficient sequence processing withou...
|
||||
|
||||
#### 2. [2605.02054](https://arxiv.org/abs/2605.02054) — Observability Conditions and Filter Design for Visual Pose Estimation via Dual Quaternions
|
||||
- **发表**: 2026-05-03 | **分类**: eess.SY, cs.CV, cs.RO
|
||||
- **作者**: Nicholas B. Andrews, Kristi A. Morgansen
|
||||
- **摘要**: This paper presents a dual quaternion framework for 6-DOF visual target tracking that addresses key limitations of perspective-n-point (P$n$P) solvers: sensitivity to noise and outliers, and inability to propagate estimates through measurement dropouts. A nonlinear observability analysis is performed using a Lie algebraic approach, deriving sufficient conditions for local observability under two s...
|
||||
|
||||
#### 3. [2604.07151](https://arxiv.org/abs/2604.07151) — An RTK-SLAM Dataset for Absolute Accuracy Evaluation in GNSS-Degraded Environments
|
||||
- **发表**: 2026-04-08 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Wei Zhang, Vincent Ress, David Skuddis, Uwe Soergel 等 (5 人)
|
||||
- **摘要**: RTK-SLAM systems integrate simultaneous localization and mapping (SLAM) with real-time kinematic (RTK) GNSS positioning, promising both relative consistency and globally referenced coordinates for efficient georeferenced surveying. A critical and underappreciated issue is that the standard evaluation metric, Absolute Trajectory Error (ATE), first fits an optimal rigid-body transformation between t...
|
||||
|
||||
#### 4. [2603.21785](https://arxiv.org/abs/2603.21785) — Image-Conditioned Adaptive Parameter Tuning for Visual Odometry Frontends
|
||||
- **发表**: 2026-03-23 | **分类**: cs.CV
|
||||
- **作者**: Simone Nascivera, Leonard Bauersfeld, Jeff Delaune, Davide Scaramuzza
|
||||
- **摘要**: Resource-constrained autonomous robots rely on sparse direct and semi-direct visual-(inertial)-odometry (VO) pipelines, as they provide a favorable tradeoff between accuracy, robustness, and computational cost. However, the performance of most systems depends critically on hand-tuned hyperparameters governing feature detection, tracking, and outlier rejection. These parameters are typically fixed ...
|
||||
|
||||
#### 5. [2603.20778](https://arxiv.org/abs/2603.20778) — PiLoT: Neural Pixel-to-3D Registration for UAV-based Ego and Target Geo-localization
|
||||
- **发表**: 2026-03-21 | **分类**: cs.CV
|
||||
- **作者**: Xiaoya Cheng, Long Wang, Yan Liu, Xinyi Liu 等 (8 人)
|
||||
- **摘要**: We present PiLoT, a unified framework that tackles UAV-based ego and target geo-localization. Conventional approaches rely on decoupled pipelines that fuse GNSS and Visual-Inertial Odometry (VIO) for ego-pose estimation, and active sensors like laser rangefinders for target localization. However, these methods are susceptible to failure in GNSS-denied environments and incur substantial hardware co...
|
||||
|
||||
#### 6. [2603.19654](https://arxiv.org/abs/2603.19654) — GravCal: Single-Image Calibration of IMU Gravity Priors with Per-Sample Confidence
|
||||
- **发表**: 2026-03-20 | **分类**: cs.CV
|
||||
- **作者**: Haichao Zhu, Qian Zhang
|
||||
- **摘要**: Gravity estimation is fundamental to visual-inertial perception, augmented reality, and robotics, yet gravity priors from IMUs are often unreliable under linear acceleration, vibration, and transient motion. Existing methods often estimate gravity directly from images or assume reasonably accurate inertial input, leaving the practical problem of correcting a noisy gravity prior from a single image...
|
||||
|
||||
#### 7. [2603.17229](https://arxiv.org/abs/2603.17229) — Visual SLAM with DEM Anchoring for Lunar Surface Navigation
|
||||
- **发表**: 2026-03-18 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Adam Dai, Guillem Casadesus Vila, Grace Gao
|
||||
- **摘要**: Future lunar missions will require autonomous rovers capable of traversing tens of kilometers across challenging terrain while maintaining accurate localization and producing globally consistent maps. However, the absence of global positioning systems, extreme illumination, and low-texture regolith make long-range navigation on the Moon particularly difficult, as visual-inertial odometry pipelines...
|
||||
|
||||
#### 8. [2603.26685](https://arxiv.org/abs/2603.26685) — Contextual Graph Representations for Task-Driven 3D Perception and Planning
|
||||
- **发表**: 2026-03-12 | **分类**: cs.RO, cs.AI, cs.CV
|
||||
- **作者**: Christopher Agia
|
||||
- **摘要**: Recent advances in computer vision facilitate fully automatic extraction of object-centric relational representations from visual-inertial data. These state representations, dubbed 3D scene graphs, are a hierarchical decomposition of real-world scenes with a dense multiplex graph structure. While 3D scene graphs claim to promote efficient task planning for robot systems, they contain numerous obje...
|
||||
|
||||
#### 9. [2603.11085](https://arxiv.org/abs/2603.11085) — Edge-Assisted Multi-Robot Visual-Inertial SLAM with Efficient Communication
|
||||
- **发表**: 2026-03-11 | **分类**: cs.RO, cs.CV, cs.MA
|
||||
- **作者**: Xin Liu, Shuhuan Wen, Jing Zhao, Tony Z. Qiu 等 (5 人)
|
||||
- **摘要**: The integration of cloud computing and edge computing is an effective way to achieve global consistent and real-time multi-robot Simultaneous Localization and Mapping (SLAM). Cloud computing effectively solves the problem of limited computing, communication and storage capacity of terminal equipment. However, limited bandwidth and extremely long communication links between terminal devices and the...
|
||||
|
||||
#### 10. [2603.09653](https://arxiv.org/abs/2603.09653) — OTPL-VIO: Robust Visual-Inertial Odometry with Optimal Transport Line Association and Adaptive Uncertainty
|
||||
- **发表**: 2026-03-10 | **分类**: cs.CV, cs.RO
|
||||
- **作者**: Zikun Chen, Wentao Zhao, Yihe Niu, Tianchen Deng 等 (5 人)
|
||||
- **摘要**: Robust stereo visual-inertial odometry (VIO) remains challenging in low-texture scenes and under abrupt illumination changes, where point features become sparse and unstable, leading to ambiguous association and under-constrained estimation. Line structures offer complementary geometric cues, yet many efficient point-line systems still rely on point-guided line association, which can break down wh...
|
||||
|
||||
---
|
||||
|
||||
## C. 3D Gaussian Splatting SLAM（融合建图）
|
||||
**项目阶段**: M3-5 / M4 | **论文数**: 10
|
||||
|
||||
把 3DGS 作为 SLAM 后端，实现实时定位+建图+渲染一体化。world model 训练的核心视觉表征。
|
||||
|
||||
#### 1. [2605.10760](https://arxiv.org/abs/2605.10760) — MAGS-SLAM: Monocular Multi-Agent Gaussian Splatting SLAM for Geometrically and Photometrically Consistent Reconstruction
|
||||
- **发表**: 2026-05-11 | **分类**: cs.RO
|
||||
- **作者**: Zhihao Cao, Qi Shao, Shuhao Zhai, Jing Zhang 等 (6 人)
|
||||
- **摘要**: Collaborative photorealistic 3D reconstruction from multiple agents enables rapid large-scale scene capture for virtual production and cooperative multi-robot exploration. While recent 3D Gaussian Splatting (3DGS) SLAM algorithms can generate high-fidelity real-time mapping, most of the existing multi-agent Gaussian SLAM methods still rely on RGB-D sensors to obtain metric depth and simplify cross...
|
||||
|
||||
#### 2. [2604.22339](https://arxiv.org/abs/2604.22339) — Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM
|
||||
- **发表**: 2026-04-24 | **分类**: cs.CV
|
||||
- **作者**: Yunsong Wang, Gim Hee Lee
|
||||
- **摘要**: Handling the dynamic environments is a significant research challenge in Visual Simultaneous Localization and Mapping (SLAM). Recent research combines 3D Gaussian Splatting (3DGS) with SLAM to achieve both robust camera pose estimation and photorealistic renderings. However, using SLAM to efficiently reconstruct both static and dynamic regions remains challenging. In this work, we propose an effic...
|
||||
|
||||
#### 3. [2604.15612](https://arxiv.org/abs/2604.15612) — GaussianFlow SLAM: Monocular Gaussian Splatting SLAM Guided by GaussianFlow
|
||||
- **发表**: 2026-04-17 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Dong-Uk Seo, Jinwoo Jeon, Eungchang Mason Lee, Hyun Myung
|
||||
- **摘要**: Gaussian splatting has recently gained traction as a compelling map representation for SLAM systems, enabling dense and photo-realistic scene modeling. However, its application to monocular SLAM remains challenging due to the lack of reliable geometric cues from monocular input. Without geometric supervision, mapping or tracking could fall in local-minima, resulting in structural degeneracies and ...
|
||||
|
||||
#### 4. [2604.13492](https://arxiv.org/abs/2604.13492) — RadarSplat-RIO: Indoor Radar-Inertial Odometry with Gaussian Splatting-Based Radar Bundle Adjustment
|
||||
- **发表**: 2026-04-15 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Pou-Chun Kung, Yuan Tian, Zhengqin Li, Yue Liu 等 (7 人)
|
||||
- **摘要**: Radar is more resilient to adverse weather and lighting conditions than visual and Lidar simultaneous localization and mapping (SLAM). However, most radar SLAM pipelines still rely heavily on frame-to-frame odometry, which leads to substantial drift. While loop closure can correct long-term errors, it requires revisiting places and relies on robust place recognition. In contrast, visual odometry m...
|
||||
|
||||
#### 5. [2604.12942](https://arxiv.org/abs/2604.12942) — RMGS-SLAM: Real-time Multi-sensor Gaussian Splatting SLAM
|
||||
- **发表**: 2026-04-14 | **分类**: cs.RO
|
||||
- **作者**: Dongen Li, Yi Liu, Junqi Liu, Zewen Sun 等 (11 人)
|
||||
- **摘要**: Achieving real-time Simultaneous Localization and Mapping (SLAM) based on 3D Gaussian splatting (3DGS) in large-scale real-world environments remains challenging, as existing methods still struggle to jointly achieve low-latency pose estimation, continuous 3D Gaussian reconstruction, and long-term global consistency. In this paper, we present a tightly coupled LiDAR-Inertial-Visual 3DGS-based SLAM...
|
||||
|
||||
#### 6. [2604.12837](https://arxiv.org/abs/2604.12837) — GGD-SLAM: Monocular 3DGS SLAM Powered by Generalizable Motion Model for Dynamic Environments
|
||||
- **发表**: 2026-04-14 | **分类**: cs.RO
|
||||
- **作者**: Yi Liu, Haoxuan Xu, Hongbo Duan, Keyu Fan 等 (8 人)
|
||||
- **摘要**: Visual SLAM algorithms achieve significant improvements through the exploration of 3D Gaussian Splatting (3DGS) representations, particularly in generating high-fidelity dense maps. However, they depend on a static environment assumption and experience significant performance degradation in dynamic environments. This paper presents GGD-SLAM, a framework that employs a generalizable motion model to...
|
||||
|
||||
#### 7. [2604.11992](https://arxiv.org/abs/2604.11992) — ReefMapGS: Enabling Large-Scale Underwater Reconstruction by Closing the Loop Between Multimodal SLAM and Gaussian Splatting
|
||||
- **发表**: 2026-04-13 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Daniel Yang, Jungseok Hong, John J. Leonard, Yogesh Girdhar
|
||||
- **摘要**: 3D Gaussian Splatting is a powerful visual representation, providing high-quality and efficient 3D scene reconstruction, but it is crucially dependent on accurate camera poses typically obtained from computationally intensive processes like structure-from-motion that are unsuitable for field robot applications. However, in these domains, multimodal sensor data from acoustic, inertial, pressure, an...
|
||||
|
||||
#### 8. [2604.10593](https://arxiv.org/abs/2604.10593) — MonoEM-GS: Monocular Expectation-Maximization Gaussian Splatting SLAM
|
||||
- **发表**: 2026-04-12 | **分类**: cs.RO
|
||||
- **作者**: Evgenii Kruzhkov, Sven Behnke
|
||||
- **摘要**: Feed-forward geometric foundation models can infer dense point clouds and camera motion directly from RGB streams, providing priors for monocular SLAM. However, their predictions are often view-dependent and noisy: geometry can vary across viewpoints and under image transformations, and local metric properties may drift between frames. We present MonoEM-GS, a monocular mapping pipeline that integr...
|
||||
|
||||
#### 9. [2604.03092](https://arxiv.org/abs/2604.03092) — Flash-Mono: Feed-Forward Accelerated Gaussian Splatting Monocular SLAM
|
||||
- **发表**: 2026-04-03 | **分类**: cs.RO
|
||||
- **作者**: Zicheng Zhang, Ke Wu, Xiangting Meng, Keyu Liu 等 (6 人)
|
||||
- **摘要**: Monocular 3D Gaussian Splatting SLAM suffers from critical limitations in time efficiency, geometric accuracy, and multi-view consistency. These issues stem from the time-consuming $\textit{Train-from-Scratch}$ optimization and the lack of inter-frame scale consistency from single-frame geometry priors. We contend that a feed-forward paradigm, leveraging multi-frame context to predict Gaussian att...
|
||||
|
||||
#### 10. [2604.02696](https://arxiv.org/abs/2604.02696) — VBGS-SLAM: Variational Bayesian Gaussian Splatting Simultaneous Localization and Mapping
|
||||
- **发表**: 2026-04-03 | **分类**: cs.CV, cs.RO
|
||||
- **作者**: Yuhan Zhu, Yanyu Zhang, Jie Xu, Wei Ren
|
||||
- **摘要**: 3D Gaussian Splatting (3DGS) has shown promising results for 3D scene modeling using mixtures of Gaussians, yet its existing simultaneous localization and mapping (SLAM) variants typically rely on direct, deterministic pose optimization against the splat map, making them sensitive to initialization and susceptible to catastrophic forgetting as map evolves. We propose Variational Bayesian Gaussian ...
|
||||
|
||||
---
|
||||
|
||||
## D. 单目深度基础模型
|
||||
**项目阶段**: M3-4 | **论文数**: 10
|
||||
|
||||
Depth Anything / Marigold / Metric3D / UniDepth 等通用深度模型，作为双目深度失效的兜底。
|
||||
|
||||
#### 1. [2605.11756](https://arxiv.org/abs/2605.11756) — Focusable Monocular Depth Estimation
|
||||
- **发表**: 2026-05-12 | **分类**: cs.CV, cs.AI
|
||||
- **作者**: Yuxin Du, Tao Lin, Zile Zhong, Runting Li 等 (10 人)
|
||||
- **摘要**: Monocular depth foundation models generalize well across scenes, yet they are typically optimized with uniform pixel-wise objectives that do not distinguish user-specified or task-relevant target regions from the surrounding context. We therefore introduce Focusable Monocular Depth Estimation (FDE), a region-aware depth estimation task in which, given a specified target region, the model is requir...
|
||||
|
||||
#### 2. [2605.07264](https://arxiv.org/abs/2605.07264) — Sat3R: Satellite DSM Reconstruction via RPC-Aware Depth Fine-tuning
|
||||
- **发表**: 2026-05-08 | **分类**: cs.CV
|
||||
- **作者**: Qiaoyi Yang, Chaoyi Zhou, Xi Liu, Run Wang 等 (12 人)
|
||||
- **摘要**: Accurate Digital Surface Model (DSM) reconstruction from satellite imagery is critical for applications such as disaster response, urban planning, and large-scale geographic mapping. Existing approaches face a fundamental trade-off: optimization-based methods achieve strong accuracy but require hours of per-scene computation, while generalizable geometry foundation models offer near-instant infere...
|
||||
|
||||
#### 3. [2605.06270](https://arxiv.org/abs/2605.06270) — Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction
|
||||
- **发表**: 2026-05-07 | **分类**: cs.CV
|
||||
- **作者**: Zecheng Tang, Jiaye Fu, Qiankun Gao, Haijie Li 等 (8 人)
|
||||
- **摘要**: Feed-forward 3D reconstruction models based on Vision Transformers can directly estimate scene geometry and camera poses from a small set of input images, but scaling them to video inputs with hundreds or thousands of frames remains challenging due to the quadratic cost of global attention layers. Recent token-merging methods accelerate these models by compressing the token sequence within the glo...
|
||||
|
||||
#### 4. [2605.04566](https://arxiv.org/abs/2605.04566) — Open-Source Image Editing Models Are Zero-Shot Vision Learners
|
||||
- **发表**: 2026-05-06 | **分类**: cs.CV, cs.CL
|
||||
- **作者**: Wei Liu, Jiaxin Lin, Rui Chen
|
||||
- **摘要**: Recent studies have shown that large generative models can solve vision tasks they were not explicitly trained for. However, existing evidence relies on closed-source models~(Veo~3, Nano Banana Pro) or requires task-specific instruction tuning, leaving open whether publicly available image-editing models possess zero-shot vision abilities out of the box. We conduct a systematic evaluation of thr...
|
||||
|
||||
#### 5. [2604.26567](https://arxiv.org/abs/2604.26567) — AirZoo: A Unified Large-Scale Dataset for Grounding Aerial Geometric 3D Vision
|
||||
- **发表**: 2026-04-29 | **分类**: cs.CV
|
||||
- **作者**: Xiaoya Cheng, Rouwan Wu, Xinyi Liu, Zeyu Cui 等 (9 人)
|
||||
- **摘要**: Despite the rapid progress in data-driven 3D vision, aerial geometric 3D vision remains a formidable challenge due to the severe scarcity of large-scale, high-fidelity training data. Existing benchmarks, predominantly biased toward ground-level or object-centric views, do not account for complex viewpoint transformations and diverse environmental conditions in UAV-based sensing. To bridge this cri...
|
||||
|
||||
#### 6. [2604.23432](https://arxiv.org/abs/2604.23432) — Sphere-Depth: A Benchmark for Depth Estimation Methods with Varying Spherical Camera Orientations
|
||||
- **发表**: 2026-04-25 | **分类**: cs.CV, cs.AI
|
||||
- **作者**: Soulayma Gazzeh, Giuseppe Mazzola, Liliana Lo Presti, Marco La Cascia
|
||||
- **摘要**: Reliable depth estimation from spherical images is crucial for 360° vision in robotic navigation and immersive scene understanding. However, the onboard spherical camera can experience unintentional pose variations in real-world robotic platforms that, along with the geometric distortions inherent in equirectangular projections, significantly impact the effectiveness of depth estimation. To study ...
|
||||
|
||||
#### 7. [2604.20329](https://arxiv.org/abs/2604.20329) — Image Generators are Generalist Vision Learners
|
||||
- **发表**: 2026-04-22 | **分类**: cs.CV, cs.AI
|
||||
- **作者**: Valentin Gabeur, Shangbang Long, Songyou Peng, Paul Voigtlaender 等 (25 人)
|
||||
- **摘要**: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining. While it has long been conjectured that the ability to create visual content implies an ability to understand it, there has been limited evidence that generative vision...
|
||||
|
||||
#### 8. [2604.18336](https://arxiv.org/abs/2604.18336) — Enhancing Glass Surface Reconstruction via Depth Prior for Robot Navigation
|
||||
- **发表**: 2026-04-20 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Jiamin Zheng, Jingwen Yu, Guangcheng Chen, Hong Zhang
|
||||
- **摘要**: Indoor robot navigation is often compromised by glass surfaces, which severely corrupt depth sensor measurements. While foundation models like Depth Anything 3 provide excellent geometric priors, they lack an absolute metric scale. We propose a training-free framework that leverages depth foundation models as a structural prior, employing a robust local RANSAC-based alignment to fuse it with raw s...
|
||||
|
||||
#### 9. [2604.17231](https://arxiv.org/abs/2604.17231) — Fringe Projection Based Vision Pipeline for Autonomous Hard Drive Disassembly
|
||||
- **发表**: 2026-04-19 | **分类**: cs.CV, cs.RO
|
||||
- **作者**: Badrinath Balasubramaniam, Vignesh Suresh, Benjamin Metcalf, Beiwen Li
|
||||
- **摘要**: Unrecovered e-waste represents a significant economic loss. Hard disk drives (HDDs) comprise a valuable e-waste stream necessitating robotic disassembly. Automating the disassembly of HDDs requires holistic 3D sensing, scene understanding, and fastener localization, however current methods are fragmented, lack robust 3D sensing, and lack fastener localization. We propose an autonomous vision pipel...
|
||||
|
||||
#### 10. [2604.14048](https://arxiv.org/abs/2604.14048) — Free Geometry: Refining 3D Reconstruction from Longer Versions of Itself
|
||||
- **发表**: 2026-04-15 | **分类**: cs.CV
|
||||
- **作者**: Yuhang Dai, Xingyi Yang
|
||||
- **摘要**: Feed-forward 3D reconstruction models are efficient but rigid: once trained, they perform inference in a zero-shot manner and cannot adapt to the test scene. As a result, visually plausible reconstructions often contain errors, particularly under occlusions, specularities, and ambiguous cues. To address this, we introduce Free Geometry, a framework that enables feed-forward 3D reconstruction model...
|
||||
|
||||
---
|
||||
|
||||
## E. 室内 RGB-D 数据集与重建
|
||||
**项目阶段**: M4-1 / M4-2 | **论文数**: 10
|
||||
|
||||
可参考的数据集设计、评测基准、室内几何重建方法。
|
||||
|
||||
#### 1. [2605.09231](https://arxiv.org/abs/2605.09231) — An Elastic Shape Variational Autoencoder for Skeleton Pose Trajectories
|
||||
- **发表**: 2026-05-10 | **分类**: cs.CV, stat.ML
|
||||
- **作者**: Arafat Rahman, Shashwat Kumar, Laura E. Barnes, Anuj Srivastava
|
||||
- **摘要**: Deep generative models provide flexible frameworks for modeling complex, structured data such as images, videos, 3D objects, and texts. However, when applied to sequences of human skeletons, standard variational autoencoders (VAEs) often allocate substantial capacity to nuisance factors-such as camera orientation, subject scale, viewpoint, and execution speed-rather than the intrinsic geometry of ...
|
||||
|
||||
#### 2. [2605.03463](https://arxiv.org/abs/2605.03463) — First Shape, Then Meaning: Efficient Geometry and Semantics Learning for Indoor Reconstruction
|
||||
- **发表**: 2026-05-05 | **分类**: cs.CV
|
||||
- **作者**: Remi Chierchia, Léo Lebrat, David Ahmedt-Aristizabal, Olivier Salvado 等 (6 人)
|
||||
- **摘要**: Neural Surface Reconstruction has become a standard methodology for indoor 3D reconstruction, with Signed Distance Functions (SDFs) proving particularly effective for representing scene geometry. A variety of applications require a detailed understanding of the scene context, driving the need for object-level semantic signals. While recent methods successfully integrate semantic labels, they often...
|
||||
|
||||
#### 3. [2604.21400](https://arxiv.org/abs/2604.21400) — You Only Gaussian Once: Controllable 3D Gaussian Splatting for Ultra-Densely Sampled Scenes
|
||||
- **发表**: 2026-04-23 | **分类**: cs.CV
|
||||
- **作者**: Jinrang Jia, Zhenjia Li, Yifeng Shi
|
||||
- **摘要**: 3D Gaussian Splatting (3DGS) has revolutionized neural rendering, yet existing methods remain predominantly research prototypes ill-suited for production-level deployment. We identify a critical "Industry-Academia Gap" hindering real-world application: unpredictable resource consumption from heuristic Gaussian growth, the "sparsity shield" of current benchmarks that rewards hallucination over phys...
|
||||
|
||||
#### 4. [2604.18336](https://arxiv.org/abs/2604.18336) — Enhancing Glass Surface Reconstruction via Depth Prior for Robot Navigation
|
||||
- **发表**: 2026-04-20 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Jiamin Zheng, Jingwen Yu, Guangcheng Chen, Hong Zhang
|
||||
- **摘要**: Indoor robot navigation is often compromised by glass surfaces, which severely corrupt depth sensor measurements. While foundation models like Depth Anything 3 provide excellent geometric priors, they lack an absolute metric scale. We propose a training-free framework that leverages depth foundation models as a structural prior, employing a robust local RANSAC-based alignment to fuse it with raw s...
|
||||
|
||||
#### 5. [2604.01605](https://arxiv.org/abs/2604.01605) — F3DGS: Federated 3D Gaussian Splatting for Decentralized Multi-Agent World Modeling
|
||||
- **发表**: 2026-04-02 | **分类**: cs.CV, cs.RO
|
||||
- **作者**: Morui Zhu, Mohammad Dehghani Tezerjani, Mátyás Szántó, Márton Vaitkus 等 (6 人)
|
||||
- **摘要**: We present F3DGS, a federated 3D Gaussian Splatting framework for decentralized multi-agent 3D reconstruction. Existing 3DGS pipelines assume centralized access to all observations, which limits their applicability in distributed robotic settings where agents operate independently, and centralized data aggregation may be restricted. Directly extending centralized training to multi-agent systems in...
|
||||
|
||||
#### 6. [2603.26690](https://arxiv.org/abs/2603.26690) — SpatialPoint: Spatial-aware Point Prediction for Embodied Localization
|
||||
- **发表**: 2026-03-16 | **分类**: cs.RO, cs.AI, cs.CV
|
||||
- **作者**: Qiming Zhu, Zhirui Fang, Tianming Zhang, Chuanxiu Liu 等 (6 人)
|
||||
- **摘要**: Embodied intelligence fundamentally requires a capability to determine where to act in 3D space. We formalize this requirement as embodied localization -- the problem of predicting executable 3D points conditioned on visual observations and language instructions. We instantiate embodied localization with two complementary target types: touchable points, surface-grounded 3D points enabling direct p...
|
||||
|
||||
#### 7. [2603.04254](https://arxiv.org/abs/2603.04254) — EmbodiedSplat: Online Feed-Forward Semantic 3DGS for Open-Vocabulary 3D Scene Understanding
|
||||
- **发表**: 2026-03-04 | **分类**: cs.CV
|
||||
- **作者**: Seungjun Lee, Zihan Wang, Yunsong Wang, Gim Hee Lee
|
||||
- **摘要**: Understanding a 3D scene immediately with its exploration is essential for embodied tasks, where an agent must construct and comprehend the 3D scene in an online and nearly real-time manner. In this study, we propose EmbodiedSplat, an online feed-forward 3DGS for open-vocabulary scene understanding that enables simultaneous online 3D reconstruction and 3D semantic understanding from the streaming ...
|
||||
|
||||
#### 8. [2512.12683](https://arxiv.org/abs/2512.12683) — Quantum Implicit Neural Representations for 3D Scene Reconstruction and Novel View Synthesis
|
||||
- **发表**: 2025-12-14 | **分类**: quant-ph, cs.AI, cs.CV
|
||||
- **作者**: Yeray Cordero, Paula García-Molina, Fernando Vilariño
|
||||
- **摘要**: Implicit neural representations (INRs) have become a powerful paradigm for continuous signal modeling and 3D scene reconstruction, yet classical networks suffer from a well-known spectral bias that limits their ability to capture high-frequency details. Quantum Implicit Representation Networks (QIREN) mitigate this limitation by employing parameterized quantum circuits with inherent Fourier struct...
|
||||
|
||||
#### 9. [2511.07412](https://arxiv.org/abs/2511.07412) — TwinOR: Photorealistic Digital Twins of Dynamic Operating Rooms for Embodied AI Research 🇨🇳
|
||||
- **发表**: 2025-11-10 | **分类**: cs.CV, cs.RO
|
||||
- **作者**: Han Zhang, Yiqing Shen, Roger D. Soberanis-Mukul, Ankita Ghosh 等 (14 人)
|
||||
- **摘要**: Developing embodied AI for intelligent surgical systems requires safe, controllable environments for continual learning and evaluation. However, safety regulations and operational constraints in operating rooms (ORs) limit agents from freely perceiving and interacting in realistic settings. Digital twins provide high-fidelity, risk-free environments for exploration and training. How we may create ...
|
||||
|
||||
#### 10. [2510.12387](https://arxiv.org/abs/2510.12387) — Scene Coordinate Reconstruction Priors
|
||||
- **发表**: 2025-10-14 | **分类**: cs.CV
|
||||
- **作者**: Wenjing Bian, Axel Barroso-Laguna, Tommaso Cavallari, Victor Adrian Prisacariu 等 (5 人)
|
||||
- **摘要**: Scene coordinate regression (SCR) models have proven to be powerful implicit scene representations for 3D vision, enabling visual relocalization and structure-from-motion. SCR models are trained specifically for one scene. If training images imply insufficient multi-view constraints SCR models degenerate. We present a probabilistic reinterpretation of training SCR models, which allows us to infuse...
|
||||
|
||||
---
|
||||
|
||||
## F. 视频世界模型（下游应用）
|
||||
**项目阶段**: M4-4 | **论文数**: 10
|
||||
|
||||
本项目数据 pipeline 的最终下游：训练能预测未来视频/动作的 world model。
|
||||
|
||||
#### 1. [2605.15185](https://arxiv.org/abs/2605.15185) — Quantitative Video World Model Evaluation for Geometric-Consistency
|
||||
- **发表**: 2026-05-14 | **分类**: cs.CV, cs.AI
|
||||
- **作者**: Jiaxin Wu, Yihao Pi, Yinling Zhang, Yuheng Li 等 (5 人)
|
||||
- **摘要**: Generative video models are increasingly studied as implicit world models, yet evaluating whether they produce physically plausible 3D structure and motion remains challenging. Most existing video evaluation pipelines rely heavily on human judgment or learned graders, which can be subjective and weakly diagnostic for geometric failures. We introduce PDI-Bench (Perspective Distortion Index), a quan...
|
||||
|
||||
#### 2. [2605.15178](https://arxiv.org/abs/2605.15178) — SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer
|
||||
- **发表**: 2026-05-14 | **分类**: cs.CV
|
||||
- **作者**: Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye 等 (9 人)
|
||||
- **摘要**: We introduce SANA-WM, an efficient 2.6B-parameter open-source world model natively trained for one-minute generation, synthesizing high-fidelity, 720p, minute-scale videos with precise camera control. SANA-WM achieves visual quality comparable to large-scale industrial baselines such as LingBot-World and HY-WorldPlay, while significantly improving efficiency. Four core designs drive our architectu...
|
||||
|
||||
#### 3. [2605.15141](https://arxiv.org/abs/2605.15141) — Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
|
||||
- **发表**: 2026-05-14 | **分类**: cs.CV
|
||||
- **作者**: Min Zhao, Hongzhou Zhu, Kaiwen Zheng, Zihan Zhou 等 (9 人)
|
||||
- **摘要**: Real-time interactive video generation requires low-latency, streaming, and controllable rollout. Existing autoregressive (AR) diffusion distillation methods have achieved strong results in the chunk-wise 4-step regime by distilling bidirectional base models into few-step AR students, but they remain limited by coarse response granularity and non-negligible sampling latency. In this paper, we stud...
|
||||
|
||||
#### 4. [2605.14937](https://arxiv.org/abs/2605.14937) — Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations
|
||||
- **发表**: 2026-05-14 | **分类**: cs.LG, cs.AI, cs.RO
|
||||
- **作者**: Jonathan Spieler, Angel Villar-Corrales, Sven Behnke
|
||||
- **摘要**: Predictive world models enable agents to model scene dynamics and reason about the consequences of their actions. Inspired by human perception, object-centric world models capture scene dynamics using object-level representations, which can be used for downstream applications such as action planning. However, most object-centric world models and reinforcement learning (RL) approaches learn reactiv...
|
||||
|
||||
#### 5. [2605.14851](https://arxiv.org/abs/2605.14851) — IFPV: An Integrated Multi-Agent Framework for Generative Operational Planning and High-Fidelity Plan Verification
|
||||
- **发表**: 2026-05-14 | **分类**: cs.MA, cs.AI
|
||||
- **作者**: Zhigao Huang, Zhengqing Hu, Dong Chen, Shaohan Zhang 等 (8 人)
|
||||
- **摘要**: Operational plan generation and verification are critical for modern complex and rapidly changing battlefield environments, yet traditional generation and verification methods still respectively face the challenges of generation infeasibility and verification insufficiency. To alleviate these limitations, we propose an Integrated Multi-Agent Framework for Generative Operational Planning and High-F...
|
||||
|
||||
#### 6. [2605.14757](https://arxiv.org/abs/2605.14757) — ChannelAgent-Empowered Electromagnetic Space World Model: A Case Study on Agent-Driven Channel Generation for 6G AI-Native Air Interface
|
||||
- **发表**: 2026-05-14 | **分类**: eess.SP
|
||||
- **作者**: Mingyue Li, Li Yu, Yuxiang Zhang, Heng Wang 等 (8 人)
|
||||
- **摘要**: As sixth-generation (6G) wireless networks evolve toward increasingly heterogeneous scenarios, tasks, and service requirements, conventional artificial intelligence (AI) models remain limited in task-aware decision-making and autonomous adaptation. To address this issue, this paper first proposes a ChannelAgent-empowered electromagnetic space world model, in which wireless intelligence is organize...
|
||||
|
||||
#### 7. [2605.14696](https://arxiv.org/abs/2605.14696) — EponaV2: Driving World Model with Comprehensive Future Reasoning
|
||||
- **发表**: 2026-05-14 | **分类**: cs.CV
|
||||
- **作者**: Jiawei Xu, Zhizhou Zhong, Zhijian Shu, Mingkai Jia 等 (11 人)
|
||||
- **摘要**: Data scaling plays a pivotal role in the pursuit of general intelligence. However, the prevailing perception-planning paradigm in autonomous driving relies heavily on expensive manual annotations to supervise trajectory planning, which severely limits its scalability. Conversely, although existing perception-free driving world models achieve impressive driving performance, their real-world reasoni...
|
||||
|
||||
#### 8. [2605.14398](https://arxiv.org/abs/2605.14398) — Coding Agent Is Good As World Simulator
|
||||
- **发表**: 2026-05-14 | **分类**: cs.AI
|
||||
- **作者**: Hongyu Wang, Jingquan Wang, Bocheng Zou, Radu Serban 等 (5 人)
|
||||
- **摘要**: World models have emerged as a powerful paradigm for building interactive simulation environments, with recent video-based approaches demonstrating impressive progress in generating visually plausible dynamics. However, because these models typically infer dynamics from video and represent them in latent states, they do not explicitly enforce physical constraints. As a result, the generated video ...
|
||||
|
||||
#### 9. [2605.14382](https://arxiv.org/abs/2605.14382) — Delta Forcing: Trust Region Steering for Interactive Autoregressive Video Generation
|
||||
- **发表**: 2026-05-14 | **分类**: cs.CV, cs.GR, cs.MM
|
||||
- **作者**: Yuheng Wu, Xiangbo Gao, Tianhao Chen, Xinghao Chen 等 (7 人)
|
||||
- **摘要**: Interactive real-time autoregressive video generation is essential for applications such as content creation and world modeling, where visual content must adapt to dynamically evolving event conditions. A fundamental challenge lies in balancing reactivity and stability: models must respond promptly to new events while maintaining temporal coherence over long horizons. Existing approaches distill b...
|
||||
|
||||
#### 10. [2605.14036](https://arxiv.org/abs/2605.14036) — Enhanced and Efficient Reasoning in Large Learning Models
|
||||
- **发表**: 2026-05-13 | **分类**: cs.AI, cs.CC, cs.CL
|
||||
- **作者**: Leslie G. Valiant
|
||||
- **摘要**: In current Large Language Models we can trust the production of smoothly flowing prose on the basis of the principles of machine learning. However, there is no comparably principled basis to justify trust in the content of the text produced. It appears to be conventional wisdom that addressing this issue by adding more principled reasoning is not computationally affordable. Here we propose a pri...
|
||||
|
||||
---
|
||||
|
||||
## G. ZED 相机相关应用工作
|
||||
**项目阶段**: 全周期 | **论文数**: 10
|
||||
|
||||
用 ZED 系列采集数据的应用论文，参考其采集协议、评测方式、参数配置。
|
||||
|
||||
#### 1. [2602.16385](https://arxiv.org/abs/2602.16385) — Adaptive Multi-Scale Channel-Spatial Attention Aggregation Framework for 3D Indoor Semantic Scene Completion Toward Assisting Visually Impaired
|
||||
- **发表**: 2026-02-18 | **分类**: cs.CV
|
||||
- **作者**: Qi He, XiangXiang Wang, Jingtao Zhang, Yongbin Yu 等 (8 人)
|
||||
- **摘要**: Independent indoor mobility remains a critical challenge for individuals with visual impairments, largely due to the limited capability of existing assistive systems in detecting fine-grained hazardous objects such as chairs, tables, and small obstacles. These perceptual blind zones substantially increase the risk of collision in unfamiliar environments. To bridge the gap between monocular 3D visi...
|
||||
|
||||
#### 2. [2602.09414](https://arxiv.org/abs/2602.09414) — Finite-time Stable Pose Estimation on TSE(3) using Point Cloud and Velocity Sensors
|
||||
- **发表**: 2026-02-10 | **分类**: eess.SY, cs.RO
|
||||
- **作者**: Nazanin S. Hashkavaei, Abhijit Dongare, Neon Srinivasu, Amit K. Sanyal
|
||||
- **摘要**: This work presents a finite-time stable pose estimator (FTS-PE) for rigid bodies undergoing rotational and translational motion in three dimensions, using measurements from onboard sensors that provide position vectors to inertially-fixed points and body velocities. The FTS-PE is a full-state observer for the pose (position and orientation) and velocities and is obtained through a Lyapunov analysi...
|
||||
|
||||
#### 3. [2512.03886](https://arxiv.org/abs/2512.03886) — A Modular Architecture Design for Autonomous Driving Racing in Controlled Environments
|
||||
- **发表**: 2025-12-03 | **分类**: cs.RO, eess.SY
|
||||
- **作者**: Brais Fontan-Costas, M. Diaz-Cacho, Ruben Fernandez-Boullon, Manuel Alonso-Carracedo 等 (5 人)
|
||||
- **摘要**: This paper presents a modular autonomous driving architecture for Formula Student Driverless competition vehicles operating in closed-circuit environments. The perception module employs YOLOv11 for real-time traffic cone detection, achieving 0.93 mAP@0.5 on the FSOCO dataset, combined with neural stereo depth estimation from a ZED 2i camera for 3D cone localization with sub-0.5 m median error at d...
|
||||
|
||||
#### 4. [2512.01108](https://arxiv.org/abs/2512.01108) — Think Fast: Real-Time Kinodynamic Belief-Space Planning for Projectile Interception
|
||||
- **发表**: 2025-11-30 | **分类**: cs.RO
|
||||
- **作者**: Gabriel Olin, Lu Chen, Nayesha Gandotra, Maxim Likhachev 等 (5 人)
|
||||
- **摘要**: Intercepting fast moving objects, by its very nature, is challenging because of its tight time constraints. This problem becomes further complicated in the presence of sensor noise because noisy sensors provide, at best, incomplete information, which results in a distribution over target states to be intercepted. Since time is of the essence, to hit the target, the planner must begin directing the...
|
||||
|
||||
#### 5. [2509.10466](https://arxiv.org/abs/2509.10466) — A Real-Time Diminished Reality Approach to Privacy in MR Collaboration
|
||||
- **发表**: 2025-08-21 | **分类**: cs.CV, cs.HC
|
||||
- **作者**: Christian Fane
|
||||
- **摘要**: Diminished reality (DR) refers to the digital removal of real-world objects by compositing background content in their place. This thesis presents a real-time, inpainting-based DR system designed to enable privacy control in shared-space mixed reality (MR) meetings. The system allows a primary headset user to selectively remove personal or sensitive items from their environment, ensuring that thos...
|
||||
|
||||
#### 6. [2504.06464](https://arxiv.org/abs/2504.06464) — Implementation of a Zed 2i Stereo Camera for High-Frequency Shoreline Change and Coastal Elevation Monitoring
|
||||
- **发表**: 2025-04-08 | **分类**: cs.CV
|
||||
- **作者**: José A. Pilartes-Congo, Matthew Kastl, Michael J. Starek, Marina Vicens-Miquel 等 (5 人)
|
||||
- **摘要**: The increasing population, thus financial interests, in coastal areas have increased the need to monitor coastal elevation and shoreline change. Though several resources exist to obtain this information, they often lack the required temporal resolution for short-term monitoring (e.g., every hour). To address this issue, this study implements a low-cost ZED 2i stereo camera system and close-range p...
|
||||
|
||||
#### 7. [2501.09490](https://arxiv.org/abs/2501.09490) — Comparison of Various SLAM Systems for Mobile Robot in an Indoor Environment
|
||||
- **发表**: 2025-01-16 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Maksim Filipenko, Ilya Afanasyev
|
||||
- **摘要**: This article presents a comparative analysis of a mobile robot trajectories computed by various ROS-based SLAM systems. For this reason we developed a prototype of a mobile robot with common sensors: 2D lidar, a monocular and ZED stereo cameras. Then we conducted experiments in a typical office environment and collected data from all sensors, running all tested SLAM systems based on the acquired d...
|
||||
|
||||
#### 8. [2501.07421](https://arxiv.org/abs/2501.07421) — Empirical Comparison of Four Stereoscopic Depth Sensing Cameras for Robotics Applications
|
||||
- **发表**: 2025-01-13 | **分类**: cs.RO
|
||||
- **作者**: Lukas Rustler, Vojtech Volprecht, Matej Hoffmann
|
||||
- **摘要**: Depth sensing is an essential technology in robotics and many other fields. Many depth sensing (or RGB-D) cameras are available on the market and selecting the best one for your application can be challenging. In this work, we tested four stereoscopic RGB-D cameras that sense the distance by using two images from slightly different views. We empirically compared four cameras (Intel RealSense D435,...
|
||||
|
||||
#### 9. [2410.20599](https://arxiv.org/abs/2410.20599) — Sensor Fusion for Autonomous Indoor UAV Navigation in Confined Spaces
|
||||
- **发表**: 2024-10-27 | **分类**: cs.RO
|
||||
- **作者**: Alice James, Avishkar Seth, Endrowednes Kuantama, Subhas Mukhopadhyay 等 (5 人)
|
||||
- **摘要**: In this paper, we address the challenge of navigating through unknown indoor environments using autonomous aerial robots within confined spaces. The core of our system involves the integration of key sensor technologies, including depth sensing from the ZED 2i camera, IMU data, and LiDAR measurements, facilitated by the Robot Operating System (ROS) and RTAB-Map. Through custom designed experiments...
|
||||
|
||||
#### 10. [2407.18695](https://arxiv.org/abs/2407.18695) — PIV3CAMS: a multi-camera dataset for multiple computer vision problems and its application to novel view-point synthesis
|
||||
- **发表**: 2024-07-26 | **分类**: cs.CV
|
||||
- **作者**: Sohyeong Kim, Martin Danelljan, Radu Timofte, Luc Van Gool 等 (5 人)
|
||||
- **摘要**: The modern approaches for computer vision tasks significantly rely on machine learning, which requires a large number of quality images. While there is a plethora of image datasets with a single type of images, there is a lack of datasets collected from multiple cameras. In this thesis, we introduce Paired Image and Video data from three CAMeraS, namely PIV3CAMS, aimed at multiple computer vision ...
|
||||
|
||||
---
|
||||
|
||||
## H. Orbbec / Femto / Azure Kinect 相关工作
|
||||
**项目阶段**: 国产化 / 替代硬件 | **论文数**: 10
|
||||
|
||||
奥比中光、乐视/微视 Femto、微软 Azure Kinect 等 RGB-D 相机的应用论文。
|
||||
|
||||
#### 1. [2605.06351](https://arxiv.org/abs/2605.06351) — SIGMA-ASL: Sensor-Integrated Multimodal Dataset for Sign Language Recognition
|
||||
- **发表**: 2026-05-07 | **分类**: cs.HC
|
||||
- **作者**: Xiaofang Xiao, Guangchao Li, Guangrong Zhao, Qi Lin 等 (8 人)
|
||||
- **摘要**: Automatic sign language recognition (SLR) has become a key enabler of inclusive human-computer interaction, fostering seamless communication between deaf individuals and hearing communities. Despite significant advances in multimodal learning, existing SLR research remains dominated by vision-based datasets, which are limited by sensitivity to lighting and occlusion, privacy concerns, and a lack o...
|
||||
|
||||
#### 2. [2509.11574](https://arxiv.org/abs/2509.11574) — Gaussian-Plus-SDF SLAM: High-fidelity 3D Reconstruction at 150+ fps
|
||||
- **发表**: 2025-09-15 | **分类**: cs.CV
|
||||
- **作者**: Zhexi Peng, Kun Zhou, Tianjia Shao
|
||||
- **摘要**: While recent Gaussian-based SLAM methods achieve photorealistic reconstruction from RGB-D data, their computational performance remains a critical bottleneck. State-of-the-art techniques operate at less than 20 fps, significantly lagging behind geometry-based approaches like KinectFusion (hundreds of fps). This limitation stems from the heavy computational burden: modeling scenes requires numerous...
|
||||
|
||||
#### 3. [2401.10037](https://arxiv.org/abs/2401.10037) — Depth Over RGB: Automatic Evaluation of Open Surgery Skills Using Depth Camera
|
||||
- **发表**: 2024-01-18 | **分类**: cs.CV
|
||||
- **作者**: Ido Zuckerman, Nicole Werner, Jonathan Kouchly, Emma Huston 等 (7 人)
|
||||
- **摘要**: Purpose: In this paper, we present a novel approach to the automatic evaluation of open surgery skills using depth cameras. This work is intended to show that depth cameras achieve similar results to RGB cameras, which is the common method in the automatic evaluation of open surgery skills. Moreover, depth cameras offer advantages such as robustness to lighting variations, camera positioning, simp...
|
||||
|
||||
#### 4. [2401.08629](https://arxiv.org/abs/2401.08629) — Immature Green Apple Detection and Sizing in Commercial Orchards using YOLOv8 and Shape Fitting Techniques
|
||||
- **发表**: 2023-12-08 | **分类**: cs.CV
|
||||
- **作者**: Ranjan Sapkota, Dawood Ahmed, Martin Churuvija, Manoj Karkee
|
||||
- **摘要**: Detecting and estimating size of apples during the early stages of growth is crucial for predicting yield, pest management, and making informed decisions related to crop-load management, harvest and post-harvest logistics, and marketing. Traditional fruit size measurement methods are laborious and timeconsuming. This study employs the state-of-the-art YOLOv8 object detection and instance segmentat...
|
||||
|
||||
#### 5. [2311.09029](https://arxiv.org/abs/2311.09029) — Self-Annotated 3D Geometric Learning for Smeared Points Removal
|
||||
- **发表**: 2023-11-15 | **分类**: cs.CV
|
||||
- **作者**: Miaowei Wang, Daniel Morris
|
||||
- **摘要**: There has been significant progress in improving the accuracy and quality of consumer-level dense depth sensors. Nevertheless, there remains a common depth pixel artifact which we call smeared points. These are points not on any 3D surface and typically occur as interpolations between foreground and background objects. As they cause fictitious surfaces, these points have the potential to harm appl...
|
||||
|
||||
#### 6. [2306.02263](https://arxiv.org/abs/2306.02263) — MAVD: The First Open Large-Scale Mandarin Audio-Visual Dataset with Depth Information
|
||||
- **发表**: 2023-06-04 | **分类**: cs.SD, cs.CV
|
||||
- **作者**: Jianrong Wang, Yuchen Huo, Li Liu, Tianyi Xu 等 (6 人)
|
||||
- **摘要**: Audio-visual speech recognition (AVSR) gains increasing attention from researchers as an important part of human-computer interaction. However, the existing available Mandarin audio-visual datasets are limited and lack the depth information. To address this issue, this work establishes the MAVD, a new large-scale Mandarin multimodal corpus comprising 12,484 utterances spoken by 64 native Chinese s...
|
||||
|
||||
#### 7. [2304.13282](https://arxiv.org/abs/2304.13282) — Machine Vision-Based Crop-Load Estimation Using YOLOv8
|
||||
- **发表**: 2023-04-26 | **分类**: cs.RO
|
||||
- **作者**: Dawood Ahmed, Ranjan Sapkota, Martin Churuvija, Manoj Karkee
|
||||
- **摘要**: Labor shortages in fruit crop production have prompted the development of mechanized and automated machines as alternatives to labor-intensive orchard operations such as harvesting, pruning, and thinning. Agricultural robots capable of identifying tree canopy parts and estimating geometric and topological parameters, such as branch diameter, length, and angles, can optimize crop yields through aut...
|
||||
|
||||
#### 8. [2304.08210](https://arxiv.org/abs/2304.08210) — ATTACH Dataset: Annotated Two-Handed Assembly Actions for Human Action Understanding
|
||||
- **发表**: 2023-04-17 | **分类**: cs.RO, cs.CV, cs.LG
|
||||
- **作者**: Dustin Aganian, Benedict Stephan, Markus Eisenbach, Corinna Stretz 等 (5 人)
|
||||
- **摘要**: With the emergence of collaborative robots (cobots), human-robot collaboration in industrial manufacturing is coming into focus. For a cobot to act autonomously and as an assistant, it must understand human actions during assembly. To effectively train models for this task, a dataset containing suitable assembly actions in a realistic setting is crucial. For this purpose, we present the ATTACH dat...
|
||||
|
||||
#### 9. [2303.16196](https://arxiv.org/abs/2303.16196) — SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis
|
||||
- **发表**: 2023-03-28 | **分类**: cs.CV
|
||||
- **作者**: Guangcong Wang, Zhaoxi Chen, Chen Change Loy, Ziwei Liu
|
||||
- **摘要**: Neural Radiance Field (NeRF) significantly degrades when only a limited number of views are available. To complement the lack of 3D information, depth-based models, such as DSNeRF and MonoSDF, explicitly assume the availability of accurate depth maps of multiple views. They linearly scale the accurate depth maps as supervision to guide the predicted depth of few-shot NeRFs. However, accurate depth...
|
||||
|
||||
#### 10. [2302.05991](https://arxiv.org/abs/2302.05991) — Digital Twin Tracking Dataset (DTTD): A New RGB+Depth 3D Dataset for Longer-Range Object Tracking Applications
|
||||
- **发表**: 2023-02-12 | **分类**: cs.CV
|
||||
- **作者**: Weiyu Feng, Seth Z. Zhao, Chuanyu Pan, Adam Chang 等 (7 人)
|
||||
- **摘要**: Digital twin is a problem of augmenting real objects with their digital counterparts. It can underpin a wide range of applications in augmented reality (AR), autonomy, and UI/UX. A critical component in a good digital-twin system is real-time, accurate 3D object tracking. Most existing works solve 3D object tracking through the lens of robotic grasping, employ older generations of depth sensors, a...
|
||||
|
||||
---
|
||||
|
||||
## I. RGB-D 室内重建
|
||||
**项目阶段**: M2-2 / M3-5 | **论文数**: 10
|
||||
|
||||
RGB-D 输入下的室内场景重建，与本项目的房间级建图任务高度对齐。
|
||||
|
||||
#### 1. [2605.03678](https://arxiv.org/abs/2605.03678) — Robust Visual SLAM for UAV Navigation in GPS-Denied and Degraded Environments: A Multi-Paradigm Evaluation and Deployment Study
|
||||
- **发表**: 2026-05-05 | **分类**: cs.RO
|
||||
- **作者**: Prasoon Kumar, Akshay Deepak, Sandeep Kumar
|
||||
- **摘要**: Reliable localization in GPS-denied, visually degraded environments is critical for autonomous UAV opera- tions. This paper presents a systematic comparative evaluation of five V-SLAM systems ORB-SLAM3, DPVO, DROID-SLAM, DUSt3R, and MASt3R spanning classical, deep learning, recurrent, and Vision Transformer (ViT) paradigms. Experiments are conducted on curated sequences from four public benchmarks...
|
||||
|
||||
#### 2. [2604.28115](https://arxiv.org/abs/2604.28115) — FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction
|
||||
- **发表**: 2026-04-30 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Zeyu Jiang, Changqing Zhou, Xingxing Zuo, Changhao Chen
|
||||
- **摘要**: Existing learning-based occupancy prediction methods rely on large-scale 3D annotations and generalize poorly across environments. We present FreeOcc, a training-free framework for open-vocabulary occupancy prediction from monocular or RGB-D sequences. Unlike prior approaches that require voxel-level supervision and ground-truth camera poses, FreeOcc operates without 3D annotations, pose ground tr...
|
||||
|
||||
#### 3. [2604.25404](https://arxiv.org/abs/2604.25404) — Robust Graph Matching through Semantic Relationship Generation for SLAM
|
||||
- **发表**: 2026-04-28 | **分类**: cs.RO
|
||||
- **作者**: David Perez-Saura, Jose Andres Millan-Romera, Miguel Fernandez-Cortizas, Holger Voos 等 (6 人)
|
||||
- **摘要**: Graph-based representations such as Scene Graphs enable localization in structured indoor environments by matching a locally observed graph, constructed from sensor data, to a prior map. This process is particularly challenging in environments with repetitive or symmetric layouts, where structural cues alone are often insufficient to resolve ambiguities. We propose a semantic-enhanced graph matchi...
|
||||
|
||||
#### 4. [2604.24707](https://arxiv.org/abs/2604.24707) — Passage-Aware Structural Mapping for RGB-D Visual SLAM
|
||||
- **发表**: 2026-04-27 | **分类**: cs.RO
|
||||
- **作者**: Ali Tourani, Miguel Fernandez-Cortizas, Saad Ejaz, David Pérez Saura 等 (7 人)
|
||||
- **摘要**: Doorways and passages are critical structural elements for indoor robot navigation, yet they remain underexplored in modern Visual SLAM (VSLAM) frameworks. This paper presents a passage-aware structural mapping approach for RGB-D VSLAM that detects doors and traversable openings by jointly fusing geometric, semantic, and topological cues. Doors are modeled as planar entities embedded within walls ...
|
||||
|
||||
#### 5. [2604.19025](https://arxiv.org/abs/2604.19025) — RoomRecon: High-Quality Textured Room Layout Reconstruction on Mobile Devices
|
||||
- **发表**: 2026-04-21 | **分类**: cs.RO
|
||||
- **作者**: Seok Joon Kim, Dinh Duc Cao, Federica Spinola, Se Jin Lee 等 (5 人)
|
||||
- **摘要**: Widespread RGB-Depth (RGB-D) sensors and advanced 3D reconstruction technologies facilitate the capture of indoor spaces, improving the fields of augmented reality (AR), virtual reality (VR), and extended reality (XR). Nevertheless, current technologies still face limitations, such as the inability to reflect minor scene changes without a complete recapture, the lack of semantic scene understandin...
|
||||
|
||||
#### 6. [2604.18336](https://arxiv.org/abs/2604.18336) — Enhancing Glass Surface Reconstruction via Depth Prior for Robot Navigation
|
||||
- **发表**: 2026-04-20 | **分类**: cs.RO, cs.CV
|
||||
- **作者**: Jiamin Zheng, Jingwen Yu, Guangcheng Chen, Hong Zhang
|
||||
- **摘要**: Indoor robot navigation is often compromised by glass surfaces, which severely corrupt depth sensor measurements. While foundation models like Depth Anything 3 provide excellent geometric priors, they lack an absolute metric scale. We propose a training-free framework that leverages depth foundation models as a structural prior, employing a robust local RANSAC-based alignment to fuse it with raw s...
|
||||
|
||||
#### 7. [2604.15052](https://arxiv.org/abs/2604.15052) — CAVERS: Multimodal SLAM Data from a Natural Karstic Cave with Ground Truth Motion Capture
|
||||
- **发表**: 2026-04-16 | **分类**: cs.RO
|
||||
- **作者**: Giacomo Franchini, David Rodríguez-Martínez, Alfonso Martínez-Petersen, C. J. Pérez-del-Pulgar 等 (5 人)
|
||||
- **摘要**: Autonomous robots operating in natural karstic caves face perception and navigation challenges that are qualitatively distinct from those encountered in mines or tunnels: irregular geometry, reflective wet surfaces, near-zero ambient light, and complex branching passages. Yet publicly available datasets targeting this environment remain scarce and offer limited sensing modalities and environmental...
|
||||
|
||||
#### 8. [2604.05621](https://arxiv.org/abs/2604.05621) — FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos
|
||||
- **发表**: 2026-04-07 | **分类**: cs.CV
|
||||
- **作者**: Alexandros Delitzas, Chenyangguang Zhang, Alexey Gavryushin, Tommaso Di Mario 等 (11 人)
|
||||
- **摘要**: We present FunRec, a method for reconstructing functional 3D digital twins of indoor scenes directly from egocentric RGB-D interaction videos. Unlike existing methods on articulated reconstruction, which rely on controlled setups, multi-state captures, or CAD priors, FunRec operates directly on in-the-wild human interaction sequences to recover interactable 3D scenes. It automatically discovers ar...
|
||||
|
||||
#### 9. [2603.13917](https://arxiv.org/abs/2603.13917) — Evaluation of Visual Place Recognition Methods for Image Pair Retrieval in 3D Vision and Robotics
|
||||
- **发表**: 2026-03-14 | **分类**: cs.CV
|
||||
- **作者**: Dennis Haitz, Athradi Shritish Shetty, Michael Weinmann, Markus Ulrich
|
||||
- **摘要**: Visual Place Recognition (VPR) is a core component in computer vision, typically formulated as an image retrieval task for localization, mapping, and navigation. In this work, we instead study VPR as an image pair retrieval front-end for registration pipelines, where the goal is to find top-matching image pairs between two disjoint image sets for downstream tasks such as scene registration, SLAM, ...
|
||||
|
||||
#### 10. [2512.12378](https://arxiv.org/abs/2512.12378) — M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstruction
|
||||
- **发表**: 2025-12-13 | **分类**: cs.CV
|
||||
- **作者**: Junqiao Fan, Yunjiao Zhou, Yizhuo Yang, Xinyuan Cui 等 (9 人)
|
||||
- **摘要**: Human mesh reconstruction (HMR) provides direct insights into body-environment interaction, which enables various immersive applications. While existing large-scale HMR datasets rely heavily on line-of-sight RGB input, vision-based sensing is limited by occlusion, lighting variation, and privacy concerns. To overcome these limitations, recent efforts have explored radio-frequency (RF) mmWave radar...
|
||||
|
||||
---
|
||||
|
||||
## J. 神经立体深度（指定 RAFT/IGEV/Foundation 家族）
|
||||
**项目阶段**: M3-4 | **论文数**: 10
|
||||
|
||||
针对 RAFT-Stereo / IGEV-Stereo / FoundationStereo 等核心立体匹配方法的衍生与改进。
|
||||
|
||||
#### 1. [2605.08213](https://arxiv.org/abs/2605.08213) — Low-Cost Stereo Vision for Robust 3D Positioning of Thin Radiata Pine Branches in Autonomous Drone Pruning
|
||||
- **发表**: 2026-05-06 | **分类**: cs.CV
|
||||
- **作者**: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield 等 (5 人)
|
||||
- **摘要**: Manual pruning of radiata pine, a species of major economic importance to New Zealand forestry, is hazardous, labour-intensive, and increasingly constrained by workforce shortages. Existing autonomous pruning platforms typically rely on expensive sensors such as LiDAR and are limited to thick branches, which restricts their wider adoption. This paper investigates whether a single low-cost stereo c...
|
||||
|
||||
#### 2. [2604.16480](https://arxiv.org/abs/2604.16480) — Positioning radiata pine branches requiring pruning by drone stereo vision
|
||||
- **发表**: 2026-04-12 | **分类**: cs.CV
|
||||
- **作者**: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield 等 (5 人)
|
||||
- **摘要**: This paper presents a stereo-vision-based system mounted on a drone for detecting and localising radiata pine branches to support autonomous pruning. The proposed pipeline comprises two stages: branch segmentation and depth estimation. For segmentation, YOLOv8, YOLOv9, and Mask R-CNN variants are compared on a custom dataset of 71 stereo image pairs captured with a ZED Mini camera. For depth estim...
|
||||
|
||||
#### 3. [2602.19763](https://arxiv.org/abs/2602.19763) — Training Deep Stereo Matching Networks on Tree Branch Imagery: A Benchmark Study for Real-Time UAV Forestry Applications
|
||||
- **发表**: 2026-02-23 | **分类**: cs.CV, eess.IV
|
||||
- **作者**: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield 等 (5 人)
|
||||
- **摘要**: Autonomous drone-based tree pruning needs accurate, real-time depth estimation from stereo cameras. Depth is computed from disparity maps using $Z = f B/d$, so even small disparity errors cause noticeable depth mistakes at working distances. Building on our earlier work that identified DEFOM-Stereo as the best reference disparity generator for vegetation scenes, we present the first study to train...
|
||||
|
||||
#### 4. [2601.19461](https://arxiv.org/abs/2601.19461) — Towards Gold-Standard Depth Estimation for Tree Branches in UAV Forestry: Benchmarking Deep Stereo Matching Methods
|
||||
- **发表**: 2026-01-27 | **分类**: cs.CV, cs.RO, eess.IV
|
||||
- **作者**: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield 等 (5 人)
|
||||
- **摘要**: Autonomous UAV forestry operations require robust depth estimation with strong cross-domain generalization, yet existing evaluations focus on urban and indoor scenarios, leaving a critical gap for vegetation-dense environments. We present the first systematic zero-shot evaluation of eight stereo methods spanning iterative refinement, foundation model, diffusion-based, and 3D CNN paradigms. All met...
|
||||
|
||||
#### 5. [2512.03427](https://arxiv.org/abs/2512.03427) — Generalization Evaluation of Deep Stereo Matching Methods for UAV-Based Forestry Applications
|
||||
- **发表**: 2025-12-03 | **分类**: cs.CV
|
||||
- **作者**: Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield 等 (5 人)
|
||||
- **摘要**: Autonomous UAV forestry operations require robust depth estimation methods with strong cross-domain generalization. However, existing evaluations focus on urban and indoor scenarios, leaving a critical gap for specialized vegetation-dense environments. We present the first systematic zero-shot evaluation of eight state-of-the-art stereo methods--RAFT-Stereo, IGEV, IGEV++, BridgeDepth, StereoAnywhe...
|
||||
|
||||
#### 6. [2507.19738](https://arxiv.org/abs/2507.19738) — Leveraging Sparse LiDAR for RAFT-Stereo: A Depth Pre-Fill Perspective
|
||||
- **发表**: 2025-07-26 | **分类**: cs.CV
|
||||
- **作者**: Jinsu Yoo, Sooyoung Jeon, Zanming Huang, Tai-Yu Pan 等 (5 人)
|
||||
- **摘要**: We investigate LiDAR guidance within the RAFT-Stereo framework, aiming to improve stereo matching accuracy by injecting precise LiDAR depth into the initial disparity map. We find that the effectiveness of LiDAR guidance drastically degrades when the LiDAR points become sparse (e.g., a few hundred points per frame), and we offer a novel explanation from a signal processing perspective. This insigh...
|
||||
|
||||
#### 7. [2507.10991](https://arxiv.org/abs/2507.10991) — Uncertainty Aware Mapping for Vision-Based Underwater Robots
|
||||
- **发表**: 2025-07-15 | **分类**: cs.RO
|
||||
- **作者**: Abhimanyu Bhowmik, Mohit Singh, Madhushree Sannigrahi, Martin Ludvigsen 等 (5 人)
|
||||
- **摘要**: Vision-based underwater robots can be useful in inspecting and exploring confined spaces where traditional sensors and preplanned paths cannot be followed. Sensor noise and situational change can cause significant uncertainty in environmental representation. Thus, this paper explores how to represent mapping inconsistency in vision-based sensing and incorporate depth estimation confidence into the...
|
||||
|
||||
#### 8. [2506.16690](https://arxiv.org/abs/2506.16690) — DepthVanish: Optimizing Adversarial Interval Structures for Stereo-Depth-Invisible Patches
|
||||
- **发表**: 2025-06-20 | **分类**: cs.CV
|
||||
- **作者**: Yun Xing, Yue Cao, Nhat Chung, Jie Zhang 等 (9 人)
|
||||
- **摘要**: Stereo depth estimation is a critical task in autonomous driving and robotics, where inaccuracies (such as misidentifying nearby objects as distant) can lead to dangerous situations. Adversarial attacks against stereo depth estimation can help reveal vulnerabilities before deployment. Previous works have shown that repeating optimized textures can effectively mislead stereo depth estimation in dig...
|
||||
|
||||
#### 9. [2505.11439](https://arxiv.org/abs/2505.11439) — SurgPose: Generalisable Surgical Instrument Pose Estimation using Zero-Shot Learning and Stereo Vision
|
||||
- **发表**: 2025-05-16 | **分类**: cs.CV, cs.AI, cs.LG
|
||||
- **作者**: Utsav Rai, Haozheng Xu, Stamatia Giannarou
|
||||
- **摘要**: Accurate pose estimation of surgical tools in Robot-assisted Minimally Invasive Surgery (RMIS) is essential for surgical navigation and robot control. While traditional marker-based methods offer accuracy, they face challenges with occlusions, reflections, and tool-specific designs. Similarly, supervised learning methods require extensive training on annotated datasets, limiting their adaptability...
|
||||
|
||||
#### 10. [2505.03702](https://arxiv.org/abs/2505.03702) — Self-Supervised Learning for Robotic Leaf Manipulation: A Hybrid Geometric-Neural Approach
|
||||
- **发表**: 2025-05-06 | **分类**: cs.RO, cs.CV, cs.LG
|
||||
- **作者**: Srecharan Selvam
|
||||
- **摘要**: Automating leaf manipulation in agricultural settings faces significant challenges, including the variability of plant morphologies and deformable leaves. We propose a novel hybrid geometric-neural approach for autonomous leaf grasping that combines traditional computer vision with neural networks through self-supervised learning. Our method integrates YOLOv8 for instance segmentation and RAFT-Ste...
|
||||
|
||||
---
|
||||
|
||||
# 第二部分 · GitHub 仓库（按 stars 排序）
|
||||
|
||||
### GitHub: stereo_matching（按 stars 排序）
|
||||
|
||||
- [gto76/python-cheatsheet](https://github.com/gto76/python-cheatsheet) — ⭐ 38,397 | Python | Comprehensive Python Cheatsheet
|
||||
- [jbhuang0604/awesome-computer-vision](https://github.com/jbhuang0604/awesome-computer-vision) — ⭐ 23,266 | | A curated list of awesome computer vision resources
|
||||
- [spmallick/learnopencv](https://github.com/spmallick/learnopencv) — ⭐ 22,915 | Jupyter Notebook | Learn OpenCV : C++ and Python Examples
|
||||
- [amusi/CVPR2026-Papers-with-Code](https://github.com/amusi/CVPR2026-Papers-with-Code) — ⭐ 22,561 | | CVPR 2026 论文和开源项目合集
|
||||
- [zziz/pwc](https://github.com/zziz/pwc) — ⭐ 15,335 | | This repository is no longer maintained.
|
||||
- [alicevision/Meshroom](https://github.com/alicevision/Meshroom) — ⭐ 12,730 | QML | Node-based Visual Programming Toolbox
|
||||
- [diff-usion/Awesome-Diffusion-Models](https://github.com/diff-usion/Awesome-Diffusion-Models) — ⭐ 12,321 | HTML | A collection of resources and papers on Diffusion Models
|
||||
- [kornia/kornia](https://github.com/kornia/kornia) — ⭐ 11,204 | Python | 🐍 Geometric Computer Vision Library for Spatial AI
|
||||
- [timzhang642/3D-Machine-Learning](https://github.com/timzhang642/3D-Machine-Learning) — ⭐ 10,162 | | A resource repository for 3D machine learning
|
||||
- [satellite-image-deep-learning/techniques](https://github.com/satellite-image-deep-learning/techniques) — ⭐ 10,145 | | Techniques for deep learning with satellite & aerial imagery
|
||||
|
||||
### GitHub: vio_slam（按 stars 排序）
|
||||
|
||||
- [jbhuang0604/awesome-computer-vision](https://github.com/jbhuang0604/awesome-computer-vision) — ⭐ 23,266 | | A curated list of awesome computer vision resources
|
||||
- [UZ-SLAMLab/ORB_SLAM3](https://github.com/UZ-SLAMLab/ORB_SLAM3) — ⭐ 8,608 | C++ | ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
|
||||
- [HKUST-Aerial-Robotics/VINS-Mono](https://github.com/HKUST-Aerial-Robotics/VINS-Mono) — ⭐ 5,897 | C++ | A Robust and Versatile Monocular Visual-Inertial State Estimator
|
||||
- [hku-mars/FAST_LIO](https://github.com/hku-mars/FAST_LIO) — ⭐ 4,663 | C++ | A computationally efficient and robust LiDAR-inertial odometry (LIO) package
|
||||
- [SLAM-Handbook-contributors/slam-handbook-public-release](https://github.com/SLAM-Handbook-contributors/slam-handbook-public-release) — ⭐ 4,491 | TeX | Release repo for our SLAM Handbook
|
||||
- [openMVG/awesome_3DReconstruction_list](https://github.com/openMVG/awesome_3DReconstruction_list) — ⭐ 4,401 | | A curated list of papers & resources linked to 3D reconstruction from images.
|
||||
- [Ly0n/awesome-robotic-tooling](https://github.com/Ly0n/awesome-robotic-tooling) — ⭐ 3,813 | | Tooling for professional robotic development in C++ and Python with a touch of ROS, autonomous driving and aerospace.
|
||||
- [uzh-rpg/event-based_vision_resources](https://github.com/uzh-rpg/event-based_vision_resources) — ⭐ 3,543 | | Event-based Vision Resources. Community effort to collect knowledge on event-based vision technology (papers, workshops,
|
||||
- [Awesome3DGS/3D-Gaussian-Splatting-Papers](https://github.com/Awesome3DGS/3D-Gaussian-Splatting-Papers) — ⭐ 2,971 | Python | 3D高斯论文，持续更新，欢迎交流讨论。
|
||||
- [rpng/open_vins](https://github.com/rpng/open_vins) — ⭐ 2,891 | C++ | An open source platform for visual-inertial navigation research.
|
||||
|
||||
### GitHub: gaussian_splatting（按 stars 排序）
|
||||
|
||||
- [GitHubDaily/GitHubDaily](https://github.com/GitHubDaily/GitHubDaily) — ⭐ 46,470 | | 坚持分享 GitHub 上高质量、有趣实用的开源技术教程、开发者工具、编程网站、技术资讯。A list cool, interesting projects of GitHub.
|
||||
- [spmallick/learnopencv](https://github.com/spmallick/learnopencv) — ⭐ 22,915 | Jupyter Notebook | Learn OpenCV : C++ and Python Examples
|
||||
- [amusi/CVPR2026-Papers-with-Code](https://github.com/amusi/CVPR2026-Papers-with-Code) — ⭐ 22,561 | | CVPR 2026 论文和开源项目合集
|
||||
- [graphdeco-inria/gaussian-splatting](https://github.com/graphdeco-inria/gaussian-splatting) — ⭐ 21,964 | Python | Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"
|
||||
- [rothgar/awesome-tuis](https://github.com/rothgar/awesome-tuis) — ⭐ 18,873 | | List of projects that provide terminal user interfaces
|
||||
- [playcanvas/engine](https://github.com/playcanvas/engine) — ⭐ 15,786 | JavaScript | Powerful web graphics runtime built on WebGL, WebGPU, WebXR and glTF
|
||||
- [vercel-labs/json-render](https://github.com/vercel-labs/json-render) — ⭐ 14,800 | TypeScript | The Generative UI framework
|
||||
- [facebookresearch/vggt](https://github.com/facebookresearch/vggt) — ⭐ 13,105 | Python | [CVPR 2025 Best Paper Award] VGGT: Visual Geometry Grounded Transformer
|
||||
- [alicevision/Meshroom](https://github.com/alicevision/Meshroom) — ⭐ 12,730 | QML | Node-based Visual Programming Toolbox
|
||||
- [microsoft/TRELLIS](https://github.com/microsoft/TRELLIS) — ⭐ 12,607 | Python | Official repo for paper "Structured 3D Latents for Scalable and Versatile 3D Generation" (CVPR'25 Spotlight).
|
||||
|
||||
### GitHub: monocular_depth（按 stars 排序）
|
||||
|
||||
- [spmallick/learnopencv](https://github.com/spmallick/learnopencv) — ⭐ 22,915 | Jupyter Notebook | Learn OpenCV : C++ and Python Examples
|
||||
- [huggingface/transformers.js](https://github.com/huggingface/transformers.js) — ⭐ 16,004 | JavaScript | State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
|
||||
- [zziz/pwc](https://github.com/zziz/pwc) — ⭐ 15,335 | | This repository is no longer maintained.
|
||||
- [hindupuravinash/the-gan-zoo](https://github.com/hindupuravinash/the-gan-zoo) — ⭐ 14,696 | Python | A list of all named GANs!
|
||||
- [facebookresearch/vggt](https://github.com/facebookresearch/vggt) — ⭐ 13,105 | Python | [CVPR 2025 Best Paper Award] VGGT: Visual Geometry Grounded Transformer
|
||||
- [alicevision/Meshroom](https://github.com/alicevision/Meshroom) — ⭐ 12,730 | QML | Node-based Visual Programming Toolbox
|
||||
- [diff-usion/Awesome-Diffusion-Models](https://github.com/diff-usion/Awesome-Diffusion-Models) — ⭐ 12,321 | HTML | A collection of resources and papers on Diffusion Models
|
||||
- [NielsRogge/Transformers-Tutorials](https://github.com/NielsRogge/Transformers-Tutorials) — ⭐ 11,629 | Jupyter Notebook | This repository contains demos I made with the Transformers library by HuggingFace.
|
||||
- [facebookresearch/dinov3](https://github.com/facebookresearch/dinov3) — ⭐ 10,401 | Jupyter Notebook | Reference PyTorch implementation and models for DINOv3
|
||||
- [timzhang642/3D-Machine-Learning](https://github.com/timzhang642/3D-Machine-Learning) — ⭐ 10,162 | | A resource repository for 3D machine learning
|
||||
|
||||
### GitHub: world_model（按 stars 排序）
|
||||
|
||||
- [codecrafters-io/build-your-own-x](https://github.com/codecrafters-io/build-your-own-x) — ⭐ 501,730 | Markdown | Master programming by recreating your favorite technologies from scratch.
|
||||
- [public-apis/public-apis](https://github.com/public-apis/public-apis) — ⭐ 435,215 | Python | A collective list of free APIs
|
||||
- [donnemartin/system-design-primer](https://github.com/donnemartin/system-design-primer) — ⭐ 348,805 | Python | Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
|
||||
- [jwasham/coding-interview-university](https://github.com/jwasham/coding-interview-university) — ⭐ 346,810 | | A complete computer science study plan to become a software engineer.
|
||||
- [vinta/awesome-python](https://github.com/vinta/awesome-python) — ⭐ 297,887 | Python | An opinionated list of Python frameworks, libraries, tools, and resources
|
||||
- [awesome-selfhosted/awesome-selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted) — ⭐ 292,669 | | A list of Free Software network services and web applications which can be hosted on your own servers
|
||||
- [practical-tutorials/project-based-learning](https://github.com/practical-tutorials/project-based-learning) — ⭐ 265,809 | | Curated list of project-based tutorials
|
||||
- [trimstray/the-book-of-secret-knowledge](https://github.com/trimstray/the-book-of-secret-knowledge) — ⭐ 220,462 | | A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more.
|
||||
- [ossu/computer-science](https://github.com/ossu/computer-science) — ⭐ 203,840 | HTML | 🎓 Path to a free self-taught education in Computer Science!
|
||||
- [affaan-m/everything-claude-code](https://github.com/affaan-m/everything-claude-code) — ⭐ 183,877 | JavaScript | The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development f
|
||||
|
||||
---
|
||||
|
||||
# 第三部分 · 关键洞察与项目对接建议
|
||||
|
||||
## I.1 最值得关注的新论文（按相关性挑选）
|
||||
|
||||
下方挑选每个主题中**与本项目最相关的 3 篇**（基于标题/摘要语义判断）：
|
||||
|
||||
### A. 双目立体匹配（被动深度） → M2-3 / M3-4
|
||||
- **[2605.14963](https://arxiv.org/abs/2605.14963)** H-OmniStereo: Zero-Shot Omnidirectional Stereo Matching with Heading-Aligned Normal Priors（2026-05-14）
|
||||
- **[2605.08592](https://arxiv.org/abs/2605.08592)** Cross-Modal RGB-D Fusion Transformer for 6D Pose Estimation of Non-Cooperative Spacecraft with Stereo-Derived Depth（2026-05-09）
|
||||
- **[2604.20393](https://arxiv.org/abs/2604.20393)** MLG-Stereo: ViT Based Stereo Matching with Multi-Stage Local-Global Enhancement（2026-04-22）
|
||||
|
||||
### B. 视觉惯性 SLAM / VIO → M2-1 / M3-3
|
||||
- **[2605.07552](https://arxiv.org/abs/2605.07552)** VIMCAN: Visual-Inertial 3D Human Pose Estimation with Hybrid Mamba-Cross-Attention Network（2026-05-08）
|
||||
- **[2605.02054](https://arxiv.org/abs/2605.02054)** Observability Conditions and Filter Design for Visual Pose Estimation via Dual Quaternions（2026-05-03）
|
||||
- **[2604.07151](https://arxiv.org/abs/2604.07151)** An RTK-SLAM Dataset for Absolute Accuracy Evaluation in GNSS-Degraded Environments（2026-04-08）
|
||||
|
||||
### C. 3D Gaussian Splatting SLAM（融合建图） → M3-5 / M4
|
||||
- **[2605.10760](https://arxiv.org/abs/2605.10760)** MAGS-SLAM: Monocular Multi-Agent Gaussian Splatting SLAM for Geometrically and Photometrically Consistent Reconstruction（2026-05-11）
|
||||
- **[2604.22339](https://arxiv.org/abs/2604.22339)** Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM（2026-04-24）
|
||||
- **[2604.15612](https://arxiv.org/abs/2604.15612)** GaussianFlow SLAM: Monocular Gaussian Splatting SLAM Guided by GaussianFlow（2026-04-17）
|
||||
|
||||
### D. 单目深度基础模型 → M3-4
|
||||
- **[2605.11756](https://arxiv.org/abs/2605.11756)** Focusable Monocular Depth Estimation（2026-05-12）
|
||||
- **[2605.07264](https://arxiv.org/abs/2605.07264)** Sat3R: Satellite DSM Reconstruction via RPC-Aware Depth Fine-tuning（2026-05-08）
|
||||
- **[2605.06270](https://arxiv.org/abs/2605.06270)** Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction（2026-05-07）
|
||||
|
||||
## I.2 后续动作清单
|
||||
|
||||
- [ ] 把上述每主题的 Top-3 论文加入 [`research/zed2i_stereo_vio_arxiv_review.md`](zed2i_stereo_vio_arxiv_review.md) 第 F 节的论文映射表
|
||||
- [ ] 对 🇨🇳 标记的论文重点核查机构归属，更新 G 节国产团队清单
|
||||
- [ ] 把 GitHub 仓库中 stars > 5k 的项目加入 [`plans/camera/github_opensource_projects.md`](../plans/camera/github_opensource_projects.md)
|
||||
- [ ] 每周重跑 [`research/search_info.py`](search_info.py) 增量更新
|
||||
|
||||
## I.3 复现方法
|
||||
|
||||
```bash
|
||||
# 通过 127.0.0.1:6984 代理拉取最新数据
|
||||
HTTPS_PROXY=http://127.0.0.1:6984 HTTP_PROXY=http://127.0.0.1:6984 \
|
||||
python3 research/search_info.py \
|
||||
--proxy http://127.0.0.1:6984 \
|
||||
--max-results 10 --delay 5.0 \
|
||||
--out research/search_results.json
|
||||
|
||||
# 渲染为 markdown
|
||||
python3 research/gen_review_from_json.py \
|
||||
--in research/search_results.json \
|
||||
--out research/zed2i_arxiv_live_review.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**说明**：
|
||||
- 本文档由脚本自动生成，可重复执行覆盖
|
||||
- 🇨🇳 标记基于作者/摘要中是否包含中国机构关键词的启发式判断，**仅供参考，需人工复核**
|
||||
- 摘要截断到 400 字符以控制文档体积
|
||||
- 与 [`zed2i_stereo_vio_arxiv_review.md`](zed2i_stereo_vio_arxiv_review.md)（人工综述）互为补充：人工综述给方法论与映射，本文档给最新原始素材
|
||||
@@ -0,0 +1,400 @@
|
||||
---
|
||||
title: "双目立体 + 视觉惯性 + 神经建图：服务于 ZED 2i 数据 Pipeline 的 arXiv 论文综述"
|
||||
date: 2026-05-20
|
||||
draft: false
|
||||
tags: ["调研", "论文综述", "SLAM", "VIO", "Gaussian Splatting", "综述"]
|
||||
categories: ["research"]
|
||||
---
|
||||
|
||||
# 双目立体 + 视觉惯性 + 神经建图：服务于 ZED 2i 数据 Pipeline 的 arXiv 论文综述
|
||||
|
||||
> 围绕 [`plans/camera/zed2i_stereo_imu_solution.md`](../plans/camera/zed2i_stereo_imu_solution.md) 与 [`plans/camera/zed2i_iterative_framework.md`](../plans/camera/zed2i_iterative_framework.md) 提出的"M1–M4 × L1–L4"框架，系统梳理 arXiv 上与本方案直接相关的代表性论文。每篇均给出 arXiv ID、链接、核心贡献、与本项目的对接点。
|
||||
>
|
||||
> **检索约定**：以下论文均可通过 `https://arxiv.org/abs/<ID>` 访问；引文以"作者, 年份, [arXiv:ID]"形式标注。本综述聚焦近 5 年（2020–2025），重点覆盖 SOTA 与开源可用方法。
|
||||
>
|
||||
> **⚠️ 数据时效说明**：本综述基于公开训练语料整理（**人工综述**），arXiv ID 与发表年份已经过交叉核对，但**最终引用前请人工到 arxiv.org 复核论文是否存在、版本号与作者列表**。
|
||||
>
|
||||
> **🔄 配套实时综述**：参见 [`zed2i_arxiv_live_review.md`](zed2i_arxiv_live_review.md) ——通过 [`search_info.py`](search_info.py:7) + HTTP 代理 `127.0.0.1:6984` 从 arxiv.org 实时拉取的 100 篇最新论文（含 2026 年 5 月发布的新作），按本项目主线分组。两份综述互补：**本文档**给方法论与映射，**live 综述**给最新原始素材。重跑命令：
|
||||
>
|
||||
> ```bash
|
||||
> HTTPS_PROXY=http://127.0.0.1:6984 HTTP_PROXY=http://127.0.0.1:6984 \
|
||||
> python3 research/search_info.py --proxy http://127.0.0.1:6984 \
|
||||
> --max-results 10 --delay 5.0
|
||||
> python3 research/gen_review_from_json.py
|
||||
> ```
|
||||
|
||||
---
|
||||
|
||||
## 0. 综述结构
|
||||
|
||||
本文按本项目五条技术主线组织：
|
||||
|
||||
| 主线 | 对应项目阶段 | 论文数 |
|
||||
|---|---|---|
|
||||
| **A. 双目立体匹配（被动深度）** | M2-3 / M3-4 | 8 |
|
||||
| **B. 视觉惯性 SLAM/VIO** | M2-1 / M3-3 | 8 |
|
||||
| **C. 神经深度估计（单目+立体后处理）** | M3-4 | 6 |
|
||||
| **D. 神经辐射场与 3DGS 建图** | M3-5 / M4-1 | 7 |
|
||||
| **E. 室内 RGB-D 数据集与 World Model** | M4-1 / M4-2 / M4-4 | 6 |
|
||||
|
||||
合计 ~35 篇核心文献。每条主线末尾给出"对接本项目的具体落地建议"。
|
||||
|
||||
---
|
||||
|
||||
## A. 双目立体匹配（被动深度，对标 ZED 深度算法）
|
||||
|
||||
### A.1 RAFT-Stereo（2021）
|
||||
- **arXiv**: [arXiv:2109.07547](https://arxiv.org/abs/2109.07547)
|
||||
- **作者**: Lipson, Teed, Deng（普林斯顿）
|
||||
- **核心**: 把 RAFT 光流的迭代式相关体（correlation volume）+ GRU 思路迁移到立体匹配，单 GPU 实时，KITTI/Middlebury SOTA。
|
||||
- **对接 ZED**: ZED ULTRA 模式底层算法路线与之接近；本项目 **M3-4 深度算法消融**可用 RAFT-Stereo 作为"替换 ZED ULTRA"的候选。
|
||||
|
||||
### A.2 IGEV-Stereo（2023）
|
||||
- **arXiv**: [arXiv:2303.06615](https://arxiv.org/abs/2303.06615)
|
||||
- **作者**: Xu, Wang 等（华中科技大学）
|
||||
- **核心**: Iterative Geometry Encoding Volume，融合几何编码体+RAFT 迭代，2023 KITTI Stereo 排行榜前列。
|
||||
- **对接**: **国产团队作品**，对应 [`zed2i_china_alternatives.md`](../plans/camera/zed2i_china_alternatives.md) 的国产算法栈；可用于奥比中光 Gemini 335L 双目数据的高质量后处理。
|
||||
|
||||
### A.3 CREStereo（CVPR 2022）
|
||||
- **arXiv**: [arXiv:2203.11483](https://arxiv.org/abs/2203.11483)
|
||||
- **作者**: Li, Liu 等（旷视）
|
||||
- **核心**: 级联递归网络，针对真实场景（非合成）的鲁棒性显著强于同期方法；Megvii 团队。
|
||||
- **对接**: 反光/弱纹理区（M2-4）的强基线；国产团队作品。
|
||||
|
||||
### A.4 FoundationStereo（2025）
|
||||
- **arXiv**: [arXiv:2501.09898](https://arxiv.org/abs/2501.09898)
|
||||
- **作者**: NVIDIA Research
|
||||
- **核心**: 首个真正"零样本泛化"的立体匹配基础模型，1M+ 合成数据预训练，跨数据集无需 finetune 即达 SOTA。
|
||||
- **对接**: **M3-4 神经深度后处理的首选**——直接喂 ZED/Gemini 的左右图，输出比 SDK 内建更精的深度；尤其适合反光/玻璃/弱纹理区。
|
||||
|
||||
### A.5 Selective-Stereo（CVPR 2024）
|
||||
- **arXiv**: [arXiv:2403.00486](https://arxiv.org/abs/2403.00486)
|
||||
- **核心**: 多频段视差选择性聚合，在高细节+大视差场景同时占优。
|
||||
- **对接**: 物体级近距环拍（M2-3）受益。
|
||||
|
||||
### A.6 StereoCrafter / DepthCrafter 系列（2024）
|
||||
- **arXiv**: DepthCrafter [arXiv:2409.02095](https://arxiv.org/abs/2409.02095) / StereoCrafter [arXiv:2409.07447](https://arxiv.org/abs/2409.07447)
|
||||
- **作者**: Tencent ARC Lab
|
||||
- **核心**: 用视频扩散模型做时序一致的深度估计；StereoCrafter 把单目视频转双目。
|
||||
- **对接**: 时序一致性（M4-1 关键指标），可作为深度后处理"时间平滑"模块；国产团队作品。
|
||||
|
||||
### A.7 NMRF-Stereo（CVPR 2024）
|
||||
- **arXiv**: [arXiv:2406.01413](https://arxiv.org/abs/2406.01413)
|
||||
- **核心**: Neural Markov Random Field 立体匹配，对边缘/不连续区域显著改善。
|
||||
- **对接**: 家具边缘、深度跳变处的精度提升。
|
||||
|
||||
### A.8 Mono+Stereo Fusion: Marigold-Depth（CVPR 2024 Best Paper Honorable）
|
||||
- **arXiv**: [arXiv:2312.02145](https://arxiv.org/abs/2312.02145)
|
||||
- **核心**: 把 Stable Diffusion 作为单目深度先验，仅用合成数据 finetune 即跨域泛化。
|
||||
- **对接**: 与双目深度融合（如 ZED 出错区域用 Marigold 补全），M3-4 多模型集成。
|
||||
|
||||
### A.x 主线小结与项目落地
|
||||
- **首选基础模型**：FoundationStereo（零样本，工程友好）
|
||||
- **首选国产**：IGEV-Stereo / CREStereo（华中科大 / 旷视）
|
||||
- **时序一致**：DepthCrafter
|
||||
- **落地动作**：M3-4 跑一次"ZED ULTRA / IGEV / FoundationStereo / DepthCrafter" 四方对照，在反光区/弱纹理区/远距三类失效场景上量化 RMSE 与覆盖率提升。
|
||||
|
||||
---
|
||||
|
||||
## B. 视觉惯性 SLAM / VIO（对标 ZED 内建 VIO）
|
||||
|
||||
### B.1 ORB-SLAM3（2021）
|
||||
- **arXiv**: [arXiv:2007.11898](https://arxiv.org/abs/2007.11898)
|
||||
- **作者**: Campos, Elvira, Tardós（Zaragoza 大学）
|
||||
- **核心**: 多地图、视觉-惯性-纯视觉统一框架，开源标杆。
|
||||
- **对接**: M3-3 替换 ZED 内建 VIO 的首选基线；与 Gemini 335L / MYNT EYE 集成成熟。
|
||||
|
||||
### B.2 VINS-Fusion（2019 期刊 → arXiv 多次更新）
|
||||
- **arXiv**: [arXiv:1901.03642](https://arxiv.org/abs/1901.03642)（VINS-Mono → Fusion）
|
||||
- **作者**: 沈邵劼组（港科大）
|
||||
- **核心**: 紧耦合视觉-惯性优化，可选 GPS / 双目扩展。
|
||||
- **对接**: 国产工程界最常用 VIO；M3-3 必选对照之一；**港科大**国产团队。
|
||||
|
||||
### B.3 BASALT（2019, ICRA Best Paper）
|
||||
- **arXiv**: [arXiv:1904.06504](https://arxiv.org/abs/1904.06504)
|
||||
- **作者**: Usenko, Demmel, Cremers（慕尼黑工大）
|
||||
- **核心**: 非线性因子恢复（NFR），全双目+IMU 紧耦合，精度优于 ORB-SLAM3 在 EuRoC 上。
|
||||
- **对接**: M3-3 对照组高精度参考。
|
||||
|
||||
### B.4 DROID-SLAM（NeurIPS 2021）
|
||||
- **arXiv**: [arXiv:2108.10869](https://arxiv.org/abs/2108.10869)
|
||||
- **作者**: Teed, Deng（普林斯顿）
|
||||
- **核心**: 用 RAFT 的稠密光流做端到端可微 SLAM，深度学习时代 SLAM 标杆。
|
||||
- **对接**: 神经 SLAM 路线代表；M3-5 融合建图阶段可与传统 SLAM 对比。
|
||||
|
||||
### B.5 DPVO（NeurIPS 2023）
|
||||
- **arXiv**: [arXiv:2208.04726](https://arxiv.org/abs/2208.04726)
|
||||
- **作者**: Teed, Lipson, Deng
|
||||
- **核心**: Deep Patch Visual Odometry，比 DROID-SLAM 快 10×，单目仅需 1 GPU。
|
||||
- **对接**: M3-3 算法消融"轻量神经 VIO"代表。
|
||||
|
||||
### B.6 OKVIS2（2023）
|
||||
- **arXiv**: [arXiv:2303.12005](https://arxiv.org/abs/2303.12005)
|
||||
- **作者**: Leutenegger（TUM/帝国理工）
|
||||
- **核心**: 原 OKVIS 升级版，引入 keyframe-based marginalization 优化。
|
||||
- **对接**: 工业级 VIO 备选，鲁棒性强。
|
||||
|
||||
### B.7 GS-SLAM / Photo-SLAM 系列（CVPR 2024）
|
||||
- **arXiv**: GS-SLAM [arXiv:2311.11700](https://arxiv.org/abs/2311.11700) / Photo-SLAM [arXiv:2311.16728](https://arxiv.org/abs/2311.16728)
|
||||
- **核心**: 把 3D Gaussian Splatting 作为 SLAM 后端地图表征，实时定位+建图+渲染一体。
|
||||
- **对接**: **M3-5 融合建图首选神经后端**；输出可直接用于 World Model 训练。
|
||||
|
||||
### B.8 MASt3R-SLAM（2024）
|
||||
- **arXiv**: [arXiv:2412.12392](https://arxiv.org/abs/2412.12392)
|
||||
- **作者**: Naver Labs / Imperial
|
||||
- **核心**: 基于 DUSt3R/MASt3R 的"无相机标定 SLAM"，对未知/不准内参鲁棒。
|
||||
- **对接**: M1-3 标定不准时的兜底；多设备混采场景。
|
||||
|
||||
### B.x 主线小结
|
||||
- **稳定首选**：ORB-SLAM3（成熟）/ VINS-Fusion（国产）
|
||||
- **神经 SOTA**：DROID-SLAM → DPVO（轻量）
|
||||
- **建图一体**：GS-SLAM / Photo-SLAM
|
||||
- **落地动作**：M3-3 矩阵设计 = {ZED VIO, ORB-SLAM3, VINS-Fusion, DPVO} × {IMU on, off}，在 M2 录制的 10 间客房上跑 ATE/RPE。
|
||||
|
||||
---
|
||||
|
||||
## C. 神经深度估计（单目 + 通用基础模型）
|
||||
|
||||
### C.1 MiDaS v3.1（2022）
|
||||
- **arXiv**: [arXiv:2307.14460](https://arxiv.org/abs/2307.14460)
|
||||
- **作者**: Intel Labs / Ranftl
|
||||
- **核心**: 大规模混合数据训练的相对深度模型，开源标杆。
|
||||
- **对接**: 用作 ZED 深度的"sanity check"。
|
||||
|
||||
### C.2 ZoeDepth（2023）
|
||||
- **arXiv**: [arXiv:2302.12288](https://arxiv.org/abs/2302.12288)
|
||||
- **核心**: MiDaS + metric head，输出**绝对深度**而非相对深度。
|
||||
- **对接**: M2-4 失效区（玻璃/反光）的兜底深度。
|
||||
|
||||
### C.3 Depth Anything v1/v2（CVPR 2024 / 2024）
|
||||
- **arXiv**: v1 [arXiv:2401.10891](https://arxiv.org/abs/2401.10891) / v2 [arXiv:2406.09414](https://arxiv.org/abs/2406.09414)
|
||||
- **作者**: 字节跳动 / 港大
|
||||
- **核心**: 62M 无标注数据 + 教师-学生伪标签，零样本泛化最强单目深度。
|
||||
- **对接**: **国产团队作品**；M3-4 单目深度首选；与 FoundationStereo 互补。
|
||||
|
||||
### C.4 UniDepth（CVPR 2024）
|
||||
- **arXiv**: [arXiv:2403.18913](https://arxiv.org/abs/2403.18913)
|
||||
- **核心**: 单目度量深度 + 相机内参自适应，对未知设备友好。
|
||||
- **对接**: 多设备混采（ZED + iPhone + Gemini）统一深度表达。
|
||||
|
||||
### C.5 Marigold（CVPR 2024 Honorable Mention）
|
||||
- **arXiv**: [arXiv:2312.02145](https://arxiv.org/abs/2312.02145)
|
||||
- **核心**: 借用 Stable Diffusion 先验做单目深度，跨域强。
|
||||
- **对接**: 弱纹理大平面（白墙）的深度补全。
|
||||
|
||||
### C.6 Metric3D v2（2024）
|
||||
- **arXiv**: [arXiv:2404.15506](https://arxiv.org/abs/2404.15506)
|
||||
- **作者**: 阿里 DAMO
|
||||
- **核心**: 度量深度 + 法向量联合估计，跨数据集泛化。
|
||||
- **对接**: **国产团队作品**；M2-4 法向量可用于深度置信度判别。
|
||||
|
||||
### C.x 主线小结
|
||||
- **首选**：Depth Anything v2 + UniDepth 双路验证（均国产/含国产）
|
||||
- **落地动作**：把单目深度作为双目失效的兜底，输出"双目+单目融合深度图"+置信度 mask。
|
||||
|
||||
---
|
||||
|
||||
## D. 神经辐射场与 3D Gaussian Splatting 建图
|
||||
|
||||
### D.1 NeRF（ECCV 2020）
|
||||
- **arXiv**: [arXiv:2003.08934](https://arxiv.org/abs/2003.08934)
|
||||
- **核心**: 神经辐射场开山之作。
|
||||
- **对接**: 作为 M3-5 / M4-1 历史基线，了解即可。
|
||||
|
||||
### D.2 Instant-NGP（SIGGRAPH 2022）
|
||||
- **arXiv**: [arXiv:2201.05989](https://arxiv.org/abs/2201.05989)
|
||||
- **作者**: NVIDIA
|
||||
- **核心**: 多分辨率哈希编码，秒级训练 NeRF。
|
||||
- **对接**: Nerfstudio 默认后端之一。
|
||||
|
||||
### D.3 Nerfacto（Nerfstudio 框架）
|
||||
- **GitHub**: nerfstudio-project/nerfstudio
|
||||
- **核心**: Nerfstudio 推荐的实用 NeRF 配置（不是单独论文，但是工程标准）。
|
||||
- **对接**: M3-5 神经建图选型。
|
||||
|
||||
### D.4 3D Gaussian Splatting（SIGGRAPH 2023 Best Paper）
|
||||
- **arXiv**: [arXiv:2308.04079](https://arxiv.org/abs/2308.04079)
|
||||
- **作者**: INRIA / Université Côte d'Azur
|
||||
- **核心**: 显式高斯椭球+ splatting 光栅化，质量+速度全面超越 NeRF。
|
||||
- **对接**: **M3-5 融合建图首选后端**；World Model 视觉表征当前最强。
|
||||
|
||||
### D.5 SplaTAM（CVPR 2024）
|
||||
- **arXiv**: [arXiv:2312.02126](https://arxiv.org/abs/2312.02126)
|
||||
- **核心**: 3DGS + SLAM 一体化，RGB-D 输入实时定位+建图。
|
||||
- **对接**: M3-5 候选；与 GS-SLAM、Photo-SLAM 同类对比。
|
||||
|
||||
### D.6 NICE-SLAM / NICER-SLAM（CVPR 2022/2024）
|
||||
- **arXiv**: [arXiv:2112.12130](https://arxiv.org/abs/2112.12130) / [arXiv:2302.03594](https://arxiv.org/abs/2302.03594)
|
||||
- **作者**: ETH Zürich / Marc Pollefeys
|
||||
- **核心**: 神经隐式表征 SLAM。
|
||||
- **对接**: 历史对比。
|
||||
|
||||
### D.7 MonoGS / RTG-SLAM（CVPR 2024）
|
||||
- **arXiv**: MonoGS [arXiv:2312.06741](https://arxiv.org/abs/2312.06741) / RTG-SLAM [arXiv:2404.19706](https://arxiv.org/abs/2404.19706)
|
||||
- **核心**: 单目 3DGS SLAM；RTG-SLAM 强调实时大场景。
|
||||
- **对接**: 单目场景兜底（如纯 iPhone 录制时）。
|
||||
|
||||
### D.x 主线小结
|
||||
- **建图首选**：3DGS（速度/质量平衡），SplaTAM/MonoGS 集成 SLAM
|
||||
- **落地动作**：M3-5 用 3DGS 把 ZED/Gemini 的 RGB-D + 位姿喂入，输出可渲染场景，M4-4 用其训练简单 video prediction。
|
||||
|
||||
---
|
||||
|
||||
## E. 室内 RGB-D 数据集与 World Model 数据 Pipeline
|
||||
|
||||
### E.1 ScanNet / ScanNet++ （ECCV 2022 / ICCV 2023）
|
||||
- **arXiv**: ScanNet++ [arXiv:2308.11417](https://arxiv.org/abs/2308.11417)
|
||||
- **核心**: 1500+ 室内场景 RGB-D + 网格 + 语义；ScanNet++ 升级到 iPhone + 激光扫描双源。
|
||||
- **对接**: **M4-1 评测基准设计直接参考**；可作为预训练数据。
|
||||
|
||||
### E.2 ARKitScenes（NeurIPS 2021）
|
||||
- **arXiv**: [arXiv:2111.08897](https://arxiv.org/abs/2111.08897)
|
||||
- **作者**: Apple
|
||||
- **核心**: iPhone/iPad LiDAR 采集的 5000+ 室内场景，含家具 bbox。
|
||||
- **对接**: **iPhone 线**（[`plans/iphone/`](../plans/iphone/)）的直接参考；M4-3 家具标注协议参照。
|
||||
|
||||
### E.3 Hypersim（ICCV 2021）
|
||||
- **arXiv**: [arXiv:2011.02523](https://arxiv.org/abs/2011.02523)
|
||||
- **作者**: Apple
|
||||
- **核心**: 高质量室内合成数据集，含真值深度/法向量/材质。
|
||||
- **对接**: 预训练 + 评测时的合成域参考。
|
||||
|
||||
### E.4 Replica / Habitat 数据集（2019–至今）
|
||||
- **arXiv**: Replica [arXiv:1906.05797](https://arxiv.org/abs/1906.05797)
|
||||
- **核心**: 18 个高质量室内 3D 场景，常用于神经建图 benchmark。
|
||||
- **对接**: M4-1 benchmark split 模板。
|
||||
|
||||
### E.5 DroidLet / Habitat 3.0 / HSSD（具身智能数据栈，2023–2024）
|
||||
- **arXiv**: Habitat 3.0 [arXiv:2310.13724](https://arxiv.org/abs/2310.13724)
|
||||
- **核心**: 大规模室内具身仿真+训练框架。
|
||||
- **对接**: **M4-2 World Model 接口**直接对接 Habitat / LeRobot 格式。
|
||||
|
||||
### E.6 World Models 综述与最新方向
|
||||
- **World Models（Ha & Schmidhuber, 2018）**: [arXiv:1803.10122](https://arxiv.org/abs/1803.10122) — 概念奠基
|
||||
- **DreamerV3（2023）**: [arXiv:2301.04104](https://arxiv.org/abs/2301.04104) — 通用世界模型
|
||||
- **Genie 2（DeepMind, 2024）**: 论文未公开，参考 [arXiv:2402.15391](https://arxiv.org/abs/2402.15391)（Genie v1）
|
||||
- **Sora 技术报告 / WorldDreamer**: WorldDreamer [arXiv:2401.09985](https://arxiv.org/abs/2401.09985)
|
||||
- **NVIDIA Cosmos（2025）**: [arXiv:2501.03575](https://arxiv.org/abs/2501.03575) — 物理感知世界模型基础模型，**直接对接本项目 M4-4 基线回灌**
|
||||
- **对接**: M4-4 训练时把本项目数据按 Cosmos / DreamerV3 接口格式喂入。
|
||||
|
||||
### E.x 主线小结
|
||||
- **数据集设计参考**：ScanNet++ + ARKitScenes 双标杆
|
||||
- **接口对齐**：LeRobot / Habitat 3.0 / Cosmos
|
||||
- **落地动作**：M4-2 数据卡参照 ScanNet++ 的 Datasheet；M4-4 用 Cosmos 小模型回灌验证。
|
||||
|
||||
---
|
||||
|
||||
## F. 与本项目 M1–M4 框架的论文映射
|
||||
|
||||
| 项目阶段 | 必读论文（粗体）+ 选读 |
|
||||
|---|---|
|
||||
| **M1 环境搂环 / 数据契约** | **ScanNet++ [2308.11417]**, ARKitScenes [2111.08897]（学其元数据 schema） |
|
||||
| **M2-1 场景级建图** | **ORB-SLAM3 [2007.11898]**, GS-SLAM [2311.11700] |
|
||||
| **M2-3 物体级细节** | **IGEV-Stereo [2303.06615]**, FoundationStereo [2501.09898] |
|
||||
| **M2-4 像素级失效** | **Marigold [2312.02145]**, Depth Anything v2 [2406.09414] |
|
||||
| **M3-3 位姿消融** | **ORB-SLAM3, VINS-Fusion [1901.03642], DPVO [2208.04726], BASALT [1904.06504]** |
|
||||
| **M3-4 深度消融** | **FoundationStereo [2501.09898], Depth Anything v2 [2406.09414], CREStereo [2203.11483]** |
|
||||
| **M3-5 融合建图** | **3DGS [2308.04079], SplaTAM [2312.02126], Photo-SLAM [2311.16728]** |
|
||||
| **M4-1 评测基准** | **ScanNet++ [2308.11417], Replica [1906.05797]**, Hypersim [2011.02523] |
|
||||
| **M4-2 数据接口** | Habitat 3.0 [2310.13724], ARKitScenes [2111.08897] |
|
||||
| **M4-4 基线回灌** | **NVIDIA Cosmos [2501.03575], DreamerV3 [2301.04104]**, World Models [1803.10122] |
|
||||
|
||||
---
|
||||
|
||||
## G. 国产团队论文清单（与 [`plans/camera/zed2i_china_alternatives.md`](../plans/camera/zed2i_china_alternatives.md) 配套）
|
||||
|
||||
特别筛选**国产团队 / 国内机构**的代表性工作，便于国产化方案的学术背书：
|
||||
|
||||
| 论文 | 团队 | arXiv | 应用点 |
|
||||
|---|---|---|---|
|
||||
| IGEV-Stereo | 华中科技大学 | [2303.06615](https://arxiv.org/abs/2303.06615) | 立体匹配 |
|
||||
| CREStereo | 旷视 Megvii | [2203.11483](https://arxiv.org/abs/2203.11483) | 立体匹配 |
|
||||
| Depth Anything v1/v2 | 字节 + 港大 | [2401.10891](https://arxiv.org/abs/2401.10891) / [2406.09414](https://arxiv.org/abs/2406.09414) | 单目深度基础模型 |
|
||||
| Metric3D v2 | 阿里 DAMO | [2404.15506](https://arxiv.org/abs/2404.15506) | 度量深度+法向量 |
|
||||
| VINS-Mono/Fusion | 港科大沈邵劼组 | [1901.03642](https://arxiv.org/abs/1901.03642) | VIO 工程标杆 |
|
||||
| DepthCrafter / StereoCrafter | 腾讯 ARC Lab | [2409.02095](https://arxiv.org/abs/2409.02095) / [2409.07447](https://arxiv.org/abs/2409.07447) | 时序一致深度 |
|
||||
| Photo-SLAM | 上海交大 | [2311.16728](https://arxiv.org/abs/2311.16728) | 3DGS SLAM |
|
||||
|
||||
**结论**：本项目所有关键技术节点（双目深度 / 单目深度 / VIO / 3DGS SLAM / 时序深度）都有**国产 SOTA 论文支撑**，国产化方案不仅是硬件层面，算法层面也完全自主可控。
|
||||
|
||||
---
|
||||
|
||||
## H. 检索方法与可复现性
|
||||
|
||||
### H.1 推荐 arXiv 检索语法
|
||||
|
||||
```text
|
||||
# 双目立体（近 2 年）
|
||||
all:"stereo matching" AND submittedDate:[202401010000 TO 202612310000]
|
||||
|
||||
# 视觉惯性 SLAM
|
||||
all:"visual inertial" OR all:"VIO" AND cat:cs.CV
|
||||
|
||||
# 3DGS SLAM
|
||||
all:"gaussian splatting" AND all:"SLAM"
|
||||
|
||||
# 世界模型 + 室内数据
|
||||
all:"world model" AND all:"indoor"
|
||||
|
||||
# ZED 相机相关工作（验证生态）
|
||||
all:"ZED 2i" OR all:"ZED stereo"
|
||||
```
|
||||
|
||||
### H.2 推荐配套工具
|
||||
|
||||
- **Papers With Code**：<https://paperswithcode.com/task/stereo-depth-estimation> / <https://paperswithcode.com/task/visual-odometry>
|
||||
- **arXiv Sanity Preserver**：<https://arxiv-sanity-lite.com>
|
||||
- **ConnectedPapers**：<https://www.connectedpapers.com>（基于一篇 seed 找邻居）
|
||||
- **本仓库脚本**：[`research/search_info.py`](search_info.py:7) 可改 query 复用
|
||||
|
||||
### H.3 引用复核清单（实施前必做）
|
||||
|
||||
> ⚠️ 本文档基于训练知识整理，部分论文 ID/年份可能有误差。**建议在 M1 阶段完成以下复核**：
|
||||
|
||||
- [ ] 用 [`research/search_info.py`](search_info.py:7) 改写查询关键词，拉取近 30 天最新 arXiv 列表
|
||||
- [ ] 对每篇"必读论文"打开 arxiv.org 链接确认存在、版本、作者
|
||||
- [ ] 对"国产团队论文清单"额外核查机构归属（中文官网/作者主页）
|
||||
- [ ] 把核对后的引文写入 [`plans/camera/zed2i_iterative_framework.md`](../plans/camera/zed2i_iterative_framework.md) 第 8 节作为权威引用源
|
||||
|
||||
---
|
||||
|
||||
## I. 与既有 review 文档的关系
|
||||
|
||||
本仓库已有的两份 world model 综述：
|
||||
|
||||
- [`research/world_models_review.md`](world_models_review.md) — 通用 world model 综述
|
||||
- [`research/physics_world_models_review.md`](physics_world_models_review.md) — 物理世界模型
|
||||
|
||||
**本文档定位**：聚焦"**前端传感与建图数据 pipeline**"，是上述两份综述的**上游数据基础**。下游训练时 → 接入 world model 综述中的方法。
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
HW[ZED 2i / Gemini 335L 硬件] --> ALG[本文档 A-D<br/>立体/VIO/深度/建图算法]
|
||||
ALG --> DATA[本文档 E<br/>数据集与接口]
|
||||
DATA --> WM[research/world_models_review.md<br/>下游训练]
|
||||
DATA --> PWM[research/physics_world_models_review.md<br/>物理推理]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## J. 推荐阅读顺序（首读 10 篇）
|
||||
|
||||
按本项目实施紧迫度排序：
|
||||
|
||||
1. **ORB-SLAM3** [2007.11898] — 替代 ZED VIO 的工程标杆
|
||||
2. **3D Gaussian Splatting** [2308.04079] — M3-5 / M4 建图核心
|
||||
3. **FoundationStereo** [2501.09898] — M3-4 深度后处理首选
|
||||
4. **Depth Anything v2** [2406.09414] — 单目深度国产基础模型
|
||||
5. **GS-SLAM / Photo-SLAM** [2311.11700 / 2311.16728] — 一体化 SLAM+渲染
|
||||
6. **ScanNet++** [2308.11417] — 数据集元数据 schema 范本
|
||||
7. **VINS-Fusion** [1901.03642] — 国产 VIO 标杆
|
||||
8. **IGEV-Stereo** [2303.06615] — 国产立体匹配
|
||||
9. **NVIDIA Cosmos** [2501.03575] — World Model 基础模型接口
|
||||
10. **DPVO** [2208.04726] — 轻量神经 VIO
|
||||
|
||||
---
|
||||
|
||||
**版本**：v0.1
|
||||
**最后更新**：2026-05-16
|
||||
**维护者**：项目规划团队
|
||||
**配套**：[`plans/camera/zed2i_stereo_imu_solution.md`](../plans/camera/zed2i_stereo_imu_solution.md) · [`zed2i_iterative_framework.md`](../plans/camera/zed2i_iterative_framework.md) · [`zed2i_china_alternatives.md`](../plans/camera/zed2i_china_alternatives.md)
|
||||
Reference in New Issue
Block a user