chore: initial commit — import worldmodel workspace (plans/, research/)
This commit is contained in:
gaojie
@@ -0,0 +1,977 @@
|
||||
# 摄像头方案对应的GitHub开源项目
|
||||
|
||||
## 📋 研究概述
|
||||
|
||||
本文档整理了与各类摄像头方案对应的GitHub成熟开源项目，提供完整的软件工具链支持。
|
||||
|
||||
**研究范围**：GitHub Stars > 500的活跃项目
|
||||
**更新时间**：2026-05-16
|
||||
|
||||
---
|
||||
|
||||
## 一、单目相机开源项目
|
||||
|
||||
### 1.1 COLMAP ⭐⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/colmap/colmap
|
||||
**Stars**: ~7,000
|
||||
**语言**: C++
|
||||
**许可证**: BSD-3-Clause
|
||||
|
||||
#### 项目简介
|
||||
|
||||
COLMAP是最成熟的单目SfM（Structure-from-Motion）和MVS（Multi-View Stereo）系统，被学术界和工业界广泛使用。
|
||||
|
||||
```yaml
|
||||
核心功能:
|
||||
- 特征提取与匹配
|
||||
- 增量式SfM重建
|
||||
- 稠密MVS重建
|
||||
- 网格生成
|
||||
|
||||
支持相机:
|
||||
- 所有单目相机
|
||||
- 鱼眼镜头
|
||||
- 全景相机
|
||||
|
||||
优势:
|
||||
- 鲁棒性强
|
||||
- 精度高
|
||||
- 文档完善
|
||||
- GUI + CLI
|
||||
|
||||
适用场景:
|
||||
- 照片建模
|
||||
- 文化遗产数字化
|
||||
- 影视特效
|
||||
```
|
||||
|
||||
#### 使用示例
|
||||
|
||||
```bash
|
||||
# 完整重建流程
|
||||
# 1. 特征提取
|
||||
colmap feature_extractor \
|
||||
--database_path database.db \
|
||||
--image_path images/ \
|
||||
--ImageReader.camera_model PINHOLE \
|
||||
--ImageReader.single_camera 1
|
||||
|
||||
# 2. 特征匹配
|
||||
colmap exhaustive_matcher \
|
||||
--database_path database.db \
|
||||
--SiftMatching.guided_matching 1
|
||||
|
||||
# 3. 稀疏重建
|
||||
colmap mapper \
|
||||
--database_path database.db \
|
||||
--image_path images/ \
|
||||
--output_path sparse/
|
||||
|
||||
# 4. 图像去畸变
|
||||
colmap image_undistorter \
|
||||
--image_path images/ \
|
||||
--input_path sparse/0 \
|
||||
--output_path dense/ \
|
||||
--output_type COLMAP
|
||||
|
||||
# 5. 稠密重建
|
||||
colmap patch_match_stereo \
|
||||
--workspace_path dense/ \
|
||||
--workspace_format COLMAP \
|
||||
--PatchMatchStereo.geom_consistency true
|
||||
|
||||
# 6. 点云融合
|
||||
colmap stereo_fusion \
|
||||
--workspace_path dense/ \
|
||||
--workspace_format COLMAP \
|
||||
--input_type geometric \
|
||||
--output_path dense/fused.ply
|
||||
|
||||
# 7. Mesh生成
|
||||
colmap poisson_mesher \
|
||||
--input_path dense/fused.ply \
|
||||
--output_path dense/meshed.ply
|
||||
```
|
||||
|
||||
#### Python接口
|
||||
|
||||
```python
|
||||
# pycolmap - Python绑定
|
||||
import pycolmap
|
||||
|
||||
# 运行SfM
|
||||
reconstruction = pycolmap.incremental_mapping(
|
||||
database_path="database.db",
|
||||
image_path="images/",
|
||||
output_path="sparse/"
|
||||
)
|
||||
|
||||
# 访问重建结果
|
||||
for image_id, image in reconstruction.images.items():
|
||||
print(f"Image {image_id}: {image.name}")
|
||||
print(f"Camera pose: {image.cam_from_world}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 1.2 OpenMVG ⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/openMVG/openMVG
|
||||
**Stars**: ~5,500
|
||||
**语言**: C++
|
||||
**许可证**: MPL-2.0
|
||||
|
||||
#### 项目简介
|
||||
|
||||
OpenMVG（Open Multiple View Geometry）是另一个强大的SfM库，强调模块化和可扩展性。
|
||||
|
||||
```yaml
|
||||
核心功能:
|
||||
- 多种SfM算法
|
||||
- 增量式/全局式重建
|
||||
- 相机标定
|
||||
- 特征匹配
|
||||
|
||||
优势:
|
||||
- 模块化设计
|
||||
- 算法多样
|
||||
- 易于扩展
|
||||
- 教学友好
|
||||
|
||||
与COLMAP对比:
|
||||
- 更模块化
|
||||
- 算法选择多
|
||||
- 速度稍慢
|
||||
- 精度相当
|
||||
```
|
||||
|
||||
#### 使用示例
|
||||
|
||||
```bash
|
||||
# OpenMVG pipeline
|
||||
# 1. 图像列表
|
||||
openMVG_main_SfMInit_ImageListing \
|
||||
-i images/ \
|
||||
-o matches/ \
|
||||
-d sensor_width_database.txt
|
||||
|
||||
# 2. 特征提取
|
||||
openMVG_main_ComputeFeatures \
|
||||
-i matches/sfm_data.json \
|
||||
-o matches/ \
|
||||
-m SIFT
|
||||
|
||||
# 3. 特征匹配
|
||||
openMVG_main_ComputeMatches \
|
||||
-i matches/sfm_data.json \
|
||||
-o matches/
|
||||
|
||||
# 4. 增量式SfM
|
||||
openMVG_main_IncrementalSfM \
|
||||
-i matches/sfm_data.json \
|
||||
-m matches/ \
|
||||
-o reconstruction/
|
||||
|
||||
# 5. 导出为COLMAP格式
|
||||
openMVG_main_openMVG2COLMAP \
|
||||
-i reconstruction/sfm_data.bin \
|
||||
-o colmap/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 1.3 Meshroom ⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/alicevision/Meshroom
|
||||
**Stars**: ~11,000
|
||||
**语言**: Python/C++
|
||||
**许可证**: MPL-2.0
|
||||
|
||||
#### 项目简介
|
||||
|
||||
Meshroom是基于AliceVision的开源3D重建软件，提供完整的GUI界面，零代码操作。
|
||||
|
||||
```yaml
|
||||
核心功能:
|
||||
- 全自动3D重建
|
||||
- 可视化节点编辑器
|
||||
- 实时预览
|
||||
- 纹理映射
|
||||
|
||||
优势:
|
||||
- 完全免费
|
||||
- GUI友好
|
||||
- 质量高
|
||||
- 社区活跃
|
||||
|
||||
适用场景:
|
||||
- 非技术用户
|
||||
- 快速原型
|
||||
- 教学演示
|
||||
- 艺术创作
|
||||
```
|
||||
|
||||
#### 使用方法
|
||||
|
||||
```bash
|
||||
# 1. 下载安装
|
||||
# https://github.com/alicevision/Meshroom/releases
|
||||
|
||||
# 2. 启动GUI
|
||||
./Meshroom
|
||||
|
||||
# 3. 拖拽图片到界面
|
||||
# 4. 点击"Start"自动重建
|
||||
# 5. 导出OBJ/FBX模型
|
||||
|
||||
# 命令行模式
|
||||
meshroom_batch \
|
||||
--input images/ \
|
||||
--output output/ \
|
||||
--save output/project.mg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 二、双目相机开源项目
|
||||
|
||||
### 2.1 ORB-SLAM3 ⭐⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/UZ-SLAMLab/ORB_SLAM3
|
||||
**Stars**: ~6,000
|
||||
**语言**: C++
|
||||
**许可证**: GPLv3
|
||||
|
||||
#### 项目简介
|
||||
|
||||
ORB-SLAM3是最先进的视觉SLAM系统，支持单目、双目、RGB-D和IMU融合。
|
||||
|
||||
```yaml
|
||||
核心功能:
|
||||
- 实时SLAM
|
||||
- 回环检测
|
||||
- 重定位
|
||||
- 地图保存/加载
|
||||
- 多地图管理
|
||||
|
||||
支持传感器:
|
||||
- 单目相机
|
||||
- 双目相机
|
||||
- RGB-D相机
|
||||
- 单目+IMU
|
||||
- 双目+IMU
|
||||
|
||||
优势:
|
||||
- 精度最高
|
||||
- 鲁棒性强
|
||||
- 实时性好
|
||||
- 学术标准
|
||||
|
||||
适用场景:
|
||||
- 机器人导航
|
||||
- AR/VR
|
||||
- 自动驾驶
|
||||
- 无人机
|
||||
```
|
||||
|
||||
#### 使用示例
|
||||
|
||||
```bash
|
||||
# 编译
|
||||
cd ORB_SLAM3
|
||||
chmod +x build.sh
|
||||
./build.sh
|
||||
|
||||
# 双目相机运行
|
||||
./Examples/Stereo/stereo_euroc \
|
||||
Vocabulary/ORBvoc.txt \
|
||||
Examples/Stereo/EuRoC.yaml \
|
||||
dataset/MH01 \
|
||||
Examples/Stereo/EuRoC_TimeStamps/MH01.txt \
|
||||
dataset-MH01_stereo
|
||||
|
||||
# 双目+IMU运行
|
||||
./Examples/Stereo-Inertial/stereo_inertial_euroc \
|
||||
Vocabulary/ORBvoc.txt \
|
||||
Examples/Stereo-Inertial/EuRoC.yaml \
|
||||
dataset/MH01 \
|
||||
Examples/Stereo-Inertial/EuRoC_TimeStamps/MH01.txt \
|
||||
dataset-MH01_stereoi
|
||||
```
|
||||
|
||||
#### Python绑定
|
||||
|
||||
```python
|
||||
# 使用orbslam3_python
|
||||
import orbslam3
|
||||
|
||||
# 初始化
|
||||
slam = orbslam3.System(
|
||||
vocab_file="Vocabulary/ORBvoc.txt",
|
||||
settings_file="Examples/Stereo/EuRoC.yaml",
|
||||
sensor_type=orbslam3.Sensor.STEREO
|
||||
)
|
||||
|
||||
# 处理帧
|
||||
for left_img, right_img, timestamp in stereo_stream:
|
||||
pose = slam.process_image_stereo(
|
||||
left_img, right_img, timestamp
|
||||
)
|
||||
|
||||
if pose is not None:
|
||||
print(f"Camera pose: {pose}")
|
||||
|
||||
# 保存地图
|
||||
slam.save_map("map.bin")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2.2 OpenCV Stereo ⭐⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/opencv/opencv
|
||||
**Stars**: ~77,000
|
||||
**语言**: C++/Python
|
||||
**许可证**: Apache 2.0
|
||||
|
||||
#### 项目简介
|
||||
|
||||
OpenCV提供了完整的双目视觉工具链，从标定到深度计算。
|
||||
|
||||
```yaml
|
||||
核心模块:
|
||||
- calib3d: 相机标定
|
||||
- stereo: 立体匹配
|
||||
- 3d: 点云处理
|
||||
|
||||
算法支持:
|
||||
- StereoBM: 块匹配
|
||||
- StereoSGBM: 半全局匹配
|
||||
- StereoBeliefPropagation: 置信传播
|
||||
- StereoConstantSpaceBP: 恒定空间BP
|
||||
|
||||
优势:
|
||||
- 文档完善
|
||||
- 社区庞大
|
||||
- 跨平台
|
||||
- 性能优化
|
||||
```
|
||||
|
||||
#### 完整示例
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
class StereoVision:
|
||||
"""双目视觉系统"""
|
||||
|
||||
def __init__(self, calib_file):
|
||||
# 加载标定参数
|
||||
calib = np.load(calib_file)
|
||||
self.K_left = calib['K_left']
|
||||
self.dist_left = calib['dist_left']
|
||||
self.K_right = calib['K_right']
|
||||
self.dist_right = calib['dist_right']
|
||||
self.R = calib['R']
|
||||
self.T = calib['T']
|
||||
|
||||
# 计算校正映射
|
||||
self.R_left, self.R_right, self.P_left, self.P_right, self.Q, \
|
||||
self.roi_left, self.roi_right = cv2.stereoRectify(
|
||||
self.K_left, self.dist_left,
|
||||
self.K_right, self.dist_right,
|
||||
(1280, 720), self.R, self.T,
|
||||
alpha=0
|
||||
)
|
||||
|
||||
self.map_left_x, self.map_left_y = cv2.initUndistortRectifyMap(
|
||||
self.K_left, self.dist_left, self.R_left,
|
||||
self.P_left, (1280, 720), cv2.CV_32FC1
|
||||
)
|
||||
|
||||
self.map_right_x, self.map_right_y = cv2.initUndistortRectifyMap(
|
||||
self.K_right, self.dist_right, self.R_right,
|
||||
self.P_right, (1280, 720), cv2.CV_32FC1
|
||||
)
|
||||
|
||||
# 创建立体匹配器
|
||||
self.stereo = cv2.StereoSGBM_create(
|
||||
minDisparity=0,
|
||||
numDisparities=128,
|
||||
blockSize=5,
|
||||
P1=8 * 3 * 5**2,
|
||||
P2=32 * 3 * 5**2,
|
||||
disp12MaxDiff=1,
|
||||
uniquenessRatio=10,
|
||||
speckleWindowSize=100,
|
||||
speckleRange=32,
|
||||
mode=cv2.STEREO_SGBM_MODE_SGBM_3WAY
|
||||
)
|
||||
|
||||
def compute_depth(self, left_img, right_img):
|
||||
"""计算深度图"""
|
||||
# 校正
|
||||
left_rect = cv2.remap(
|
||||
left_img, self.map_left_x, self.map_left_y,
|
||||
cv2.INTER_LINEAR
|
||||
)
|
||||
right_rect = cv2.remap(
|
||||
right_img, self.map_right_x, self.map_right_y,
|
||||
cv2.INTER_LINEAR
|
||||
)
|
||||
|
||||
# 立体匹配
|
||||
disparity = self.stereo.compute(
|
||||
left_rect, right_rect
|
||||
).astype(np.float32) / 16.0
|
||||
|
||||
# 视差转深度
|
||||
depth = cv2.reprojectImageTo3D(disparity, self.Q)
|
||||
|
||||
return depth, disparity
|
||||
|
||||
def get_pointcloud(self, left_img, right_img):
|
||||
"""生成点云"""
|
||||
depth, disparity = self.compute_depth(left_img, right_img)
|
||||
|
||||
# 过滤无效点
|
||||
mask = (disparity > 0) & (disparity < 128)
|
||||
|
||||
points = depth[mask]
|
||||
colors = left_img[mask] / 255.0
|
||||
|
||||
return points, colors
|
||||
|
||||
# 使用
|
||||
stereo = StereoVision('stereo_calib.npz')
|
||||
depth, disparity = stereo.compute_depth(left_img, right_img)
|
||||
points, colors = stereo.get_pointcloud(left_img, right_img)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2.3 libelas ⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/jlowenz/libelas
|
||||
**Stars**: ~300
|
||||
**语言**: C++
|
||||
**许可证**: GPLv3
|
||||
|
||||
#### 项目简介
|
||||
|
||||
ELAS（Efficient Large-Scale Stereo）是一个高效的双目立体匹配库。
|
||||
|
||||
```yaml
|
||||
核心特点:
|
||||
- 速度快
|
||||
- 精度高
|
||||
- 内存效率高
|
||||
- 适合大图像
|
||||
|
||||
优势:
|
||||
- 实时性能
|
||||
- 边缘保持
|
||||
- 鲁棒性好
|
||||
|
||||
适用场景:
|
||||
- 自动驾驶
|
||||
- 机器人
|
||||
- 实时应用
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、深度相机（RGB-D）开源项目
|
||||
|
||||
### 3.1 Azure Kinect SDK ⭐⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/microsoft/Azure-Kinect-Sensor-SDK
|
||||
**Stars**: ~1,500
|
||||
**语言**: C/C++
|
||||
**许可证**: MIT
|
||||
|
||||
#### 项目简介
|
||||
|
||||
Microsoft官方的Azure Kinect开发套件，提供完整的硬件访问接口。
|
||||
|
||||
```yaml
|
||||
核心功能:
|
||||
- RGB相机访问
|
||||
- 深度相机访问
|
||||
- IMU数据读取
|
||||
- 多机同步
|
||||
- 骨骼追踪
|
||||
|
||||
支持平台:
|
||||
- Windows
|
||||
- Linux
|
||||
- ROS
|
||||
|
||||
优势:
|
||||
- 官方支持
|
||||
- 文档完善
|
||||
- 性能优化
|
||||
- 示例丰富
|
||||
```
|
||||
|
||||
#### 使用示例
|
||||
|
||||
```c
|
||||
// C API
|
||||
#include <k4a/k4a.h>
|
||||
|
||||
int main()
|
||||
{
|
||||
// 打开设备
|
||||
k4a_device_t device = NULL;
|
||||
k4a_device_open(0, &device);
|
||||
|
||||
// 配置
|
||||
k4a_device_configuration_t config = K4A_DEVICE_CONFIG_INIT_DISABLE_ALL;
|
||||
config.color_format = K4A_IMAGE_FORMAT_COLOR_BGRA32;
|
||||
config.color_resolution = K4A_COLOR_RESOLUTION_1080P;
|
||||
config.depth_mode = K4A_DEPTH_MODE_NFOV_UNBINNED;
|
||||
config.camera_fps = K4A_FRAMES_PER_SECOND_30;
|
||||
|
||||
// 启动相机
|
||||
k4a_device_start_cameras(device, &config);
|
||||
|
||||
// 采集帧
|
||||
k4a_capture_t capture = NULL;
|
||||
k4a_device_get_capture(device, &capture, K4A_WAIT_INFINITE);
|
||||
|
||||
// 获取图像
|
||||
k4a_image_t color_image = k4a_capture_get_color_image(capture);
|
||||
k4a_image_t depth_image = k4a_capture_get_depth_image(capture);
|
||||
|
||||
// 处理...
|
||||
|
||||
// 释放
|
||||
k4a_image_release(color_image);
|
||||
k4a_image_release(depth_image);
|
||||
k4a_capture_release(capture);
|
||||
k4a_device_stop_cameras(device);
|
||||
k4a_device_close(device);
|
||||
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
#### Python绑定
|
||||
|
||||
```python
|
||||
# pyk4a - Python包装
|
||||
from pyk4a import PyK4A, Config
|
||||
|
||||
# 配置
|
||||
config = Config(
|
||||
color_resolution=PyK4A.ColorResolution.RES_1080P,
|
||||
depth_mode=PyK4A.DepthMode.NFOV_UNBINNED,
|
||||
camera_fps=PyK4A.FPS.FPS_30,
|
||||
synchronized_images_only=True
|
||||
)
|
||||
|
||||
# 启动
|
||||
k4a = PyK4A(config=config)
|
||||
k4a.start()
|
||||
|
||||
# 采集
|
||||
while True:
|
||||
capture = k4a.get_capture()
|
||||
|
||||
if capture.color is not None and capture.depth is not None:
|
||||
rgb = capture.color
|
||||
depth = capture.depth
|
||||
|
||||
# 处理RGB和深度
|
||||
process_frame(rgb, depth)
|
||||
|
||||
k4a.stop()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.2 librealsense ⭐⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/IntelRealSense/librealsense
|
||||
**Stars**: ~7,500
|
||||
**语言**: C++/Python
|
||||
**许可证**: Apache 2.0
|
||||
|
||||
#### 项目简介
|
||||
|
||||
Intel RealSense官方SDK，支持全系列RealSense相机。
|
||||
|
||||
```yaml
|
||||
支持设备:
|
||||
- D400系列（结构光）
|
||||
- D500系列（结构光）
|
||||
- L500系列（LiDAR）
|
||||
- T200系列（追踪）
|
||||
|
||||
核心功能:
|
||||
- 深度流
|
||||
- RGB流
|
||||
- IMU数据
|
||||
- 点云生成
|
||||
- 后处理滤波
|
||||
|
||||
优势:
|
||||
- 跨平台
|
||||
- Python/C++/C#
|
||||
- ROS集成
|
||||
- 实时性能
|
||||
```
|
||||
|
||||
#### Python示例
|
||||
|
||||
```python
|
||||
import pyrealsense2 as rs
|
||||
import numpy as np
|
||||
|
||||
class RealSenseCamera:
|
||||
"""RealSense相机封装"""
|
||||
|
||||
def __init__(self):
|
||||
# 创建pipeline
|
||||
self.pipeline = rs.pipeline()
|
||||
self.config = rs.config()
|
||||
|
||||
# 配置流
|
||||
self.config.enable_stream(
|
||||
rs.stream.depth, 640, 480, rs.format.z16, 30
|
||||
)
|
||||
self.config.enable_stream(
|
||||
rs.stream.color, 1920, 1080, rs.format.bgr8, 30
|
||||
)
|
||||
|
||||
# 启动
|
||||
self.profile = self.pipeline.start(self.config)
|
||||
|
||||
# 获取内参
|
||||
depth_stream = self.profile.get_stream(rs.stream.depth)
|
||||
self.intrinsics = depth_stream.as_video_stream_profile().get_intrinsics()
|
||||
|
||||
# 创建对齐对象
|
||||
self.align = rs.align(rs.stream.color)
|
||||
|
||||
# 后处理滤波器
|
||||
self.decimation = rs.decimation_filter()
|
||||
self.spatial = rs.spatial_filter()
|
||||
self.temporal = rs.temporal_filter()
|
||||
self.hole_filling = rs.hole_filling_filter()
|
||||
|
||||
def get_frames(self):
|
||||
"""获取对齐的RGB-D帧"""
|
||||
# 等待帧
|
||||
frames = self.pipeline.wait_for_frames()
|
||||
|
||||
# 对齐到RGB
|
||||
aligned_frames = self.align.process(frames)
|
||||
|
||||
# 获取帧
|
||||
depth_frame = aligned_frames.get_depth_frame()
|
||||
color_frame = aligned_frames.get_color_frame()
|
||||
|
||||
if not depth_frame or not color_frame:
|
||||
return None, None
|
||||
|
||||
# 深度后处理
|
||||
depth_frame = self.decimation.process(depth_frame)
|
||||
depth_frame = self.spatial.process(depth_frame)
|
||||
depth_frame = self.temporal.process(depth_frame)
|
||||
depth_frame = self.hole_filling.process(depth_frame)
|
||||
|
||||
# 转换为numpy
|
||||
depth_image = np.asanyarray(depth_frame.get_data())
|
||||
color_image = np.asanyarray(color_frame.get_data())
|
||||
|
||||
return color_image, depth_image
|
||||
|
||||
def get_pointcloud(self):
|
||||
"""生成点云"""
|
||||
frames = self.pipeline.wait_for_frames()
|
||||
aligned_frames = self.align.process(frames)
|
||||
|
||||
depth_frame = aligned_frames.get_depth_frame()
|
||||
color_frame = aligned_frames.get_color_frame()
|
||||
|
||||
# 创建点云
|
||||
pc = rs.pointcloud()
|
||||
pc.map_to(color_frame)
|
||||
points = pc.calculate(depth_frame)
|
||||
|
||||
# 导出
|
||||
vertices = np.asanyarray(points.get_vertices())
|
||||
texcoords = np.asanyarray(points.get_texture_coordinates())
|
||||
|
||||
return vertices, texcoords
|
||||
|
||||
def stop(self):
|
||||
self.pipeline.stop()
|
||||
|
||||
# 使用
|
||||
camera = RealSenseCamera()
|
||||
|
||||
while True:
|
||||
rgb, depth = camera.get_frames()
|
||||
if rgb is not None:
|
||||
# 处理
|
||||
pass
|
||||
|
||||
camera.stop()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.3 Open3D ⭐⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/isl-org/Open3D
|
||||
**Stars**: ~11,000
|
||||
**语言**: C++/Python
|
||||
**许可证**: MIT
|
||||
|
||||
#### 项目简介
|
||||
|
||||
Open3D是一个现代化的3D数据处理库，完美支持RGB-D数据。
|
||||
|
||||
```yaml
|
||||
核心功能:
|
||||
- 点云处理
|
||||
- 网格处理
|
||||
- RGB-D集成
|
||||
- SLAM
|
||||
- 可视化
|
||||
|
||||
支持设备:
|
||||
- Azure Kinect
|
||||
- RealSense
|
||||
- 通用RGB-D
|
||||
|
||||
优势:
|
||||
- API简洁
|
||||
- 性能优秀
|
||||
- 文档完善
|
||||
- 可视化强大
|
||||
```
|
||||
|
||||
#### RGB-D SLAM示例
|
||||
|
||||
```python
|
||||
import open3d as o3d
|
||||
import numpy as np
|
||||
|
||||
class RGBD_SLAM:
|
||||
"""基于Open3D的RGB-D SLAM"""
|
||||
|
||||
def __init__(self, intrinsics):
|
||||
self.intrinsics = o3d.camera.PinholeCameraIntrinsic(
|
||||
width=intrinsics['width'],
|
||||
height=intrinsics['height'],
|
||||
fx=intrinsics['fx'],
|
||||
fy=intrinsics['fy'],
|
||||
cx=intrinsics['cx'],
|
||||
cy=intrinsics['cy']
|
||||
)
|
||||
|
||||
self.volume = o3d.pipelines.integration.ScalableTSDFVolume(
|
||||
voxel_length=0.01,
|
||||
sdf_trunc=0.04,
|
||||
color_type=o3d.pipelines.integration.TSDFVolumeColorType.RGB8
|
||||
)
|
||||
|
||||
self.poses = []
|
||||
self.current_pose = np.eye(4)
|
||||
|
||||
def process_frame(self, rgb, depth):
|
||||
"""处理RGB-D帧"""
|
||||
# 创建RGB-D图像
|
||||
rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(
|
||||
o3d.geometry.Image(rgb),
|
||||
o3d.geometry.Image(depth),
|
||||
depth_scale=1000.0,
|
||||
depth_trunc=3.0,
|
||||
convert_rgb_to_intensity=False
|
||||
)
|
||||
|
||||
# 如果是第一帧
|
||||
if len(self.poses) == 0:
|
||||
self.poses.append(self.current_pose)
|
||||
self.volume.integrate(
|
||||
rgbd, self.intrinsics, np.linalg.inv(self.current_pose)
|
||||
)
|
||||
return self.current_pose
|
||||
|
||||
# 里程计估计
|
||||
option = o3d.pipelines.odometry.OdometryOption()
|
||||
odo_init = np.eye(4)
|
||||
|
||||
[success, trans, info] = o3d.pipelines.odometry.compute_rgbd_odometry(
|
||||
rgbd, self.prev_rgbd,
|
||||
self.intrinsics, odo_init,
|
||||
o3d.pipelines.odometry.RGBDOdometryJacobianFromHybridTerm(),
|
||||
option
|
||||
)
|
||||
|
||||
if success:
|
||||
# 更新位姿
|
||||
self.current_pose = self.current_pose @ trans
|
||||
self.poses.append(self.current_pose.copy())
|
||||
|
||||
# 集成到TSDF
|
||||
self.volume.integrate(
|
||||
rgbd, self.intrinsics, np.linalg.inv(self.current_pose)
|
||||
)
|
||||
|
||||
self.prev_rgbd = rgbd
|
||||
return self.current_pose
|
||||
|
||||
def extract_mesh(self):
|
||||
"""提取网格"""
|
||||
mesh = self.volume.extract_triangle_mesh()
|
||||
mesh.compute_vertex_normals()
|
||||
return mesh
|
||||
|
||||
def get_pointcloud(self):
|
||||
"""提取点云"""
|
||||
pcd = self.volume.extract_point_cloud()
|
||||
return pcd
|
||||
|
||||
# 使用
|
||||
slam = RGBD_SLAM({
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
'fx': 1066.778,
|
||||
'fy': 1067.487,
|
||||
'cx': 960.0,
|
||||
'cy': 540.0
|
||||
})
|
||||
|
||||
for rgb, depth in rgbd_stream:
|
||||
pose = slam.process_frame(rgb, depth)
|
||||
print(f"Current pose: {pose}")
|
||||
|
||||
# 提取最终模型
|
||||
mesh = slam.extract_mesh()
|
||||
o3d.io.write_triangle_mesh("output.ply", mesh)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 四、LiDAR开源项目
|
||||
|
||||
### 4.1 FAST-LIO2 ⭐⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/hku-mars/FAST_LIO
|
||||
**Stars**: ~2,500
|
||||
**语言**: C++
|
||||
**许可证**: GPLv2
|
||||
|
||||
#### 项目简介
|
||||
|
||||
FAST-LIO2是最先进的LiDAR-惯性里程计，支持固态和机械式LiDAR。
|
||||
|
||||
```yaml
|
||||
核心特点:
|
||||
- 实时性能
|
||||
- 高精度
|
||||
- 鲁棒性强
|
||||
- 支持多种LiDAR
|
||||
|
||||
支持设备:
|
||||
- Livox系列
|
||||
- Velodyne
|
||||
- Ouster
|
||||
- Hesai
|
||||
|
||||
优势:
|
||||
- 速度快
|
||||
- 精度高
|
||||
- 抗退化
|
||||
- 开源免费
|
||||
```
|
||||
|
||||
#### 使用示例
|
||||
|
||||
```bash
|
||||
# 编译
|
||||
cd FAST_LIO
|
||||
mkdir build && cd build
|
||||
cmake ..
|
||||
make
|
||||
|
||||
# 运行（Livox Mid-360）
|
||||
roslaunch fast_lio mapping_mid360.launch
|
||||
|
||||
# 保存地图
|
||||
rosservice call /map_save "resolution: 0.01
|
||||
destination: '/home/user/map.pcd'"
|
||||
```
|
||||
|
||||
#### 配置文件
|
||||
|
||||
```yaml
|
||||
# config/mid360.yaml
|
||||
common:
|
||||
lid_topic: "/livox/lidar"
|
||||
imu_topic: "/livox/imu"
|
||||
time_sync_en: false
|
||||
|
||||
preprocess:
|
||||
lidar_type: 1 # 1: Livox
|
||||
scan_line: 6
|
||||
blind: 0.5
|
||||
|
||||
mapping:
|
||||
acc_cov: 0.1
|
||||
gyr_cov: 0.1
|
||||
b_acc_cov: 0.0001
|
||||
b_gyr_cov: 0.0001
|
||||
det_range: 100.0
|
||||
|
||||
publish:
|
||||
path_en: true
|
||||
scan_publish_en: true
|
||||
dense_publish_en: true
|
||||
scan_bodyframe_pub_en: true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4.2 LIO-SAM ⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/TixiaoShan/LIO-SAM
|
||||
**Stars**: ~3,000
|
||||
**语言**: C++
|
||||
**许可证**: BSD-3-Clause
|
||||
|
||||
#### 项目简介
|
||||
|
||||
LIO-SAM是一个紧耦合的LiDAR-惯性-视觉SLAM框架。
|
||||
|
||||
```yaml
|
||||
核心特点:
|
||||
- 因子图优化
|
||||
- 回环检测
|
||||
- 全局一致性
|
||||
- 多传感器融合
|
||||
|
||||
支持传感器:
|
||||
- LiDAR
|
||||
- IMU
|
||||
- GPS（可选）
|
||||
- 相机（可选）
|
||||
|
||||
优势:
|
||||
- 精度高
|
||||
- 全局优化
|
||||
- 长时稳定
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4.3 Livox SDK ⭐⭐⭐⭐
|
||||
|
||||
**GitHub**: https://github.com/Livox-SDK/Livox-
|
||||
Reference in New Issue
Block a user