Mars_reproduction

这是对论文《MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving》的复现。
官方开源链接为：MARS GitHub 仓库 由于数据量过于庞大的原因，有两个文件夹dataset和outputs并没有上传。 不过在这里，你可以查看四组消融实验的wanb报告，里面有完整可视化的数据。 MARS模型-KITTI报告 MARS模型-KITTI报告

Quantitative Evaluation for Ablation Studies

ID	Settings							KITTI			V-KITTI
	Model	Sampler	Category	$\mathcal{L}_\text{sky}$	$\mathcal{L}_\text{depth}$	$\mathcal{L}_\text{sem}$	$\mathcal{L}_\texttt{accum}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
1*	Grid / Ours	prop / c2f $\dagger$						25.04	0.782	0.175	28.37	0.907	0.108
2	MLP / Ours	c2f / c2f						20.14	0.589	0.476	22.19	0.664	0.409
3	Grid / Ours	prop / c2f					×	21.35	0.713	0.242	27.30	0.881	0.130
4	Grid / Ours	prop / c2f		×		×		23.68	0.774	0.181	27.32	0.881	0.129
4	Grid / Ours	prop / c2f		×		×		28.27	0.896	0.056	27.43	0.880	0.114
5	Grid / Ours	prop / c2f			×			23.66	0.769	0.184	27.30	0.880	0.128
6	Grid / Ours	prop / c2f		×				20.07	0.723	0.251	27.42	0.863	0.148
6	Grid / Ours	prop / c2f		×				18.60	0.589	0.402	20.44	0.631	0.373
7	Grid / MLP	prop / c2f						20.46	0.709	0.255	26.46	0.875	0.132
8	Grid / Grid	prop / prop						22.23	0.741	0.211	25.22	0.871	0.134
9	Grid / MLP	prop / c2f	×					20.98	0.699	0.257	27.27	0.881	0.130
10	Grid / Grid	prop / prop	×					23.71	0.763	0.193	26.65	0.882	0.125
11*	MLP / MLP	c2f / c2f						20.42	0.592	0.472	21.77	0.659	0.410

Notes

• $\dagger$ prop represents proposal sampler, c2f represents coarse-to-fine sampler.
• • ID 1 is our default setting.
• • Red text(the second bold line) represents results from 300k iterations on the KITTI scene 06.
• • Blue text(the third bold line) represents results from 200k iterations on the V-KITTI scene 02.
    Sorry, colors cannot be displayed.

渲染结果

KITTI

KITTI-ID1 的渲染结果:

load-pretrain.mp4

KITTI-ID4 的渲染结果:

kitti-id4.mp4

KITTI-ID6 的渲染结果:

kitti-id6.mp4

VKITTI

The reason for the video repetition is due to the binocular camera.
VKITTI-ID4 的渲染结果:

vkitti-id4.mp4

VKITTI-ID6 的渲染结果:

vkitti-id6.mp4

Setting Up Environment

I use CUDA 11.8 and PyTorch 2.0.1.

Step 1: Create and Activate Conda Environment

First, create a new Conda environment with Python 3.9:

conda create --name mars -y python=3.9
conda activate mars

Step 2: Install Required Packages (this need torch >= 1.13.1)

pip install mars-nerfstudio  

Step 3: Clone and Build tiny-cuda-nn

git clone --recursive https://github.com/nvlabs/tiny-cuda-nn
cd tiny-cuda-nn

Build the project using CMake:

cmake -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.4/bin/nvcc . -B build
cmake --build build --config RelWithDebInfo -j

Install the PyTorch extension:

cd bindings/torch
python setup.py install

References:

• https://blog.csdn.net/weixin_42896263/article/details/131978295
• https://blog.csdn.net/weixin_45818708/article/details/137588790