Regrading Pedestrian and Cyclist detection

[Anirbanbhk88]

Hi @pangsu0613 Thanks for sharing your work. I am performing my task in sensor fusion based on the CLOCs paper. So I am trying to implement it. I am able to get the same results that you have posted for while Cars. I am now trying to perform the detections for cyclists and pedestrians. In the cyclist/pedestrian detection section you have mentioned:

 **Step1:Prepare the config files** 
Under the ./configs directory, edit the file pedestrian.fhd.config or cyclist.fhd.config:

- Change the value of detection_2d_path to your pedestrian 2D detections.
- Set steps and steps_per_eval to your desired value (tweak based on training set size, for example, if we have 3712 training examples, setting “steps” equal to 74240 (3712 x 20) means training for 20 epochs. “steps_per_eval” means the step interval for each evaluation, for example, if you set “steps_per_eval” to 3712, it means doing evaluation for each epoch.)

For the step Change the value of detection_2d_path to your pedestrian 2D detections. I could not find any pedestrian 2D detections in the drive link
Do we need to train the Cascade-RCNN separately and generate the 2D detections for pedestrian and cyclist. Or else could you share them?

[Joeless]

@Anirbanbhk88 hi~
Can you share your config about car ?
I modify the car config using README.md
But I get a lower AP than this paper

[Anirbanbhk88]

@Joeless I have attached the config car file in zip.
car.fhd.zip

[Anirbanbhk88]

@pangsu0613 For Pedestrian detection I have downloaded 3 more files from the drive link: second_pedestrian_model.zip ,clocs_pedestrian_trained.zip and mscnn_ped_cyc_trainval_sigmoid_data_scale_1000.zip. If I am not wrong I assume mscnn_ped_cyc_trainval_sigmoid_data_scale_1000.zip contains the 2D detections for pedestrian and cycles. The second_pedestrian_model.zip contains the SECOND detector pretrained for pedestrian and clocs_pedestrian_trained.zip contains the pretrained fusion model for pedestrian...I have extracted all the files and put in the respective paths as instructed in the Readme. When I run my code for evaluating the pedestrian model, the SECOND pretrained model(voxelnet-100224.tckpt) is getting loaded. But there is a an error loading the pretrained Fusion model(fusion_layer-37120.tckpt). The error is in the line

CLOCs-master/torchplus/train/checkpoint.py, line 118, in restore
model.load_state_dict(torch.load(ckpt_path))

_pickle.UnpicklingError: A load persistent id instruction was encountered,
but no persistent_load function was specified.

I guess the pytorch is not able to load the pickle file for Fusion model. Whereas it can load the pickle file for SECOND model without erros. Dont know the reason why...Do anyone face this issue

Also another strange thing I noticed the checkpoint.json file for the Fusion model is pointing to the checkpoint "fusion_layer": "fusion_layer-74240.tckpt"

Whereas the pretrained fusion model in clocs_pedestrian_trained.zip has another checkpoint fusion_layer-37120.tckpt.

So I updated the checkpoint.json file to point to the file fusion_layer-37120.tckpt. But still torch could not load it.

I am using Pytorch 1.0

[Joeless]

@Joeless I have attached the config car file in zip. car.fhd.zip

Thanks for your config
I checked the config and found that we are the same
Are you get the right AP like this paper?
my result :
Car [email protected], 0.70, 0.70:
bbox AP:99.33, 91.36, 88.33
bev AP:96.16, 87.57, 84.28
3d AP:89.89, 77.75, 72.10
aos AP:99.18, 90.86, 87.39

paper result:
Car: [email protected] [email protected] [email protected]
bbox: AP: 99.33 / 97.87, 93.75 / 92.37, 92.89 / 89.87
bev: AP: 96.51 / 95.61, 92.37 / 89.54, 89.41 / 86.96
3d: AP: 92.74 / 90.97, 82.90 / 79.94, 77.75 / 77.09

[Rtakaha]

@Anirbanbhk88

I guess the pytorch is not able to load the pickle file for Fusion model. Whereas it can load the pickle file for SECOND model without erros. Dont know the reason why...Do anyone face this issue

I faced the same issue and figured out how to fix it. It seems that the checkpoint fusion_layer-37120.tckpt is saved with torch>=1.6 which saves checkpoints with zip format as a default. See the answer of stackoverflow below.
https://stackoverflow.com/a/69171444

So I loaded fusion_layer-37120.tckpt with torch==1.6.0, and saved with _use_new_zipfile_serialization=False.

import torch # make sure it's >=1.6

checkpoint = torch.load('/path/to/fusion_layer-37120.tckpt')
torch.save(checkpoint, '/SAVEDIR/new_fusion_layer-37120.tckpt', _use_new_zipfile_serialization=False)

Finally I was able to load newer version of the checkpoint with train.py.

The result is slightly different from what the author shows in README, but I guess it's close enough.

Pedestrian [email protected], 0.50, 0.50:
bbox AP:74.45, 70.72, 62.43
bev  AP:68.28, 62.94, 56.54
3d   AP:62.88, 56.20, 50.10
aos  AP:38.57, 37.14, 33.05
Pedestrian [email protected], 0.25, 0.25:
bbox AP:74.45, 70.72, 62.43
bev  AP:90.55, 87.94, 78.45
3d   AP:90.45, 85.93, 76.42
aos  AP:38.57, 37.14, 33.05

Hope this helps!