Reproduce HiP-AD

[yyongjae]

Thank you for your contribution with your research on HiP-AD.

I am currently in the process of reproducing your model, but I have encountered difficulties in matching the performance reported in the paper. I would like to ask a few questions regarding the evaluation and training setups.

• when evaluating the model using the official checkpoint provided in your GitHub repository, the Avg L2 (2s) value was 1.0xx, which is higher than the 0.69 reported in the paper.

• when I conducted the training from scratch following the environment described in the paper, the result was significantly worse, yielding an Avg L2 of 1.14 (even it trained 18 epochs). During this process, I noticed several discrepancies between the paper's description and the provided codebase:

Batch Size: The paper specifies a total batch size of 32 using 8 NVIDIA 4090 GPUs, but the default configuration in the code is set to 64.

Weight Decay: The paper mentions a weight decay of 0.01, whereas the code uses 0.001.

Since my training results using the "paper settings" were suboptimal, could you please clarify which hyperparameters are the correct ones for achieving the reported performance? Additionally, I would appreciate any further insights or specific training recipes required to successfully reproduce the results

Thank you

[xiongqi123123]

Thank you for your contribution with your research on HiP-AD.

I am currently in the process of reproducing your model, but I have encountered difficulties in matching the performance reported in the paper. I would like to ask a few questions regarding the evaluation and training setups.

• when evaluating the model using the official checkpoint provided in your GitHub repository, the Avg L2 (2s) value was 1.0xx, which is higher than the 0.69 reported in the paper.
• when I conducted the training from scratch following the environment described in the paper, the result was significantly worse, yielding an Avg L2 of 1.14 (even it trained 18 epochs). During this process, I noticed several discrepancies between the paper's description and the provided codebase:

Batch Size: The paper specifies a total batch size of 32 using 8 NVIDIA 4090 GPUs, but the default configuration in the code is set to 64.

Weight Decay: The paper mentions a weight decay of 0.01, whereas the code uses 0.001.

Since my training results using the "paper settings" were suboptimal, could you please clarify which hyperparameters are the correct ones for achieving the reported performance? Additionally, I would appreciate any further insights or specific training recipes required to successfully reproduce the results

Thank you

Hi, I am also reproducing HiP-AD and encountered the same issue.

Using the official released checkpoint, I get an Avg L2 (2s) of around 1.0x, and when I train it myself, I get around 1.1x, both of which are much higher than the 0.69 reported in the paper.

Were you able to resolve this issue in the end? If so, could you share what setting or detail made the difference?

Thank you.

[yyongjae]

@xiongqi123123
No..
Despite using the identical evaluation code provided in the Bench2Drive repository, I have been struggling with reproducing the results.
But I reproduced closed-loop driving score around 86.xx.

[xiongqi123123]

@yyongjae
So…basically open-loop is hard to reproduce, but closed-loop is pretty close? I’ll try the closed-loop myself in the next couple of days and we can compare results then.

[Yzichen]

@xiongqi123123 可以加个微信一起交流一下吗？ 感觉这套代码好多问题

[xiongqi123123]

@Yzichen 我通过你主页加你吧

[yyongjae]

@xiongqi123123 @Yzichen 如果您已经解决了问题或取得了任何进展，请分享一下。 ;-;