Questions on ICQ_softmax

[qizhg]

Dear author, thanks for making the code available.

I have two questions regarding ICQ_softmax, where the weight is approximated by the softmax over the minibatch:

1. Why len(weights), e.g., this line, is needed to scale the softmax distribution?
2. Why the softmax is performed wrt the TD error in this line, instead of wrt the Q value suggested in the paper?

Thanks!

[YiqinYang]

Dear qizhg,

Thanks for your comments.

1. The softmax approximation is an approximation method since it is easy to implement. You can also accurately calculate the re-weighting ratio. Please refer to the ICQ_mu.

2. We find the TD error will have a more stable performance. You can also adopt the Q-value, which will have a sound performance. Both two approaches have the same insights.

Sorry for the late reply, and Thanks again for your question!

Best

Yiqin