walker_stand critic loss explosion

[seolhokim]

Hi, thank you for quality code. but I wonder why walker_stand task critic loss is too high(up to 1e+3) in my experiment. In my case, I used your conda.yaml and changed env :walker_stand and action_repeat : 2 and batch_size : 512 as you mentioned in paper. how can I get stable critic loss?(for example, reward scaling)

Thank you for reading.

[godnpeter]

I've recently tried to reproduce the original paper in my own way (mostly based on this repository), but I am asl experiencing this issue where some random seeds just show really bad critic loss (mostly explosion) for various tasks such as walker stand and finger spin and also different environments. Is there some trick within the code which prevents this? To me, the code seems super intuitive but I am just experiencing this issue over and over again :(