should update the weight every time step ?

[fi000]

should update the weight every time step ? (I think it is better to update the weight every for instance 10 steps in time step T/10==0 then saveweight) but in code it is updated for every 10 steps of episodes?

[pskrunner14]

@fi000 actually since we're randomly sampling a batch from memory at each of these time steps, it would essentially only decrease the number of iterations/ batch updates. I'm not very clear on whether you're referring to weight updates or weight saves.

[fi000]

Thank @pskrunner14,
1- I has applied this code to my problem and I saw that loading the weight is not useful at all cases and leads to divergence! What we can say about this?
2- Also, what is normally iteration steps to save the weights?As the low amount is not useful and higher amount leads to few saving

[WorksWellWithOthers]

1. Loading just the weights of the model then sampling from a new set of experiences may be leading to divergence.
2. How about saving after each episode? How about saving when there is an improvement?