Performance of PPO is very poor?

[jkwang93]

Hello~ I ran your PPO code, but unexpectedly, it performed very poorly in the optimization task. The current reward of Celecoxib has been around 0.1. Why is this happening?

[JoshuaMeyers]

Hey @jkwang93, thanks for the question. I finally got round to running this code on the celecoxib task, as you say the performance is poor, particularly since the best molecule in the dataset has a similarity of 0.505 to celecoxib versus 0.461538 for the PPO implementation -- see log below.

I'm not sure why this is and it would take more research to improve, however there are more modern implementations to follow now versus when these benchmarks were implemented such as this and many others...

INFO : Benchmarking goal-directed molecule generation, version v2
INFO : Number of benchmarks: 20
INFO : Running benchmark 1/20: Celecoxib rediscovery
INFO : epoch:       0 | current_reward: 0.112 | running_reward: 0.001 | valid_ratio: 0.957 | unique_ratio: 1.000
INFO : epoch:      10 | current_reward: -0.434 | running_reward: -0.017 | valid_ratio: 0.509 | unique_ratio: 0.993
INFO : Results for the benchmark "Celecoxib rediscovery":
INFO :   Score: 0.461538
INFO :   Execution time: 0:39:18
INFO :   Metadata: {'top_1': 0.46153846153846156, 'internal_similarity_max': 0.0, 'internal_similarity_mean': 0.0, 'internal_similarity_histogram_density': ([10.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],), 'internal_similarity_histogram_bins': ([0.0, 0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9, 1.0],)}

[jkwang93]

Thank you very much for your answer~