fix: used initial observation the entire time

[markste-in]

Fixes a bug where the initial observation is used over and over again.

Ensures the network is rendered even if the environment is not solved or aborted

[ntraft]

You could put an if best_genomes: here to handle the case where the user terminates within the first 5 generations (if I read it correctly, best_genomes will be undefined in that case).

[markste-in]

good point. I defined best_genomes = None in the top to make sure the variable exists and can be checked and added an if best_genomes on the bottom

[markste-in]

But I still feel like this example is broken. I can't get it to learn after almost 8000 gens.

I had a simpler example with the Lunar Lander and this package a few years ago that worked quite well. Tho I can't find that code rn. Maybe I find some time to troubleshoot it at some point.

@ntraft Can you get it to work?

[ntraft]

Oh, really?? As in, you were able to "solve" the Lunar Lander, or at least do well? I assumed that it never worked, when I started using this package ~9 months ago!

Did you see my issue #256 about the Lander example? Did you address the issue with the fitness function? I wonder what kind of fitness you were using when you had it working before?

I started using the Bipedal Walker instead, and it works kinda okay on flat terrain, but nowhere near as well as anything you see in these Evolution Strategies examples. But even to get to that point I had to fix the FeedForwardNetwork so that it will run nodes even if they don't have an input. I should probably also implement something to prevent the dangling nodes from appearing in the first place, like you did. I was trying to avoid that, at first, because I wasn't sure if it was intentional and wasn't sure about the best way to fix it. I put this project away for awhile but it's seeming like if I go much further with it I would have to address that issue.

It would be great to see it solve Lunar Lander! I just assumed it was too hard. I'm also curious, do you remember how important the speciation parameters were for your success? Tuning the speciation is seeming like a hot mess to me.

[markste-in]

No I wasn't able to solve it with the current code example even after thousands of generations. I think the fitness function is wrong or at least buggy but I haven't had enough time to debug it yet. I also think the fitness function is overly complicated. I will check my old hard dist because I am pretty sure that my own code from like a year a ago solved it kinda quickly.
If i find the code I will add it to this Pull Request.
The lunar lander is actually quite simple and should be solved quickly. Removing the dangling nodes doesn't make a big difference in this case since the network should never grow that big. In more complicated environments it helps to make the learning more "stable" in my experience

[ntraft]

Note some other issues that are suspicious and may be part of the problem:

• Incorrect behaviour of Elitism #272
• neat.Population.run() does not return the best genome #271

I think elitism isn't working correctly, and best genomes might not be returned correctly, and this might also be negatively impacting speciation.

I'm going to try to investigate some of it this week.

[allinduetime]

These ever get solved?

[Finebouche]

@allinduetime check https://github.com/Finebouche/neat-gymnasium. We could think about merging the project here in the examples actually. But it look like @CodeReclaimers is not maintaining it ATM