How to extract entities that require going over the max token limit?

[barapa]

I am extracting entities from a webpage. However, I am finding that the amount of tokens I need to generate to extract them all goes over the 4096 limit of the top models. Does anyone have any strategies for dealing with this?

[Mr-Ruben]

https://en.wikipedia.org/wiki/Divide-and-conquer_algorithm

In any case, if you are using so many tokens... probably each output quality is suffering...