Possible hero.detach Issue

[morgano86]

This query works on all websites:

    const nodeList = await hero.querySelectorAll(
      'button[type="submit"]:not(:empty)'
    );

Trying to detach the same query results in an infinite loop (on some but not all websites):

    const detachedNodeList = await hero.detach(
      hero.querySelectorAll('button[type="submit"]:not(:empty)')
    );

The log shows something like this over and over:

2024-11-22T22:23:39.533Z INFO [unblocked-agent/lib/Resources] MitmRequest {
url: 'https://assets.msn.com/staticsb/statics//latest/community/img/avatar-placeholder.svg',
method: 'GET',
id: 1577,
context: { sessionId: 'F9l4TvWwclZaoVGJbjXHW' }
}
2024-11-22T22:23:39.534Z INFO [unblocked-agent-mitm/handlers/BaseHttpHandler] Http.RequestBlocked {
url: 'https://assets.msn.com/staticsb/statics//latest/community/img/avatar-placeholder.svg',
context: {}
}

I assume Hero is waiting for stability before it detaches - is it possible to force a detachment in the current state at the time of the request as it seems this website is doing something in the background that effectively stops Hero from detaching anything?

Of course, it's not ideal, but the button elements I'm trying to detach are unaffected by whatever is happening in the log and we should be able to detach them.

On a side note, working with elements in Hero without detaching them is incredibly slow. I moved my app from Playwright to Hero and after modifying the implementation to use Hero for analysing dom elements the processing time has increased from 3s -> 12s for smaller websites and 6s -> 20-30s with more complex setups.

For example, finding buttons and clicking them took around 500ms in Playwright and takes anywhere from 3s in Hero.
Finding custom elements which have shadowRoot took a second or so using playwright.evaluate (not good for detection) and around 3s by analysing elements directly, whereas Hero takes anywhere from 6s to 20s to go through the elements and process them.

Not a dig at Hero, because it is far superior overall!

[blakebyrnes]

Regarding the detaching, it should be doing it at the moment you request it. If you can figure out a test script not involving the end site that fails, happy to try to fix it.

For the interaction, are you comparing the default human emulator to playwright just using dom apis? If you need the human emulator to move faster, there are a few ways you can turn down the "emulation" so that it runs faster (You would set the static human emulator variables in your Hero Core process. A unit test configuring some of these options is

hero/plugins/default-human-emulator/test/interact.test.ts

Line 24 in d249770

HumanEmulator.minScrollVectorPoints = 2;

).

Fwiw, finding elements should be really fast unless you are recursing properties. Each "awaited dom" step does have to go across from Hero <-> Hero Core <-> Chrome and back, so can be slow if you have lots of individual lookups. But if you're just doing one, it will just add the roundtrip once.