Skip to content

Commit

Permalink
feat: Add "Speeding up migrations" section to "State persistence" (#1365
Browse files Browse the repository at this point in the history
)

We've recently added support to speed up Actor migrations by rebooting
the run when it's migrating. This adds docs about that, with code
examples.

The support for this in the Apify SDK is not released yet, it'll go out
in the next SDK release, I'll only merge the docs then.

---------

Co-authored-by: Michał Olender <[email protected]>
  • Loading branch information
fnesveda and TC-MO authored Feb 7, 2025
1 parent 20d6a1e commit 982f4b6
Showing 1 changed file with 51 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ By default, an Actor keeps its state in the server's memory. During a server swi

The [Apify SDKs](/sdk) handle state persistence automatically.

This is done using the `Actor.on()` method and the `migrating` event.
This is done using the `Actor.on()` method and the `migrating` event.

- The `migrating` event is triggered just before a migration occurs, allowing you to save your state.
- To retrieve previously saved state, you can use the [`Actor.getValue`](/sdk/js/reference/class/Actor#getValue)/[`Actor.get_value`](/sdk/python/reference/class/Actor#get_value) methods.
Expand Down Expand Up @@ -81,15 +81,15 @@ await Actor.exit();
<TabItem value="Python" label="Python">

```python
from apify import Actor
from apify import Actor, Event

async def actor_migrate():
async def actor_migrate(_event_data):
await Actor.set_value('my-crawling-state', {'foo': 'bar'})

async def main():
async with Actor:
# ...
Actor.on('migrating', actor_migrate)
Actor.on(Event.MIGRATING, actor_migrate)
# ...
```

Expand Down Expand Up @@ -128,3 +128,50 @@ async def main():
</Tabs>

For improved Actor performance consider [caching repeated page data](/academy/expert-scraping-with-apify/saving-useful-stats).

## Speeding up migrations

Once your Actor receives the `migrating` event, the Apify platform will shut it down and restart it on a new server within one minute.
To speed this process up, once you have persisted the Actor state,
you can manually reboot the Actor in the `migrating` event handler using the `Actor.reboot()` method
available in the [Apify SDK for JavaScript](/sdk/js/reference/class/Actor#reboot) or [Apify SDK for Python](/sdk/python/reference/class/Actor#reboot).

<Tabs groupId="main">
<TabItem value="JavaScript" label="JavaScript">

```js
import { Actor } from 'apify';

await Actor.init();
// ...
Actor.on('migrating', async () => {
// ...
// save state
// ...
await Actor.reboot();
});
// ...
await Actor.exit();
```

</TabItem>
<TabItem value="Python" label="Python">

```python
from apify import Actor, Event

async def actor_migrate(_event_data):
# ...
# save state
# ...
await Actor.reboot()

async def main():
async with Actor:
# ...
Actor.on(Event.MIGRATING, actor_migrate)
# ...
```

</TabItem>
</Tabs>

0 comments on commit 982f4b6

Please sign in to comment.