Skip to content

Commit

Permalink
Add guides for redis hosting and cart storage in redis
Browse files Browse the repository at this point in the history
  • Loading branch information
keulinho committed Sep 17, 2024
1 parent a9179ae commit bc19a89
Show file tree
Hide file tree
Showing 8 changed files with 131 additions and 9 deletions.
3 changes: 3 additions & 0 deletions guides/hosting/infrastructure/database-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ shopware:
It is recommended to use a persistent Redis connection to avoid connection issues in high-load scenarios. There is also a `cart:migrate` command to migrate the existing carts between MySQL and Redis, so the migration does not influence end-user experience.

For a detailed explanation refer to the cart storage docs:
<PageRef page="../performance/cart-storage" />

Check warning on line 49 in guides/hosting/infrastructure/database-cluster.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/database-cluster.md#L49

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/database-cluster.md:49:14: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

Check warning on line 49 in guides/hosting/infrastructure/database-cluster.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/database-cluster.md#L49

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/database-cluster.md:49:42: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

## Configure the database cluster

To use the MySQL cluster, you have to configure the following in the `.env` file:
Expand Down
2 changes: 1 addition & 1 deletion guides/hosting/infrastructure/elasticsearch/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
nav:
title: Elasticsearch
position: 10
position: 5

---

Expand Down
56 changes: 56 additions & 0 deletions guides/hosting/infrastructure/redis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
nav:
title: Redis
position: 7

---

# Redis

[Redis](https://redis.io/docs/latest/get-started/) is an in-memory data storage, that offers high performance and can be used as a cache, message broker, and database. It is a key-value store that supports various data structures like strings, hashes, lists, sets, and sorted sets.
Especially in high-performance and high-throughput scenarios it can give better results, than relying on a traditional relational database.
Therefore, multiple adapter exist in shopware, to offload some tasks from the DB to Redis.

However, as the data that is stored in Redis differs and also the access patterns to this data differ, it makes sense to use different Redis instances with different configurations for different tasks.

The data stored in Redis can be roughly classified into those three categories:
1. Ephemeral data: This data is not critical and can be easily recreated when lost, e.g. caches.
2. Durable, but "aging" data: This data is important and can not easily be recreated, but the relevance of the data decreases over time, e.g. sessions.
3. Durable and critical data: This data is important and can not easily be recreated, e.g. carts, number ranges.

## Ephemeral data

As ephemeral data can easily be restored and is most often used in cases where high performance matters, this data can be stored with no durable persistence.
This means the data is only stored in memory and is lost when the Redis instance is restarted.

For key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as the application explicitly manages the TTL for each cache item.

The caching data (HTTP-Cache & Object cache) is what should be stored in this instance.

<PageRef page="../performance/caches" />

Check warning on line 30 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L30

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:30:14: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

Check warning on line 30 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L30

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:30:36: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

## Durable, but "aging" data

As the data stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details.

`allkeys-lru` should be used as key eviction policy here, as by default more recent data is more important than older data, therefore the oldest values should be discarded, when Redis reach the max memory.

The session data is what should be stored in this instance.

<PageRef page="../performance/session" />

Check warning on line 40 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L40

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:40:14: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

Check warning on line 40 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L40

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:40:37: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

## Durable and critical data

Again this is durable data, that can not easily be recreated, therefore it should be persisted as well.

As the data is critical it is important to use a key eviction policy that will not delete data that is not expired, therefore `volatile-lru` should be used.

The cart, number range, lock store and increment data is what should be stored in this instance.

<PageRef page="../performance/cart-storage" />

Check warning on line 50 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L50

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:50:14: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

Check warning on line 50 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L50

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:50:42: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

<PageRef page="../performance/number-ranges" />

Check warning on line 52 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L52

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:52:14: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

Check warning on line 52 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L52

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:52:43: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

<PageRef page="../performance/lock-store" />

Check warning on line 54 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L54

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:54:14: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

Check warning on line 54 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L54

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:54:40: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

<PageRef page="../performance/increment" />

Check warning on line 56 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L56

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:56:14: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION

Check warning on line 56 in guides/hosting/infrastructure/redis.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/infrastructure/redis.md#L56

Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/infrastructure/redis.md:56:39: Unpaired symbol: ‘"’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION
5 changes: 5 additions & 0 deletions guides/hosting/performance/caches.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,8 @@ framework:
default_redis_provider: 'redis://host:port'
```

### Redis configuration

As the cached information is ephemeral and can be recreated, it is not necessary to configure Redis to store the data on disk. For maximum performance you can configure Redis to use no persistence, refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details.

As key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as the application explicitly manages the TTL for each cache item. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/).
48 changes: 48 additions & 0 deletions guides/hosting/performance/cart-storage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
nav:
title: Cart Storage
position: 65

---

# Cart Storage

By default, shopware stores the cart in the database. This can be a performance bottleneck in scenarios where high throughput is required (e.g., thousands of orders per minute), especially if a DB cluster with a read/write-split is used.
Additionally, as the content in that table can change quite quickly, it can lead to an explosion of the databases binlog file.

Redis is better suited in high-throughput scenarios, therefore you should use Redis as storage for the cart in such scenarios.

## Using Redis as storage

To use Redis, create a `config/packages/shopware.yml` file with the following content:

```yaml
shopware:
cart:
redis_url: 'redis://host:port/dbindex?persistent=1'

Check warning on line 22 in guides/hosting/performance/cart-storage.md

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] guides/hosting/performance/cart-storage.md#L22

Unpaired symbol: ‘'’ seems to be missing (EN_UNPAIRED_QUOTES) URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US Category: PUNCTUATION
Raw output
guides/hosting/performance/cart-storage.md:22:15: Unpaired symbol: ‘'’ seems to be missing (EN_UNPAIRED_QUOTES)
 URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses 
 Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_QUOTES?lang=en-US
 Category: PUNCTUATION
```
It is recommended to use a persistent Redis connection to avoid connection issues in high-load scenarios.
## Migrating between storages
You can migrate the current carts from the DB to Redis by running the following CLI command:
```shell
bin/console cart:migrate {fromStorage} {redisUrl?}
```

::: info
Providing the redis URL is optional. If not provided, the value from the configuration will be used. If it is not configured in the yaml file, you need to provide the URL.
:::

For example, if you want to migrate from the default `SQL` storage to the high-performing `Redis` storage, the command is:

```shell
bin/console cart:migrate sql
```

## Redis configuration

As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details.

As key eviction policy you should use `volatile-lru`, which only automatically deletes carts that are expired, as otherwise you might risk of losing data. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/).
10 changes: 6 additions & 4 deletions guides/hosting/performance/increment.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,6 @@ nav:

# Increment Storage

::: info
This feature has been introduced with Shopware version 6.4.7.0
:::

The increment storage is used to store status and display it in the Administration. This can include

* Status of the message queue
Expand Down Expand Up @@ -38,6 +34,12 @@ shopware:
url: 'redis://host:port/dbindex'
```
### Redis configuration
As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details.
As key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as otherwise you might risk of losing data. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/).

## Disabling the increment storage

The usage of the increment storage is optional and can be disabled. When this feature is disabled, Queue Notification and Module Usage Overview will not work in the Administration.
Expand Down
10 changes: 6 additions & 4 deletions guides/hosting/performance/number-ranges.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,6 @@ nav:

# Number Ranges

::: info
The Redis storage and the configuration options for this have been introduced with Shopware version 6.4.11.0.
:::

Number Ranges provide a consistent way to generate a consecutive number sequence that is used for order numbers, invoice numbers, etc.
The generation of the number ranges is an **atomic** operation. This guarantees that the sequence is consecutive and that no number is generated twice.

Expand All @@ -29,6 +25,12 @@ shopware:
redis_url: 'redis://host:port/dbindex'
```
### Redis configuration
As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details.
As key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as otherwise you might risk of losing data. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/).

## Migrating between storages

You can migrate the current state of the number ranges from your current storage to a new one by running the following CLI command:
Expand Down
6 changes: 6 additions & 0 deletions guides/hosting/performance/session.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ framework:
handler_id: "redis://host:port"
```
### Redis configuration
As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details.
As key eviction policy you should use `allkeys-lru`, which only automatically deletes the last recently used entries when Redis reaches max memory consumption. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/).

### Other adapters

Symfony also provides PHP implementations of some adapters:
Expand Down

0 comments on commit bc19a89

Please sign in to comment.