From bc19a89cc2239e02b58b351a75eee9664d55c0b0 Mon Sep 17 00:00:00 2001 From: Jonas Elfering Date: Tue, 17 Sep 2024 13:29:04 +0200 Subject: [PATCH] Add guides for redis hosting and cart storage in redis --- .../infrastructure/database-cluster.md | 3 + .../infrastructure/elasticsearch/index.md | 2 +- guides/hosting/infrastructure/redis.md | 56 +++++++++++++++++++ guides/hosting/performance/caches.md | 5 ++ guides/hosting/performance/cart-storage.md | 48 ++++++++++++++++ guides/hosting/performance/increment.md | 10 ++-- guides/hosting/performance/number-ranges.md | 10 ++-- guides/hosting/performance/session.md | 6 ++ 8 files changed, 131 insertions(+), 9 deletions(-) create mode 100644 guides/hosting/infrastructure/redis.md create mode 100644 guides/hosting/performance/cart-storage.md diff --git a/guides/hosting/infrastructure/database-cluster.md b/guides/hosting/infrastructure/database-cluster.md index 2841cce68..10aa2b54b 100644 --- a/guides/hosting/infrastructure/database-cluster.md +++ b/guides/hosting/infrastructure/database-cluster.md @@ -45,6 +45,9 @@ shopware: It is recommended to use a persistent Redis connection to avoid connection issues in high-load scenarios. There is also a `cart:migrate` command to migrate the existing carts between MySQL and Redis, so the migration does not influence end-user experience. +For a detailed explanation refer to the cart storage docs: + + ## Configure the database cluster To use the MySQL cluster, you have to configure the following in the `.env` file: diff --git a/guides/hosting/infrastructure/elasticsearch/index.md b/guides/hosting/infrastructure/elasticsearch/index.md index 75ccd4909..0170b3b1d 100644 --- a/guides/hosting/infrastructure/elasticsearch/index.md +++ b/guides/hosting/infrastructure/elasticsearch/index.md @@ -1,7 +1,7 @@ --- nav: title: Elasticsearch - position: 10 + position: 5 --- diff --git a/guides/hosting/infrastructure/redis.md b/guides/hosting/infrastructure/redis.md new file mode 100644 index 000000000..39905433e --- /dev/null +++ b/guides/hosting/infrastructure/redis.md @@ -0,0 +1,56 @@ +--- +nav: + title: Redis + position: 7 + +--- + +# Redis + +[Redis](https://redis.io/docs/latest/get-started/) is an in-memory data storage, that offers high performance and can be used as a cache, message broker, and database. It is a key-value store that supports various data structures like strings, hashes, lists, sets, and sorted sets. +Especially in high-performance and high-throughput scenarios it can give better results, than relying on a traditional relational database. +Therefore, multiple adapter exist in shopware, to offload some tasks from the DB to Redis. + +However, as the data that is stored in Redis differs and also the access patterns to this data differ, it makes sense to use different Redis instances with different configurations for different tasks. + +The data stored in Redis can be roughly classified into those three categories: +1. Ephemeral data: This data is not critical and can be easily recreated when lost, e.g. caches. +2. Durable, but "aging" data: This data is important and can not easily be recreated, but the relevance of the data decreases over time, e.g. sessions. +3. Durable and critical data: This data is important and can not easily be recreated, e.g. carts, number ranges. + +## Ephemeral data + +As ephemeral data can easily be restored and is most often used in cases where high performance matters, this data can be stored with no durable persistence. +This means the data is only stored in memory and is lost when the Redis instance is restarted. + +For key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as the application explicitly manages the TTL for each cache item. + +The caching data (HTTP-Cache & Object cache) is what should be stored in this instance. + + + +## Durable, but "aging" data + +As the data stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details. + +`allkeys-lru` should be used as key eviction policy here, as by default more recent data is more important than older data, therefore the oldest values should be discarded, when Redis reach the max memory. + +The session data is what should be stored in this instance. + + + +## Durable and critical data + +Again this is durable data, that can not easily be recreated, therefore it should be persisted as well. + +As the data is critical it is important to use a key eviction policy that will not delete data that is not expired, therefore `volatile-lru` should be used. + +The cart, number range, lock store and increment data is what should be stored in this instance. + + + + + + + + diff --git a/guides/hosting/performance/caches.md b/guides/hosting/performance/caches.md index e38034fa0..ec989a540 100644 --- a/guides/hosting/performance/caches.md +++ b/guides/hosting/performance/caches.md @@ -94,3 +94,8 @@ framework: default_redis_provider: 'redis://host:port' ``` +### Redis configuration + +As the cached information is ephemeral and can be recreated, it is not necessary to configure Redis to store the data on disk. For maximum performance you can configure Redis to use no persistence, refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details. + +As key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as the application explicitly manages the TTL for each cache item. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/). \ No newline at end of file diff --git a/guides/hosting/performance/cart-storage.md b/guides/hosting/performance/cart-storage.md new file mode 100644 index 000000000..b08fe2b40 --- /dev/null +++ b/guides/hosting/performance/cart-storage.md @@ -0,0 +1,48 @@ +--- +nav: + title: Cart Storage + position: 65 + +--- + +# Cart Storage + +By default, shopware stores the cart in the database. This can be a performance bottleneck in scenarios where high throughput is required (e.g., thousands of orders per minute), especially if a DB cluster with a read/write-split is used. +Additionally, as the content in that table can change quite quickly, it can lead to an explosion of the databases binlog file. + +Redis is better suited in high-throughput scenarios, therefore you should use Redis as storage for the cart in such scenarios. + +## Using Redis as storage + +To use Redis, create a `config/packages/shopware.yml` file with the following content: + +```yaml +shopware: + cart: + redis_url: 'redis://host:port/dbindex?persistent=1' +``` +It is recommended to use a persistent Redis connection to avoid connection issues in high-load scenarios. + +## Migrating between storages + +You can migrate the current carts from the DB to Redis by running the following CLI command: + +```shell +bin/console cart:migrate {fromStorage} {redisUrl?} +``` + +::: info +Providing the redis URL is optional. If not provided, the value from the configuration will be used. If it is not configured in the yaml file, you need to provide the URL. +::: + +For example, if you want to migrate from the default `SQL` storage to the high-performing `Redis` storage, the command is: + +```shell +bin/console cart:migrate sql +``` + +## Redis configuration + +As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details. + +As key eviction policy you should use `volatile-lru`, which only automatically deletes carts that are expired, as otherwise you might risk of losing data. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/). \ No newline at end of file diff --git a/guides/hosting/performance/increment.md b/guides/hosting/performance/increment.md index 531fbca91..9ba8bd484 100644 --- a/guides/hosting/performance/increment.md +++ b/guides/hosting/performance/increment.md @@ -7,10 +7,6 @@ nav: # Increment Storage -::: info -This feature has been introduced with Shopware version 6.4.7.0 -::: - The increment storage is used to store status and display it in the Administration. This can include * Status of the message queue @@ -38,6 +34,12 @@ shopware: url: 'redis://host:port/dbindex' ``` +### Redis configuration + +As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details. + +As key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as otherwise you might risk of losing data. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/). + ## Disabling the increment storage The usage of the increment storage is optional and can be disabled. When this feature is disabled, Queue Notification and Module Usage Overview will not work in the Administration. diff --git a/guides/hosting/performance/number-ranges.md b/guides/hosting/performance/number-ranges.md index 4c75cfdfd..a4a9e8608 100644 --- a/guides/hosting/performance/number-ranges.md +++ b/guides/hosting/performance/number-ranges.md @@ -7,10 +7,6 @@ nav: # Number Ranges -::: info -The Redis storage and the configuration options for this have been introduced with Shopware version 6.4.11.0. -::: - Number Ranges provide a consistent way to generate a consecutive number sequence that is used for order numbers, invoice numbers, etc. The generation of the number ranges is an **atomic** operation. This guarantees that the sequence is consecutive and that no number is generated twice. @@ -29,6 +25,12 @@ shopware: redis_url: 'redis://host:port/dbindex' ``` +### Redis configuration + +As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details. + +As key eviction policy you should use `volatile-lru`, which only automatically deletes data that is expired, as otherwise you might risk of losing data. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/). + ## Migrating between storages You can migrate the current state of the number ranges from your current storage to a new one by running the following CLI command: diff --git a/guides/hosting/performance/session.md b/guides/hosting/performance/session.md index e81808dce..8b4001014 100644 --- a/guides/hosting/performance/session.md +++ b/guides/hosting/performance/session.md @@ -33,6 +33,12 @@ framework: handler_id: "redis://host:port" ``` +### Redis configuration + +As the information stored here is durable and should be persistent, even in the case of a Redis restart, it is recommended to configure the used Redis instance that it will not just keep the data in memory, but also store it on the disk. This can be done by using snapshots (RDB) and Append Only Files (AOF), refer to the [Redis docs](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) for details. + +As key eviction policy you should use `allkeys-lru`, which only automatically deletes the last recently used entries when Redis reaches max memory consumption. For a detailed overview of Redis key eviction policies see the [Redis docs](https://redis.io/docs/latest/develop/reference/eviction/). + ### Other adapters Symfony also provides PHP implementations of some adapters: