Skip to content

Latest commit

 

History

History
165 lines (127 loc) · 9.95 KB

PRODUCT_SYNC.md

File metadata and controls

165 lines (127 loc) · 9.95 KB

commercetools product sync

Utility which provides API for building CTP product update actions and product synchronisation.

Usage

Sync list of product drafts

Prerequisites

  1. The sync expects a list of non-null ProductDraft objects that have their key fields set to match the products from the source to the target. Also the target project is expected to have the key fields set, otherwise they won't be matched.

NOTE: PLEASE MAKE SURE THE SKU FIELDS OF ALL PRODUCTS ARE SET AS THE SYNC LIBRARY WILL BE MIGRATED TO MATCH PRODUCTS BY SKU INSTEAD OF KEY IN THE FUTURE.

  1. Every product may have several references including product type, categories, taxCategory, etc.. Variants of the product also have prices, where each prices also has some references including a reference to the Type of its custom fields and a reference to a channel. All these referenced resources are matched by their key Therefore, in order for the sync to resolve the actual ids of those references, those keys have to be supplied in one of two ways:

    • Provide the key value on the id field of the reference. This means that calling getId() on the reference would return its key. Note that the library will check that this key is not provided in UUID format by default. However, if you want to provide the key in UUID format, you can set it through the sync options.

    • Provide the reference expanded. This means that calling getObj() on the reference should not return null, but return the Type object, from which the its key can be directly accessible.

      Note: This library provides you with a utility method replaceProductsReferenceIdsWithKeys that replaces the references id fields with keys, in order to make them ready for reference resolution by the sync:

      // Puts the keys in the reference id fields to prepare for reference resolution
final List<ProductDraft> productDrafts = replaceProductsReferenceIdsWithKeys(products);

  2. It is an important responsibility of the user of the library to instantiate a sphereClient that has the following properties:

    • Limits the amount of concurrent requests done to CTP. This can be done by decorating the sphereClient with QueueSphereClientDecorator
    • Retries on 5xx errors with a retry strategy. This can be achieved by decorating the sphereClient with the RetrySphereClientDecorator

    You can use the same client instantiating used in the integration tests for this library found here.

  3. After the sphereClient is setup, a ProductSyncOptions should be be built as follows:

// instantiating a ProductSyncOptions
final ProductSyncOptions productSyncOptions = ProductSyncOptionsBuilder.of(sphereClient).build();

The options can be used to provide additional optional configuration for the sync as well:

  • errorCallBack a callback that is called whenever an event occurs during the sync process that represents an error. Currently, these events.

  • warningCallBack a callback that is called whenever an event occurs during the sync process that represents a warning. Currently, these events.

  • syncFilter represents either a blacklist or a whitelist for filtering certain update action groups.

    • Blacklisting an update action group means that everything in products will be synced except for any group in the blacklist. A typical use case it to blacklist prices when syncing products, so as to sync everything in products except prices. Here is an example where the sync is performed while blacklisting product categories.

    • Whitelisting an update action group means that the groups in this whitelist will be the only group synced in products. One use case could be to whitelist prices when syncing products, so as to only sync prices in products and nothing else. Here is an example where the sync is performed while whitelisting product names.

    • The list of action groups allowed to be blacklist or whitelisted on products can be found here.

  • beforeUpdateCallback a filter function which can be applied on a generated list of update actions. It allows the user to intercept product update and modify (add/remove) update actions just before they are send to CTP API.

  • beforeCreateCallback a filter function which can be applied on a product draft before a request to create it on CTP is issued. It allows the user to intercept product create requests modify the draft before the create request is sent to CTP API.

  • allowUuid a flag, if set to true, enables the user to use keys with UUID format for references. By default, it is set to false.

Example of options usage, that sets the error and warning callbacks to output the message to the log error and warning streams, would look as follows:

final Logger logger = LoggerFactory.getLogger(MySync.class);
final ProductSyncOptions productsyncOptions = ProductSyncOptionsBuilder.of(sphereClient)
                                                                       .errorCallBack(logger::error)
                                                                       .warningCallBack(logger::warn)
                                                                       .build();

Running the sync

After all the aforementioned points in the previous section have been fulfilled, to run the sync:

// instantiating a product sync
final ProductSync productSync = new ProductSync(productSyncOptions);

// execute the sync on your list of products
CompletionStage<ProductSyncStatistics> syncStatisticsStage = productSync.sync(productDrafts);

The result of the completing the syncStatisticsStage in the previous code snippet contains a ProductSyncStatistics which contains all the stats of the sync process; which includes a report message, the total number of updated, created, failed, processed categories and the processing time of the sync in different time units and in a human readable format.

final ProductSyncStatistics stats = syncStatisticsStage.toCompletebleFuture().join();
stats.getReportMessage(); 
/*"Summary: 2000 products were processed in total (1000 created, 995 updated and 5 failed to sync)."*/

Note The statistics object contains the processing time of the last batch only. This is due to two reasons:

  1. The sync processing time should not take into account the time between supplying batches to the sync.
  2. It is not not known by the sync which batch is going to be the last one supplied.

More examples of how to use the sync

  1. From another CTP project as source can be found here.
  2. From an external source can be found here.
  3. Syncing with blacklisting/whitelisting here.

Build all update actions

A utility method provided by the library to compare a Product with a new ProductDraft and results in a list of product update actions.

List<UpdateAction<Product>> updateActions = ProductSyncUtils.buildActions(product, productDraft, productSyncOptions, attributesMetaData);

Examples of its usage can be found in the tests here.

Build particular update action(s)

Utility methods provided by the library to compare the specific fields of a Product and a new ProductDraft, and in turn builds the update action. One example is the buildChangeNameUpdateAction which compares names:

Optional<UpdateAction<Product>> updateAction = buildChangeNameUpdateAction(oldProduct, productDraft);

More examples of those utils for different fields can be found here.

Caveats

  1. Products are either created or updated. Currently the tool does not support product deletion.
  2. The library doesn't sync product variant assets yet #3, but it will not delete them.
  3. The library will not sync attribute field types with ReferenceType and SetType field definitions, except for Product references. (See more: #87 #160)