Skip to content

Commit

Permalink
PS-9181 [DOCS] - document background dictionary cache reload function…
Browse files Browse the repository at this point in the history
…ality in the Masking Functions component 8.0

	modified:   docs/data-masking-function-list.md
	modified:   docs/data-masking-overview.md
  • Loading branch information
patrickbirch committed Dec 31, 2024
1 parent e7b9f6d commit ff13b47
Show file tree
Hide file tree
Showing 2 changed files with 82 additions and 0 deletions.
59 changes: 59 additions & 0 deletions docs/data-masking-function-list.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ The feature is in [tech preview](glossary.md#tech-preview).

| **Name** | **Usage** |
|---------------------------------------------------|-------------------------------------------------------|
| [dictionaries_flush_interval_seconds (integer, unsigned)](#dictionaries_flush_interval_secondsinteger-unsigned) | The number of seconds between updates to the internal dictionary cache to match changes in the dictionaries table.|
| [`gen_blocklist(str, from_dictionary_name, to_dictionary_name)`](#gen_blockliststr-from_dictionary_name-to_dictionary_name) | Replace a term from a dictionary |
| [`gen_dictionary(dictionary_name)`](#gen_dictionarydictionary_name) | Returns a random term from a dictionary |
| [`gen_range(lower, upper)`](#gen_rangelower-upper) | Returns a number from a range |
Expand All @@ -24,14 +25,40 @@ The feature is in [tech preview](glossary.md#tech-preview).
| [`mask_ssn(str [,mask_char])`](#mask_ssnstr-mask_char) | Masks the US Social Security number |
| [`mask_uk_nin(str [,mask_char])`](#mask_uk_ninstr-mask_char) | Masks the United Kingdom National Insurance number |
| [`mask_uuid(str [,mask_char])`](#mask_uuidstr-mask_char) | Masks the Universally Unique Identifier |
| [`masking_dictionaries(str)`](#masking_dictionariesstr) | Specify a different database name that contains the dictionaries table. |
| [`masking_dictionaries_flush()`](#masking_dictionaries_flush) | Resyncs the internal dictionary term cache |
| [`masking_dictionary_remove(dictionary_name)`](#masking_dictionary_removedictionary_name) | Removes the dictionary |
| [`masking_dictionary_term_add(dictionary_name, term_name)`](#masking_dictionary_term_adddictionary_name-term_name) | Adds a term to the masking dictionary |
| [`masking_dictionary_term_remove(dictionary_name, term_name)`](#masking_dictionary_term_removedictionary_name-term_name) | Removes a term from the masking dictionary |


## dictionaries_flush_interval_seconds(integer, unsigned)

The number of seconds between a synchronization between the dictionaries table and the internal dictionary cache.

This variable is read-only. The default value is 0 (zero) and there is no synchronization.

## Version update

Percona Server for MySQL 8.0.41 introduces this variable.

### Parameters

| Parameter | Optional | Description | Type |
| --- | --- | --- | --- |
| `seconds` | Yes | The number of seconds between the dictionary internal cache and dictionaries table synchronization | Integer |


## gen_blocklist(str, from_dictionary_name, to_dictionary_name)

Replaces a term from one dictionary with a randomly selected term in another dictionary.

### Version update

Percona Server for MySQL 8.0.41 introduces an internal term cache. Instead of querying the underlying `mysql.masking_dictionaries` table each time a function is executed, the server now utilizes internal in-memory data structures for lookups. This enhancement significantly improves performance, particularly when processing multiple rows.



### Parameters

| Parameter | Optional | Description | Type |
Expand Down Expand Up @@ -760,6 +787,38 @@ mysql> SELECT mask_uuid('9a3b642c-06c6-11ee-be56-0242ac120002');
+-------------------------------------------------------+
```

## masking_database(string)

Specify a database name that contains the dictionaries table. The default is `mysql`.


## masking_dictionaries_flush()

Resyncs the internal dictionary term cache.

### Parameters

None

### Returns

Returns a string value of `1` (one) when successful.

### Example

```{.bash data-prompt="mysql>"}
mysql> SELECT masking_dictionaries_flush();
```
??? example "Expected output"

```{.text .no-copy}
+------------------------------+
| masking_dictionaries_flush() |
+------------------------------+
| 1 |
+---------------------------- +
```

## masking_dictionary_remove(dictionary_name)

Removes all of the terms and then removes the dictionary.
Expand Down
23 changes: 23 additions & 0 deletions docs/data-masking-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,29 @@ These examples underscore how data masking serves as a crucial safeguard for sen

Data masking helps to limit the exposure of sensitive data by preventing access to non-authorized users. Masking provides a way to create a version of the data in situations, such as a presentation, sales demo, or software testing, when the real data should not be used. Data masking changes the data values while using the same format and cannot be reverse engineered. Masking reduces an organization's risk by making the data useless to an outside party.

## Version updates

Percona Server for MySQL 8.0.41 introduces an internal term cache for the
following functions in the [data masking component](data-masking-function-list.md):

* [gen_blocklist](data-masking-function-list.md#gen_blockliststr-from_dictionary_name-to_dictionary_name)

* [gen_dictionary](data-masking-function-list.md#gen_dictionarydictionary_name)

Instead of querying the underlying `mysql.masking_dictionaries` table each time a function is executed, the server now utilizes internal in-memory data structures for lookups. This enhancement significantly improves performance, particularly when processing multiple rows.

With this redesign, the internal dictionary term cache might get out of sync with the underlying dictionaries table (default is `mysql.masking_dictionaries`). This can happen if you directly change the table instead of using the dedicated dictionary manipulation functions (`[masking_dictionary_term_add()]((data-masking-function-list.md#masking_dictionary_term_adddictionary_name-term_name`), [`masking_dictionary_term_remove()`](data-masking-function-list.md#masking_dictionary_term_removedictionary_name-term_name), [`masking_dictionary_remove()`](data-masking-function-list.md#masking_dictionary_removedictionary_name).

To resync the internal dictionary term cache, we added a new function called [`masking_dictionaries_flush()`](data-masking-function-list.md#masking_dictionaries_flush). This function takes no arguments and returns 1 when it succeeds.

This redesign also affects row-based replication. Changes to the dictionaries table, either through dedicated functions or directly on the source, are sent to a replica via the binary log. The applier thread reads these binary log events on the replica and applies them successfully. However, the dictionary term cache on the replica doesn't update automatically.

We introduced a new system variable called `component_masking_functions.dictionaries_flush_interval_seconds (read-only, integer, unsigned, default 0)`.

When you set this variable to any value other than 0, the component starts a background thread at startup that periodically syncs the dictionaries table with the internal dictionary term cache. The value specifies the number of seconds between each sync.

If this variable has a non-zero value on a replica, the dictionary term cache eventually syncs with the underlying dictionaries table after receiving those binary log events.

## Data masking techniques

The common data masking techniques are the following:
Expand Down

0 comments on commit ff13b47

Please sign in to comment.