Hot backup for S3:errmsg:EntityTooLarge : Unable to parse ExceptionName: EntityTooLarge Message #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GitOrigin-RevId: 30b6d737b98addcb101720824cd6e7df47ad3157
…#20584) GitOrigin-RevId: a785a61bbaf969bc78699887d3bda848ea6a5d30
….cpp (#20571) GitOrigin-RevId: 8f0910c87cca6b11418eb619f4f36f71c7e94a3d
…-series buckets (#20592) GitOrigin-RevId: 6427f47f6a678c18d04028cddc4d3abfd4b96ce0
… variants. (#20301) GitOrigin-RevId: c685333ff1656a47021c0ac2447143e5ea512c5a
…nParticipant (#20600) GitOrigin-RevId: 3694d922076738055e2166b4e9d5f235e19772f5
GitOrigin-RevId: 46c629fba804b3f70f74a455084df3acf18c1fee
… retryable update that modifies doc shard key (#20067) GitOrigin-RevId: c44f704d60f7354bb1d872b39eff0fdfb75de6b2
GitOrigin-RevId: 0271d12f5b0794ff735a1f77fcd8422b19a66a22
GitOrigin-RevId: 0e7522126001f966e314c9ccc99391088c85b465
GitOrigin-RevId: 433b8b0569226c6637d4bcc906ac7f1bb3a75950
…#17292) GitOrigin-RevId: 5c31098540b36db9940067b32de6f0bdc1f75ec9
…suite tag (#19971)" This reverts commit dd77915a25fdbbd9c015729ebcbf3806642ac197. GitOrigin-RevId: dc287509f0cdb0be79caf90add1774289dd76124
GitOrigin-RevId: f1ae9e962d9c2881915ed0d3545fde31f2f2179a
…rge issue." This reverts commit ca5486367db5ff3962349ff2e2da1991a5669108. GitOrigin-RevId: d741b7c53409ab8c11c77515dfbe2558655c6082
GitOrigin-RevId: 066ef12a1c2913a017789c5186bad758fa07d451
GitOrigin-RevId: f79436d6edf636df862c584ad65dccf30f6642e8
…nfig_transitions.yml (#20576) GitOrigin-RevId: dbc0b04c409fcc1b79e3c76bb89dec4ff4cf5a69
…yml (#20575) GitOrigin-RevId: 9771c3a05d59254420c94700d60ea9e7bbbe0854
GitOrigin-RevId: b754b02c9e125f049e481bf222adde74e717c252
…e transition to primary. (#20605) GitOrigin-RevId: 169d167815586116f2a48965959b28d7d6d97156
…riants (#20584)" This reverts commit a785a61bbaf969bc78699887d3bda848ea6a5d30. GitOrigin-RevId: 815528b0ce41ad5c45a3c5a89366c985c9b242bd
(cherry picked from commit 60837f8d1d37ebb5d3effb205154a60148f76b11) GitOrigin-RevId: 67da0d665d605978ffcfb1e34d9a5681cb623cb0
… only Point Types (#20287) GitOrigin-RevId: 7784fe68588c5559b326c6423c0c09e35b7b156f
…ll types GitOrigin-RevId: 7be925c148c771a96c4d369bec768961039c42ca
…through suite (#20439) GitOrigin-RevId: e0c1addcb9fad1516cefbe3c339e95bf76a43dc2
…ark tests (#19821) GitOrigin-RevId: 5b68292aaf4ce1958c846eabff2318197e1e0449
GitOrigin-RevId: e63fa63710563ca2ef85383a719fbb67e07a3208
GitOrigin-RevId: 89f5b3de631bb3e8affd755e483e63a1ab2c3045
…om getClusterParameter reply GitOrigin-RevId: 96c758199c1db69d30d16c828063352f3abf824f
'8.1.0-alpha' Conflicts: src/third_party/wiredtiger/src/conn/conn_api.c
Telemetry Phase 1 for the master branch
Don't create a new secret version on a Vault server when `max_versions` is reached.
* PSMDB-1535 change the warning message in assert * PSMDB-1535 use regex for the warning message in assert * PSMDB-1535 fix regex for the warning message in assert (cherry picked from commit bbc64f2)
New metric 'intial_sync_method' will be added if node is part of replica set
We cannot use the same escaping implementation for each part of LDAP query. This is because LDAP query filters (see RFC 4515) require escaping of some special characters which are used to build filter expressions. Thus substituting placeholders in filter expression differs from substituting in other parts of LDAP query.
It is not documented formally but we can assume more or less that `{USER}` placeholder will often contain distinguished name string values. For example if we use `userToDNMapping` mechanism and its `ldapQuery` to get the `{USER}` value it will in most cases contain DN value returned by LDAP server. Nevertheless `{PROVIDED_USER}` can be anything: DN, email, some other user identifier.
When we receive DN value from LDAP server as a result of `ldapQuery` it is already in escaped form - defined in RFC 4514 - thus in this case we don't need to do DN escaping. But if we don't use `userToDNMapping` mechanism values in `{PROVIDED_USER}` and `{USER}` are provided directly by the client application - that means it is client's responsibility to escape DN according to RFC 4514 if client wants to use DN as a user name.
In `substitution` no escaping happens during replacing placeholders. Result string becomes value for `{USER}` placeholder and that value is escaped when that placeholder is substituted in the `queryTemplate`. But if `substitution` contains parts of DN those parts should be escaped by config maintainer using DN escaping rules (RFC 4514).
`ldapQuery` can contain LDAP search filter. That means that if substitution placeholders are found in search filter server will need to do additional escaping on substitution value - as described in RFC 4515. Also URL escaping of substituted values is done before executing the query.
mongoDB server always escapes substitution values using URL escaping rules (RFC 4516). If that substitution occurs inside LDAP search filter then URL escaping is going AFTER search filter escaping.
LDAP queries specified in `queryTemplate` and `ldapQuery` configuration parameters must be escaped by config maintainer. All three escaping mechanisms should be used if necessary:
- DN escaping - when full or partial DN is hardcoded into query
- search filter escaping - when query contains search filter
- URL escaping - when query contains some special characters which can alter query interpretation (think about question mark character for example)
Update README.md
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use version:4.0.12-6
Use S3 equipment for hot backup
command:
db.runCommand({createBackup: 1, s3: {bucket: "mongobackup",
path: "wjs/mongodb_backup",
scheme: "HTTP",
endpoint: "**********",
useVirtualAddressing: false,
profile: "localminio"}})
error:
{
"ok" : 0,
"errmsg" : "Cannot backup '/-29-7545783987985807831.wt' : EntityTooLarge : Unable to parse ExceptionName: EntityTooLarge Message: ",
"operationTime" : Timestamp(1576636321, 24),
"$clusterTime" : {
"clusterTime" : Timestamp(1576636321, 24),
"signature" : {
"hash" : BinData(0,"bON+Efo7mgez50xriHl5Yo/ZKHw="),
"keyId" : NumberLong("6737552503321133058")
}
}
}
I looked at some documentation:
aws/aws-sdk-cpp#386
I think it's probably an SDK usage issue
so !
Can you help locate the problem