Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Avoid allocations while constructing path segments #482

Merged
merged 1 commit into from
May 3, 2024
Merged

perf: Avoid allocations while constructing path segments #482

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
### Performance

- Optimize building `JSONPointer` for validation errors by allocating the exact amount of memory needed.
- Avoid cloning path segments during validation.

### Changed

Expand Down
16 changes: 8 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ $ cargo test

## Performance

There is a comparison with other JSON Schema validators written in Rust - `jsonschema_valid==0.4.0` and `valico==3.6.0`.
There is a comparison with other JSON Schema validators written in Rust - `jsonschema_valid==0.5.2` and `valico==4.0.0`.

Test machine i8700K (12 cores), 32GB RAM.

Expand All @@ -250,20 +250,20 @@ Here is the average time for each contender to validate. Ratios are given agains

| Case | jsonschema_valid | valico | jsonschema (validate) | jsonschema (is_valid) |
| -------------- | ----------------------- | ----------------------- | --------------------- | ---------------------- |
| OpenAPI | - (1) | - (1) | 4.717 ms | 4.279 ms (**x0.90**) |
| Swagger | - (2) | 83.357 ms (**x12.47**) | 6.681 ms | 4.533 ms (**x0.67**) |
| Canada | 32.987 ms (**x31.38**) | 141.41 ms (**x134.54**) | 1.051 ms | 1.046 ms (**x0.99**) |
| CITM catalog | 4.735 ms (**x2.00**) | 13.222 ms (**x5.58**) | 2.367 ms | 535.07 us (**x0.22**) |
| Fast (valid) | 2.00 us (**x3.85**) | 3.18 us (**x6.13**) | 518.39 ns | 97.91 ns (**x0.18**) |
| Fast (invalid) | 339.28 ns (**x0.50**) | 3.34 us (**x5.00**) | 667.55 ns | 5.41ns (**x0.01**) |
| OpenAPI | - (1) | - (1) | 3.500 ms | 3.147 ms (**x0.89**) |
| Swagger | - (2) | 180.65 ms (**x32.12**) | 5.623 ms | 3.634 ms (**x0.64**) |
| Canada | 40.363 ms (**x33.13**) | 427.40 ms (**x350.90**) | 1.218 ms | 1.217 ms (**x0.99**) |
| CITM catalog | 5.357 ms (**x2.51**) | 39.215 ms (**x18.44**) | 2.126 ms | 569.23 us (**x0.26**) |
| Fast (valid) | 2.27 us (**x4.87**) | 6.55 us (**x14.05**) | 465.89 ns | 113.94 ns (**x0.24**) |
| Fast (invalid) | 412.21 ns (**x0.46**) | 6.69 us (**x7.61**) | 878.23 ns | 4.21ns (**x0.004**) |

Notes:

1. `jsonschema_valid` and `valico` do not handle valid path instances matching the `^\\/` regex.

2. `jsonschema_valid` fails to resolve local references (e.g. `#/definitions/definitions`).

You can find benchmark code in `benches/jsonschema.rs`, Rust version is `1.57`.
You can find benchmark code in `benches/jsonschema.rs`, Rust version is `1.78`.

## Support

Expand Down
2 changes: 1 addition & 1 deletion jsonschema-test-suite/proc_macro/src/mockito_mocks.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ pub(crate) fn setup(json_schema_test_suite_path: &Path) -> Vec<TokenStream> {
let path = remote_path
.trim_start_matches(base_path)
.replace(std::path::MAIN_SEPARATOR, "/");
if let Ok(file_content) = std::fs::read_to_string(remote_path) {
if let Ok(file_content) = fs::read_to_string(remote_path) {
Some(quote! {
mockito::mock("GET", #path)
.with_body(
Expand Down
8 changes: 4 additions & 4 deletions jsonschema/src/compilation/context.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use super::options::CompilationOptions;
use crate::{
compilation::DEFAULT_SCOPE,
paths::{JSONPointer, JsonPointerNode, PathChunk},
paths::{JSONPointer, JsonPointerNode, PathChunkRef},
resolver::Resolver,
schemas,
};
Expand All @@ -17,7 +17,7 @@ pub(crate) struct CompilationContext<'a> {
base_uri: BaseUri<'a>,
pub(crate) config: Arc<CompilationOptions>,
pub(crate) resolver: Arc<Resolver>,
pub(crate) schema_path: JsonPointerNode<'a>,
pub(crate) schema_path: JsonPointerNode<'a, 'a>,
}

#[derive(Debug, Clone)]
Expand Down Expand Up @@ -118,7 +118,7 @@ impl<'a> CompilationContext<'a> {
}

#[inline]
pub(crate) fn with_path(&'a self, chunk: impl Into<PathChunk>) -> Self {
pub(crate) fn with_path(&'a self, chunk: impl Into<PathChunkRef<'a>>) -> Self {
let schema_path = self.schema_path.push(chunk);
CompilationContext {
base_uri: self.base_uri.clone(),
Expand All @@ -136,7 +136,7 @@ impl<'a> CompilationContext<'a> {

/// Create a JSON Pointer from the current `schema_path` & a new chunk.
#[inline]
pub(crate) fn as_pointer_with(&self, chunk: impl Into<PathChunk>) -> JSONPointer {
pub(crate) fn as_pointer_with(&'a self, chunk: impl Into<PathChunkRef<'a>>) -> JSONPointer {
self.schema_path.push(chunk).into()
}

Expand Down
4 changes: 2 additions & 2 deletions jsonschema/src/compilation/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ pub struct JSONSchema {
}

pub(crate) static DEFAULT_SCOPE: Lazy<Url> =
Lazy::new(|| url::Url::parse(DEFAULT_ROOT_URL).expect("Is a valid URL"));
Lazy::new(|| Url::parse(DEFAULT_ROOT_URL).expect("Is a valid URL"));

impl JSONSchema {
/// Return a default `CompilationOptions` that can configure
Expand Down Expand Up @@ -200,7 +200,7 @@ pub(crate) fn compile_validators<'a>(
}
// Check if this keyword is overridden, then check the standard definitions
if let Some(factory) = context.config.get_keyword_factory(keyword) {
let path = context.as_pointer_with(keyword.to_owned());
let path = context.as_pointer_with(keyword.as_str());
let validator = CustomKeyword::new(factory.init(object, subschema, path)?);
let validator: BoxedValidator = Box::new(validator);
validators.push((keyword.clone(), validator));
Expand Down
28 changes: 14 additions & 14 deletions jsonschema/src/keywords/additional_properties.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ macro_rules! is_valid_patterns {

macro_rules! validate {
($node:expr, $value:ident, $instance_path:expr, $property_name:expr) => {{
let instance_path = $instance_path.push($property_name.clone());
let instance_path = $instance_path.push($property_name.as_str());
$node.validate($value, &instance_path)
}};
}
Expand Down Expand Up @@ -125,12 +125,12 @@ impl Validate for AdditionalPropertiesValidator {
let mut matched_props = Vec::with_capacity(item.len());
let mut output = BasicOutput::default();
for (name, value) in item {
let path = instance_path.push(name.to_string());
let path = instance_path.push(name.as_str());
output += self.node.apply_rooted(value, &path);
matched_props.push(name.clone());
}
let mut result: PartialApplication = output.into();
result.annotate(serde_json::Value::from(matched_props).into());
result.annotate(Value::from(matched_props).into());
result
} else {
PartialApplication::valid_empty()
Expand Down Expand Up @@ -298,7 +298,7 @@ impl<M: PropertiesValidatorsMap> Validate for AdditionalPropertiesNotEmptyFalseV
let mut output = BasicOutput::default();
for (property, value) in item {
if let Some((_name, node)) = self.properties.get_key_validator(property) {
let path = instance_path.push(property.clone());
let path = instance_path.push(property.as_str());
output += node.apply_rooted(value, &path);
} else {
unexpected.push(property.clone())
Expand Down Expand Up @@ -424,7 +424,7 @@ impl<M: PropertiesValidatorsMap> Validate for AdditionalPropertiesNotEmptyValida
let mut matched_propnames = Vec::with_capacity(map.len());
let mut output = BasicOutput::default();
for (property, value) in map {
let path = instance_path.push(property.clone());
let path = instance_path.push(property.as_str());
if let Some((_name, property_validators)) =
self.properties.get_key_validator(property)
{
Expand All @@ -436,7 +436,7 @@ impl<M: PropertiesValidatorsMap> Validate for AdditionalPropertiesNotEmptyValida
}
let mut result: PartialApplication = output.into();
if !matched_propnames.is_empty() {
result.annotate(serde_json::Value::from(matched_propnames).into());
result.annotate(Value::from(matched_propnames).into());
}
result
} else {
Expand Down Expand Up @@ -561,7 +561,7 @@ impl Validate for AdditionalPropertiesWithPatternsValidator {
let mut pattern_matched_propnames = Vec::with_capacity(item.len());
let mut additional_matched_propnames = Vec::with_capacity(item.len());
for (property, value) in item {
let path = instance_path.push(property.clone());
let path = instance_path.push(property.as_str());
let mut has_match = false;
for (pattern, node) in &self.patterns {
if pattern.is_match(property).unwrap_or(false) {
Expand All @@ -580,13 +580,13 @@ impl Validate for AdditionalPropertiesWithPatternsValidator {
self.pattern_keyword_path.clone(),
instance_path.into(),
self.pattern_keyword_absolute_path.clone(),
serde_json::Value::from(pattern_matched_propnames).into(),
Value::from(pattern_matched_propnames).into(),
)
.into();
}
let mut result: PartialApplication = output.into();
if !additional_matched_propnames.is_empty() {
result.annotate(serde_json::Value::from(additional_matched_propnames).into())
result.annotate(Value::from(additional_matched_propnames).into())
}
result
} else {
Expand Down Expand Up @@ -707,7 +707,7 @@ impl Validate for AdditionalPropertiesWithPatternsFalseValidator {
let mut unexpected = Vec::with_capacity(item.len());
let mut pattern_matched_props = Vec::with_capacity(item.len());
for (property, value) in item {
let path = instance_path.push(property.clone());
let path = instance_path.push(property.as_str());
let mut has_match = false;
for (pattern, node) in &self.patterns {
if pattern.is_match(property).unwrap_or(false) {
Expand All @@ -725,7 +725,7 @@ impl Validate for AdditionalPropertiesWithPatternsFalseValidator {
self.pattern_keyword_path.clone(),
instance_path.into(),
self.pattern_keyword_absolute_path.clone(),
serde_json::Value::from(pattern_matched_props).into(),
Value::from(pattern_matched_props).into(),
)
.into();
}
Expand Down Expand Up @@ -905,7 +905,7 @@ impl<M: PropertiesValidatorsMap> Validate for AdditionalPropertiesWithPatternsNo
let mut output = BasicOutput::default();
let mut additional_matches = Vec::with_capacity(item.len());
for (property, value) in item {
let path = instance_path.push(property.clone());
let path = instance_path.push(property.as_str());
if let Some((_name, node)) = self.properties.get_key_validator(property) {
output += node.apply_rooted(value, &path);
for (pattern, node) in &self.patterns {
Expand All @@ -928,7 +928,7 @@ impl<M: PropertiesValidatorsMap> Validate for AdditionalPropertiesWithPatternsNo
}
}
let mut result: PartialApplication = output.into();
result.annotate(serde_json::Value::from(additional_matches).into());
result.annotate(Value::from(additional_matches).into());
result
} else {
PartialApplication::valid_empty()
Expand Down Expand Up @@ -1097,7 +1097,7 @@ impl<M: PropertiesValidatorsMap> Validate
let mut unexpected = vec![];
// No properties are allowed, except ones defined in `properties` or `patternProperties`
for (property, value) in item {
let path = instance_path.push(property.clone());
let path = instance_path.push(property.as_str());
if let Some((_name, node)) = self.properties.get_key_validator(property) {
output += node.apply_rooted(value, &path);
for (pattern, node) in &self.patterns {
Expand Down
4 changes: 2 additions & 2 deletions jsonschema/src/keywords/contains.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,12 +86,12 @@ impl Validate for ContainsValidator {
.into(),
);
} else {
result.annotate(serde_json::Value::from(indices).into());
result.annotate(Value::from(indices).into());
}
result
} else {
let mut result = PartialApplication::valid_empty();
result.annotate(serde_json::Value::Array(Vec::new()).into());
result.annotate(Value::Array(Vec::new()).into());
result
}
}
Expand Down
6 changes: 3 additions & 3 deletions jsonschema/src/keywords/dependencies.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ impl DependenciesValidator {
let keyword_context = context.with_path("dependencies");
let mut dependencies = Vec::with_capacity(map.len());
for (key, subschema) in map {
let item_context = keyword_context.with_path(key.to_string());
let item_context = keyword_context.with_path(key.as_str());
let s = match subschema {
Value::Array(_) => {
let validators = vec![required::compile_with_path(
Expand Down Expand Up @@ -106,7 +106,7 @@ impl DependentRequiredValidator {
let keyword_context = context.with_path("dependentRequired");
let mut dependencies = Vec::with_capacity(map.len());
for (key, subschema) in map {
let item_context = keyword_context.with_path(key.to_string());
let item_context = keyword_context.with_path(key.as_str());
if let Value::Array(dependency_array) = subschema {
if !unique_items::is_unique(dependency_array) {
return Err(ValidationError::unique_items(
Expand Down Expand Up @@ -196,7 +196,7 @@ impl DependentSchemasValidator {
let keyword_context = context.with_path("dependentSchemas");
let mut dependencies = Vec::with_capacity(map.len());
for (key, subschema) in map {
let item_context = keyword_context.with_path(key.to_string());
let item_context = keyword_context.with_path(key.as_str());
let schema_nodes = compile_validators(subschema, &item_context)?;
dependencies.push((key.clone(), schema_nodes));
}
Expand Down
16 changes: 8 additions & 8 deletions jsonschema/src/keywords/pattern_properties.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ impl PatternPropertiesValidator {
let keyword_context = context.with_path("patternProperties");
let mut patterns = Vec::with_capacity(map.len());
for (pattern, subschema) in map {
let pattern_context = keyword_context.with_path(pattern.to_string());
let pattern_context = keyword_context.with_path(pattern.as_str());
patterns.push((
match Regex::new(pattern) {
Ok(r) => r,
Expand Down Expand Up @@ -71,7 +71,7 @@ impl Validate for PatternPropertiesValidator {
item.iter()
.filter(move |(key, _)| re.is_match(key).unwrap_or(false))
.flat_map(move |(key, value)| {
let instance_path = instance_path.push(key.clone());
let instance_path = instance_path.push(key.as_str());
node.validate(value, &instance_path)
})
})
Expand All @@ -93,14 +93,14 @@ impl Validate for PatternPropertiesValidator {
for (pattern, node) in &self.patterns {
for (key, value) in item {
if pattern.is_match(key).unwrap_or(false) {
let path = instance_path.push(key.clone());
let path = instance_path.push(key.as_str());
matched_propnames.push(key.clone());
sub_results += node.apply_rooted(value, &path);
}
}
}
let mut result: PartialApplication = sub_results.into();
result.annotate(serde_json::Value::from(matched_propnames).into());
result.annotate(Value::from(matched_propnames).into());
result
} else {
PartialApplication::valid_empty()
Expand Down Expand Up @@ -135,7 +135,7 @@ impl SingleValuePatternPropertiesValidator {
context: &CompilationContext,
) -> CompilationResult<'a> {
let keyword_context = context.with_path("patternProperties");
let pattern_context = keyword_context.with_path(pattern.to_string());
let pattern_context = keyword_context.with_path(pattern);
Ok(Box::new(SingleValuePatternPropertiesValidator {
pattern: match Regex::new(pattern) {
Ok(r) => r,
Expand Down Expand Up @@ -175,7 +175,7 @@ impl Validate for SingleValuePatternPropertiesValidator {
.iter()
.filter(move |(key, _)| self.pattern.is_match(key).unwrap_or(false))
.flat_map(move |(key, value)| {
let instance_path = instance_path.push(key.clone());
let instance_path = instance_path.push(key.as_str());
self.node.validate(value, &instance_path)
})
.collect();
Expand All @@ -195,13 +195,13 @@ impl Validate for SingleValuePatternPropertiesValidator {
let mut outputs = BasicOutput::default();
for (key, value) in item {
if self.pattern.is_match(key).unwrap_or(false) {
let path = instance_path.push(key.clone());
let path = instance_path.push(key.as_str());
matched_propnames.push(key.clone());
outputs += self.node.apply_rooted(value, &path);
}
}
let mut result: PartialApplication = outputs.into();
result.annotate(serde_json::Value::from(matched_propnames).into());
result.annotate(Value::from(matched_propnames).into());
result
} else {
PartialApplication::valid_empty()
Expand Down
8 changes: 4 additions & 4 deletions jsonschema/src/keywords/properties.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ impl PropertiesValidator {
let context = context.with_path("properties");
let mut properties = Vec::with_capacity(map.len());
for (key, subschema) in map {
let property_context = context.with_path(key.clone());
let property_context = context.with_path(key.as_str());
properties.push((
key.clone(),
compile_validators(subschema, &property_context)?,
Expand Down Expand Up @@ -68,7 +68,7 @@ impl Validate for PropertiesValidator {
.flat_map(move |(name, node)| {
let option = item.get(name);
option.into_iter().flat_map(move |item| {
let instance_path = instance_path.push(name.clone());
let instance_path = instance_path.push(name.as_str());
node.validate(item, &instance_path)
})
})
Expand All @@ -89,13 +89,13 @@ impl Validate for PropertiesValidator {
let mut matched_props = Vec::with_capacity(props.len());
for (prop_name, node) in &self.properties {
if let Some(prop) = props.get(prop_name) {
let path = instance_path.push(prop_name.clone());
let path = instance_path.push(prop_name.as_str());
matched_props.push(prop_name.clone());
result += node.apply_rooted(prop, &path);
}
}
let mut application: PartialApplication = result.into();
application.annotate(serde_json::Value::from(matched_props).into());
application.annotate(Value::from(matched_props).into());
application
} else {
PartialApplication::valid_empty()
Expand Down
Loading