Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(torii-indexer): natural priority in parallelized tasks for model… #2912

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

Larkooo
Copy link
Collaborator

@Larkooo Larkooo commented Jan 15, 2025

Summary by CodeRabbit

  • Refactor
    • Updated internal data structure for task management to improve performance and processing efficiency.
    • Refined event processing logic with a more nuanced approach to task identification and parallel processing, allowing for prioritized execution of tasks.
    • Enhanced error handling and logging in various processors to provide clearer insights into processing flow and potential issues.

Copy link

coderabbitai bot commented Jan 15, 2025

Walkthrough

Ohayo, sensei! The pull request introduces significant modifications to the task management system in the Engine struct. The primary change involves switching from a HashMap to a BTreeMap for storing tasks, which impacts the initialization and processing of events. The modification enhances the event processing logic by implementing a more structured task identifier generation mechanism based on priority levels, allowing for prioritized processing of tasks while maintaining concurrency within the same priority level.

Changes

File Change Summary
crates/torii/indexer/src/engine.rs - Replaced HashMap with BTreeMap for tasks field
- Updated new method to initialize tasks with BTreeMap::new()
- Restructured task processing to handle tasks by priority sequentially while allowing concurrent processing of events within the same task
- Added type aliases for TaskPriority and TaskId
crates/torii/indexer/src/processors/store_del_record.rs - Updated process method to use config parameter
- Enhanced error handling to check config.namespaces before skipping model retrieval errors
- Wrapped errors in anyhow::anyhow! for better context
crates/torii/indexer/src/processors/store_set_record.rs - Renamed _config to config in process method
- Improved error handling logic to depend on config.namespaces state
crates/torii/indexer/src/processors/store_update_member.rs - Updated process method to include debug logging for model retrieval errors
- Enhanced error reporting with anyhow::anyhow!
crates/torii/indexer/src/processors/store_update_record.rs - Changed _config to config in process method
- Improved error handling to check config.namespaces before skipping errors

Possibly related PRs

  • fix(torii-core): correct parallelized task hash #2860: The changes in this PR involve modifications to the process_event method within the Engine struct, which is directly related to the Engine struct modifications in the main PR, particularly in how task identifiers are generated and processed.

Suggested Reviewers

  • glihm

Sensei, the changes look quite intriguing! The transition to a BTreeMap and the more nuanced event processing approach should bring some interesting improvements to the system. Ohayo and happy coding! 🍵🖥️


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2ad2772 and f461cea.

📒 Files selected for processing (5)
  • crates/torii/indexer/src/engine.rs (6 hunks)
  • crates/torii/indexer/src/processors/store_del_record.rs (2 hunks)
  • crates/torii/indexer/src/processors/store_set_record.rs (2 hunks)
  • crates/torii/indexer/src/processors/store_update_member.rs (3 hunks)
  • crates/torii/indexer/src/processors/store_update_record.rs (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • crates/torii/indexer/src/processors/store_set_record.rs
  • crates/torii/indexer/src/processors/store_update_member.rs
  • crates/torii/indexer/src/processors/store_del_record.rs
  • crates/torii/indexer/src/processors/store_update_record.rs
⏰ Context from checks skipped due to timeout of 90000ms (3)
  • GitHub Check: clippy
  • GitHub Check: docs
  • GitHub Check: build
🔇 Additional comments (4)
crates/torii/indexer/src/engine.rs (4)

210-211: Ohayo, sensei! The task prioritization structure looks solid!

The use of BTreeMap for priority-based ordering combined with HashMap for efficient task lookup within each priority level is a well-thought-out design choice.

Also applies to: 222-222


604-667: Ohayo, sensei! Excellent implementation of prioritized concurrent processing!

The implementation achieves a good balance between ordered execution (across priority levels) and concurrency (within priority levels), while maintaining event order within each task.


889-895: ⚠️ Potential issue

Ohayo, sensei! Please add bounds checking for event keys access.

The code accesses event.keys[1] and event.keys[2] without verifying the length of the keys vector, which could lead to panics.

Apply this diff to add bounds checking:

            "StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
+               if event.keys.len() < 3 {
+                   warn!(target: LOG_TARGET, "Insufficient event keys for store operations");
+                   return (2usize, 0);
+               }
                let mut hasher = DefaultHasher::new();
                event.keys[1].hash(&mut hasher);
                event.keys[2].hash(&mut hasher);
                let hash = hasher.finish();
                (2usize, hash) // Priority 2 (lower) for store operations
            }

Likely invalid or redundant comment.


900-911: Ohayo, sensei! Verify task identifier generation logic.

Events with task_identifier=0 bypass the priority queue and are processed immediately. Please verify that this doesn't lead to unintended immediate processing of tasks that should be queued.

Run this script to analyze the task identifier generation:

✅ Verification successful

Ohayo, sensei! Task identifier zero is working as intended!

The task_identifier=0 case is explicitly used for non-parallelized events, while parallelized operations (model registration and store operations) use hash-based identifiers that are guaranteed to be non-zero. This design ensures proper event handling and queuing behavior.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Analyze task identifier generation to ensure zero is not a valid task ID
# Test: Search for potential hash collisions leading to zero task IDs
ast-grep --pattern 'let hash = hasher.finish()'

Length of output: 308


Script:

#!/bin/bash
# Get context around hash generation in engine.rs
rg -B 5 -A 5 "let hash = hasher\.finish\(\)" crates/torii/indexer/src/engine.rs

Length of output: 947


🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
crates/torii/indexer/src/engine.rs (2)

881-896: Document priority levels and consider using constants, sensei!

The task identifier generation logic cleverly uses bit manipulation for priorities, but could benefit from better documentation and constant definitions:

+ // Priority levels for different event types
+ const PRIORITY_MODEL_EVENT: u64 = 0;
+ const PRIORITY_STORE_OPS: u64 = 2;
+ // Mask to clear priority bits
+ const PRIORITY_MASK: u64 = 0x00FFFFFFFFFFFFFF;
+ 
  match processor.event_key().as_str() {
      "ModelRegistered" | "EventRegistered" => {
          let mut hasher = DefaultHasher::new();
          event.keys[1].hash(&mut hasher);
-         // Priority 0 (highest) for model/event registration
-         let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
-         hash | (0u64 << 56)
+         // Use highest 8 bits for priority, lower 56 bits for hash-based grouping
+         let hash = hasher.finish() & PRIORITY_MASK;
+         hash | (PRIORITY_MODEL_EVENT << 56)
      }

601-657: Enhance error handling with more context, sensei!

The task processing implementation looks solid, but could benefit from improved error handling. Consider adding more context to errors and enhancing debug logging:

  if let Err(e) = processor
      .process(
          &world,
          &mut local_db,
          event.block_number,
          event.block_timestamp,
          &event.event_id,
          &event.event,
          &event_processor_config,
      )
      .await
  {
      error!(
          target: LOG_TARGET,
          event_name = processor.event_key(),
+         block_number = %event.block_number,
+         timestamp = %event.block_timestamp,
          error = %e,
          task_id = %task_id,
+         "Failed to process event. This might impact dependent operations.",
-         "Processing parallelized event."
      );
+     // Propagate error with additional context
+     return Err(anyhow::anyhow!("Failed to process event {}: {}", event.event_id, e));
  }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a8af3d6 and 9db6ad0.

📒 Files selected for processing (1)
  • crates/torii/indexer/src/engine.rs (4 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: fmt
🔇 Additional comments (1)
crates/torii/indexer/src/engine.rs (1)

219-219: Ohayo! Nice choice using BTreeMap for ordered task processing, sensei!

The switch from HashMap to BTreeMap enables natural priority ordering of tasks, which is crucial for maintaining the correct processing sequence of model registrations and store operations.

Also applies to: 253-253

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
crates/torii/indexer/src/engine.rs (2)

256-256: Ohayo, sensei! Confirm consistent initialization of tasks.

Initializing tasks with BTreeMap::new() is appropriate. However, if there are default priorities or initial tasks, consider initializing them here for clarity.


603-666: Ohayo, sensei! Potential task starvation due to sequential priority processing.

Processing tasks strictly sequentially by priority level may cause lower-priority tasks to be delayed indefinitely if higher-priority tasks are continuously added. Consider implementing a more dynamic scheduling strategy to balance task priorities and prevent starvation.

Do you want assistance in proposing an enhanced task scheduling mechanism?

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9db6ad0 and 312befa.

📒 Files selected for processing (1)
  • crates/torii/indexer/src/engine.rs (5 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: fmt
🔇 Additional comments (3)
crates/torii/indexer/src/engine.rs (3)

210-212: Ohayo, sensei! Good use of type aliases for clarity.

Introducing TaskPriority and TaskId improves code readability and ensures type safety.


222-222: Ohayo, sensei! Verify the implications of changing tasks to a BTreeMap.

Switching from HashMap to BTreeMap for tasks introduces ordered task management based on priority. Ensure this change aligns with the desired task processing logic and does not introduce unintended side effects.


889-903: ⚠️ Potential issue

Ohayo, sensei! Potential hash collisions in task_identifier generation.

Using truncated hash values for task_identifier may lead to collisions, causing unrelated tasks to share the same identifier. Consider using a more collision-resistant method or including additional unique data to ensure task identifiers are unique.

Please run the following script to check for hash collisions:

Comment on lines 907 to 922
self.tasks
.entry(task_priority)
.or_default()
.entry(task_identifier)
.or_default()
.push((
contract_type,
ParallelizedEvent {
event_id: event_id.to_string(),
event: event.clone(),
block_number,
block_timestamp,
},
));
} else {
// Process non-parallelized events immediately
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Ohayo, sensei! Review handling of task_identifier equal to zero.

When task_identifier is zero, events are processed immediately outside of the priority queue. Ensure that zero cannot be a valid task_identifier that should be queued, or adjust the logic to handle zero appropriately to prevent unintended bypassing of the task queue.

Comment on lines 889 to 903
let (task_priority, task_identifier) = match processor.event_key().as_str() {
"ModelRegistered" | "EventRegistered" => {
let mut hasher = DefaultHasher::new();
event.keys.iter().for_each(|k| k.hash(&mut hasher));
let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
(0usize, hash) // Priority 0 (highest) for model/event registration
}
"StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
let mut hasher = DefaultHasher::new();
// model selector
event.keys[1].hash(&mut hasher);
// entity id
event.keys[2].hash(&mut hasher);
hasher.finish()
let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
(2usize, hash) // Priority 2 (lower) for store operations
}
_ => 0,
_ => (0, 0) // No parallelization for other events
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Ohayo, sensei! Guard against index out-of-bounds when accessing event.keys.

Accessing event.keys[1] and event.keys[2] without checking their existence may lead to panics if the keys vector is shorter than expected. Ensure that the event.keys vector has the required length before accessing these indices.

Apply this diff to add necessary checks:

+ if event.keys.len() >= 3 {
    let mut hasher = DefaultHasher::new();
    event.keys[1].hash(&mut hasher);
    event.keys[2].hash(&mut hasher);
    let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
    (2usize, hash) // Priority 2 (lower) for store operations
+ } else {
+     warn!(target: LOG_TARGET, "Insufficient event keys for hashing. Using default task identifier.");
+     (2usize, 0) // Handle with default or error as appropriate
+ }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let (task_priority, task_identifier) = match processor.event_key().as_str() {
"ModelRegistered" | "EventRegistered" => {
let mut hasher = DefaultHasher::new();
event.keys.iter().for_each(|k| k.hash(&mut hasher));
let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
(0usize, hash) // Priority 0 (highest) for model/event registration
}
"StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
let mut hasher = DefaultHasher::new();
// model selector
event.keys[1].hash(&mut hasher);
// entity id
event.keys[2].hash(&mut hasher);
hasher.finish()
let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
(2usize, hash) // Priority 2 (lower) for store operations
}
_ => 0,
_ => (0, 0) // No parallelization for other events
let (task_priority, task_identifier) = match processor.event_key().as_str() {
"ModelRegistered" | "EventRegistered" => {
let mut hasher = DefaultHasher::new();
event.keys.iter().for_each(|k| k.hash(&mut hasher));
let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
(0usize, hash) // Priority 0 (highest) for model/event registration
}
"StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
if event.keys.len() >= 3 {
let mut hasher = DefaultHasher::new();
event.keys[1].hash(&mut hasher);
event.keys[2].hash(&mut hasher);
let hash = hasher.finish() & 0x00FFFFFFFFFFFFFF;
(2usize, hash) // Priority 2 (lower) for store operations
} else {
warn!(target: LOG_TARGET, "Insufficient event keys for hashing. Using default task identifier.");
(2usize, 0) // Handle with default or error as appropriate
}
}
_ => (0, 0) // No parallelization for other events

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
crates/torii/indexer/src/engine.rs (1)

895-901: ⚠️ Potential issue

Add safety check for event keys access.

Guard against potential index out-of-bounds when accessing event.keys.

             "StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
+                if event.keys.len() < 3 {
+                    warn!(
+                        target: LOG_TARGET,
+                        "Insufficient event keys for hashing. Using default task identifier."
+                    );
+                    return (2usize, 0);
+                }
                 let mut hasher = DefaultHasher::new();
                 event.keys[1].hash(&mut hasher);
                 event.keys[2].hash(&mut hasher);
                 let hash = hasher.finish();
                 (2usize, hash)
             }
🧰 Tools
🪛 GitHub Actions: ci

[warning] 900-900: Missing comma in match expression

🧹 Nitpick comments (6)
crates/torii/indexer/src/processors/store_del_record.rs (1)

66-72: Excellent error message improvement, but there's a small formatting issue, sensei!

The enhanced error message with hex-formatted selector will make debugging much easier. However, there's a formatting issue to fix.

Fix the extra closing parenthesis on line 68 as flagged by the CI:

-                return Err(anyhow::anyhow!(
-                    "Failed to retrieve model with selector {:#x}: {}",
-                    event.selector,
-                    e
-                ))
+                return Err(anyhow::anyhow!(
+                    "Failed to retrieve model with selector {:#x}: {}",
+                    event.selector,
+                    e
+                ));
🧰 Tools
🪛 GitHub Actions: ci

[warning] 68-68: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/processors/store_update_record.rs (1)

71-75: Ohayo! Small formatting fix needed, sensei!

The CI pipeline indicates an extra closing parenthesis. Please fix the formatting to pass the pipeline checks.

-                return Err(anyhow::anyhow!(
-                    "Failed to retrieve model with selector {:#x}: {}",
-                    event.selector,
-                    e
-                ))
+                return Err(anyhow::anyhow!("Failed to retrieve model with selector {:#x}: {}", event.selector, e))
🧰 Tools
🪛 GitHub Actions: ci

[warning] 72-72: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/processors/store_set_record.rs (1)

67-73: Fix formatting and enhance error context, sensei!

The error handling provides good context, but there are two improvements to consider:

  1. Fix the formatting issue with the extra closing parenthesis.
  2. Consider including the model namespace/name in the error message for better debugging.

Apply this diff:

-            Err(e) => {
-                return Err(anyhow::anyhow!(
-                    "Failed to retrieve model with selector {:#x}: {}",
-                    event.selector,
-                    e
-                ))
-            }
+            Err(e) => return Err(anyhow::anyhow!(
+                "Failed to retrieve model with selector {:#x} (namespace: {}, name: {}): {}",
+                event.selector,
+                model.namespace,
+                model.name,
+                e
+            ))
🧰 Tools
🪛 GitHub Actions: ci

[warning] 69-69: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/processors/store_update_member.rs (1)

64-71: Enhanced error handling with debug logging, sensei!

The addition of debug logging provides better visibility into model absence cases.

Consider using string interpolation for a more idiomatic debug message:

-                debug!(
-                    target: LOG_TARGET,
-                    selector = %model_selector,
-                    "Model does not exist, skipping."
-                );
+                debug!(
+                    target: LOG_TARGET,
+                    "Model with selector {model_selector:#x} does not exist, skipping."
+                );
crates/torii/indexer/src/engine.rs (2)

600-662: Clean up formatting issues.

Please address the following formatting issues:

  • Remove extra whitespace lines at 600, 607, and 616
  • Split the chain method calls at lines 623-625 for better readability
- let processor = processors.iter()
-     .find(|p| p.validate(&event.event))
-     .expect("Must find at least one processor for the event");
+ let processor = processors
+     .iter()
+     .find(|p| p.validate(&event.event))
+     .expect("Must find at least one processor for the event");
🧰 Tools
🪛 GitHub Actions: ci

[warning] 600-600: Extra whitespace line found, formatting issue


[warning] 607-607: Extra whitespace line found, formatting issue


[warning] 616-616: Extra whitespace line found, formatting issue


[warning] 623-625: Chain method calls should be split into multiple lines for better readability


907-920: Format method chain for better readability.

Split the long chain of method calls across multiple lines.

-            self.tasks
-                .entry(task_priority)
-                .or_default()
-                .entry(task_identifier)
-                .or_default()
-                .push((
-                    contract_type,
-                    ParallelizedEvent {
-                        event_id: event_id.to_string(),
-                        event: event.clone(),
-                        block_number,
-                        block_timestamp,
-                    },
-                ));
+            self.tasks
+                .entry(task_priority)
+                .or_default()
+                .entry(task_identifier)
+                .or_default()
+                .push((
+                    contract_type,
+                    ParallelizedEvent {
+                        event_id: event_id.to_string(),
+                        event: event.clone(),
+                        block_number,
+                        block_timestamp,
+                    },
+                ));
🧰 Tools
🪛 GitHub Actions: ci

[warning] 907-917: Long chain of method calls should be formatted across multiple lines

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 312befa and 2ad2772.

📒 Files selected for processing (5)
  • crates/torii/indexer/src/engine.rs (5 hunks)
  • crates/torii/indexer/src/processors/store_del_record.rs (2 hunks)
  • crates/torii/indexer/src/processors/store_set_record.rs (2 hunks)
  • crates/torii/indexer/src/processors/store_update_member.rs (3 hunks)
  • crates/torii/indexer/src/processors/store_update_record.rs (2 hunks)
🧰 Additional context used
🪛 GitHub Actions: ci
crates/torii/indexer/src/processors/store_set_record.rs

[warning] 69-69: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/processors/store_update_member.rs

[warning] 74-74: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/processors/store_del_record.rs

[warning] 68-68: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/processors/store_update_record.rs

[warning] 72-72: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/engine.rs

[warning] 600-600: Extra whitespace line found, formatting issue


[warning] 607-607: Extra whitespace line found, formatting issue


[warning] 616-616: Extra whitespace line found, formatting issue


[warning] 623-625: Chain method calls should be split into multiple lines for better readability


[warning] 900-900: Missing comma in match expression


[warning] 907-917: Long chain of method calls should be formatted across multiple lines

🔇 Additional comments (11)
crates/torii/indexer/src/processors/store_del_record.rs (2)

38-38: Ohayo! Parameter renaming looks good, sensei!

Removing the underscore prefix is appropriate since we're now using the config parameter.


58-65: Nice error handling improvement, sensei!

The additional check for non-empty namespaces prevents silently ignoring errors when no namespaces are configured, making the behavior more explicit and safer.

crates/torii/indexer/src/processors/store_update_record.rs (2)

39-39: Ohayo! Parameter renaming looks good, sensei!

The change from _config to config properly reflects that this parameter is now being utilized in the implementation.


62-76: Excellent error handling improvements, sensei!

The changes enhance error handling in two valuable ways:

  1. The condition !config.namespaces.is_empty() ensures we only skip missing models when explicitly filtering by namespaces
  2. The error message now includes the selector in hex format for better debugging
🧰 Tools
🪛 GitHub Actions: ci

[warning] 72-72: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/processors/store_set_record.rs (2)

39-39: Ohayo! Parameter rename looks good, sensei!

The removal of the underscore prefix is appropriate since the config parameter is now actively used in the error handling logic.


59-66: Ohayo! Nice enhancement to the namespace-specific indexing logic, sensei!

The condition !config.namespaces.is_empty() effectively handles the case where only specific namespaces are being indexed.

crates/torii/indexer/src/processors/store_update_member.rs (3)

10-10: Ohayo! Import changes look good, sensei!

The addition of debug to the tracing import aligns well with the new debug logging capability.


40-40: Parameter rename reflects its usage, sensei!

The rename from _config to config accurately reflects that the parameter is now actively used in the function.


73-77: ⚠️ Potential issue

Fix the formatting issue, sensei!

There's an extra closing parenthesis as indicated by the pipeline failure.

Apply this fix:

-                return Err(anyhow::anyhow!(
-                    "Failed to retrieve model with selector {:#x}: {}",
-                    event.selector,
-                    e
-                ))
+                return Err(anyhow::anyhow!(
+                    "Failed to retrieve model with selector {:#x}: {}",
+                    event.selector,
+                    e
+                ))

Likely invalid or redundant comment.

🧰 Tools
🪛 GitHub Actions: ci

[warning] 74-74: Extra closing parenthesis, formatting issue

crates/torii/indexer/src/engine.rs (2)

Line range hint 210-256: Ohayo, sensei! LGTM - Clean type definitions and struct changes.

The introduction of type aliases and the switch to BTreeMap for tasks improves code organization and enables priority-based processing.


603-666: Ohayo, sensei! Excellent implementation of priority-based task processing.

The sequential processing of priority levels while maintaining concurrency within each level is well-designed.

🧰 Tools
🪛 GitHub Actions: ci

[warning] 607-607: Extra whitespace line found, formatting issue


[warning] 616-616: Extra whitespace line found, formatting issue


[warning] 623-625: Chain method calls should be split into multiple lines for better readability

Comment on lines 889 to 903
let (task_priority, task_identifier) = match processor.event_key().as_str() {
"ModelRegistered" | "EventRegistered" => {
let mut hasher = DefaultHasher::new();
event.keys.iter().for_each(|k| k.hash(&mut hasher));
let hash = hasher.finish();
(0usize, hash) // Priority 0 (highest) for model/event registration
}
"StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
let mut hasher = DefaultHasher::new();
// model selector
event.keys[1].hash(&mut hasher);
// entity id
event.keys[2].hash(&mut hasher);
hasher.finish()
let hash = hasher.finish();
(2usize, hash) // Priority 2 (lower) for store operations
}
_ => 0,
_ => (0, 0) // No parallelization for other events
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix match expression syntax.

Add missing comma in the match expression.

             "StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
                 let mut hasher = DefaultHasher::new();
                 event.keys[1].hash(&mut hasher);
                 event.keys[2].hash(&mut hasher);
                 let hash = hasher.finish();
-                (2usize, hash) // Priority 2 (lower) for store operations
+                (2usize, hash), // Priority 2 (lower) for store operations
             }
             _ => (0, 0) // No parallelization for other events
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let (task_priority, task_identifier) = match processor.event_key().as_str() {
"ModelRegistered" | "EventRegistered" => {
let mut hasher = DefaultHasher::new();
event.keys.iter().for_each(|k| k.hash(&mut hasher));
let hash = hasher.finish();
(0usize, hash) // Priority 0 (highest) for model/event registration
}
"StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
let mut hasher = DefaultHasher::new();
// model selector
event.keys[1].hash(&mut hasher);
// entity id
event.keys[2].hash(&mut hasher);
hasher.finish()
let hash = hasher.finish();
(2usize, hash) // Priority 2 (lower) for store operations
}
_ => 0,
_ => (0, 0) // No parallelization for other events
let (task_priority, task_identifier) = match processor.event_key().as_str() {
"ModelRegistered" | "EventRegistered" => {
let mut hasher = DefaultHasher::new();
event.keys.iter().for_each(|k| k.hash(&mut hasher));
let hash = hasher.finish();
(0usize, hash) // Priority 0 (highest) for model/event registration
}
"StoreSetRecord" | "StoreUpdateRecord" | "StoreUpdateMember" | "StoreDelRecord" => {
let mut hasher = DefaultHasher::new();
event.keys[1].hash(&mut hasher);
event.keys[2].hash(&mut hasher);
let hash = hasher.finish();
(2usize, hash), // Priority 2 (lower) for store operations
}
_ => (0, 0) // No parallelization for other events
🧰 Tools
🪛 GitHub Actions: ci

[warning] 900-900: Missing comma in match expression

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant