fix:improve cache #40
Conversation
cache domain intents in their own folder to avoid retraining when config changes improve error handling
|
Warning Rate limit exceeded@JarbasAl has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 7 minutes and 24 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
WalkthroughThe pull request introduces modifications to the Padatious intent matching system, focusing on improving intent caching, stemming functionality, and training management. The changes enhance the flexibility of intent processing by adding a Changes
Sequence DiagramsequenceDiagram
participant PM as PadatiousPipeline
participant TM as TrainingManager
participant IC as IntentContainer
PM->>PM: Configure use_stemmer
PM->>IC: Initialize with pre-loaded cache
TM->>TM: Add training objects
TM->>TM: Log training details
TM->>IC: Train objects
TM->>TM: Track failed training attempts
Possibly related PRs
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
ovos_padatious/training_manager.py (1)
156-163: Consider adding a retry mechanism with backoff.While the error handling is good, implementing a retry mechanism with exponential backoff could help recover from temporary failures.
from time import sleep from random import uniform def retry_with_backoff(func, max_retries=3): """Retry a function with exponential backoff.""" for i in range(max_retries): try: return func() except Exception as e: if i == max_retries - 1: raise sleep_time = (2 ** i) + uniform(0, 1) sleep(sleep_time)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
ovos_padatious/opm.py(1 hunks)ovos_padatious/training_manager.py(2 hunks)ovos_padatious/util.py(1 hunks)
🔇 Additional comments (5)
ovos_padatious/training_manager.py (3)
88-91: LGTM! Enhanced logging improves debugging capabilities.The added debug logs provide better visibility into training decisions, clearly indicating first-time training scenarios and hash changes.
145-146: LGTM! Thread safety improvement.Creating a copy of
objects_to_trainprevents potential concurrent modification issues during training.
148-153: LGTM! Robust error handling for training failures.The try-except block gracefully handles training failures, logs errors, and maintains failed objects for potential retry, improving system resilience.
ovos_padatious/opm.py (1)
277-286: LGTM! Well-organized cache directory structure.The cache directory organization cleanly separates different configurations (domain vs. regular, stemmer vs. non-stemmer), preventing cache conflicts.
ovos_padatious/util.py (1)
31-31: LGTM! Improved hash consistency.Sorting lines before hashing ensures consistent cache keys regardless of line order, preventing unnecessary retraining.
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
ovos_padatious/opm.py (1)
Line range hint
297-301: Add debug logging for stemmer initialization.Consider adding debug logging to indicate when stemming is enabled/disabled and which languages have stemmer support.
if use_stemmer: + LOG.debug(f"Initializing stemmers with stemming enabled") self.stemmers = {lang: Stemmer(lang) for lang in langs if Stemmer.supports_lang(lang)} + LOG.debug(f"Initialized stemmers for languages: {list(self.stemmers.keys())}") else: + LOG.debug("Stemming disabled, skipping stemmer initialization") self.stemmers = {}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
ovos_padatious/opm.py(1 hunks)
🔇 Additional comments (2)
ovos_padatious/opm.py (2)
277-286: LGTM! Improved caching mechanism with better organization.The changes enhance the caching system by:
- Separating domain intents into their own cache folder with
_domainsuffix- Adding a stemmer-specific cache folder with
_stemmersuffix when stemming is enabledThis prevents unnecessary retraining when switching between configurations, improving efficiency.
290-295: LGTM! Implemented error handling for pre-loading cached intents.The implementation matches the previously suggested error handling improvement, ensuring graceful handling of failures during pre-loading of cached intents.
cache domain intents in their own folder to avoid retraining when config changes improve error handling
cache domain intents in their own folder to avoid retraining when config changes
improve error handling
Summary by CodeRabbit
New Features
Bug Fixes
Chores