Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement ebpf_check_constraints_at_label #729

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Alan-Jowett
Copy link
Contributor

@Alan-Jowett Alan-Jowett commented Oct 14, 2024

Resolves: #728

This pull request exposes a new API "ebpf_check_constraints_at_label" which permits the caller to compare a set of constraints against the model at a specific label. If the model and the constraints differ it will return the two sets of constraints.

Enhancements and New Features:

Test Case Updates:

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Introduced a new function to parse linear constraints and handle pre-invariants.
    • Added support for checking invariants in YAML test cases.
    • Enhanced the RawTestCase structure to include invariants to check.
    • Added new member variables to manage invariants in the TestCase struct.
  • Bug Fixes

    • Improved error handling for invariant checks during verification.
  • Documentation

    • Updated test cases to include new examples for checking concrete states.
  • Chores

    • Added new headers and modified existing structures to accommodate new features.

Copy link

coderabbitai bot commented Oct 14, 2024

Walkthrough

The changes introduce several new functionalities and modifications across multiple files. A new function parse_linear_constraints is added to handle linear constraints, and a new constructor for label_t is introduced to parse string labels. The ebpf_verifier_options_t structure is updated to include a boolean member for storing pre-invariants. Additionally, several enhancements are made to the crab_verifier and YAML parsing functionalities to support the management and verification of invariants in test cases.

Changes

File Change Summary
src/asm_parse.hpp Added function: std::vector<crab::linear_constraint_t> parse_linear_constraints(const std::set<std::string>& constraints, std::vector<crab::interval_t>& numeric_ranges); included <vector>, "crab/interval.hpp", and "crab/linear_constraint.hpp".
src/asm_syntax.hpp Added constructor: explicit label_t(std::string_view string_label) for parsing string labels into from, to, and stack_frame_prefix.
src/config.hpp Updated structure: struct ebpf_verifier_options_t with new member bool store_pre_invariants;.
src/config.cpp Added field: .store_pre_invariants in ebpf_verifier_default_options, initialized to false.
src/crab_verifier.cpp Added variable: static thread_local std::optional<crab::invariant_table_t> saved_pre_invariants; added functions: void save_invariants_if_needed(...), bool ebpf_check_constraints_at_label(...), and std::set<std::string> ebpf_get_invariants_at_label(...); updated ebpf_verifier_clear_thread_local_state.
src/crab_verifier.hpp Added function declarations: bool ebpf_check_constraints_at_label(std::ostream& os, const std::string& label, const std::set<std::string>& constraints) and std::set<std::string> ebpf_get_invariants_at_label(const std::string& label).
src/ebpf_yaml.cpp Modified RawTestCase structure to include std::map<std::string, std::set<std::string>> invariants_to_check; added function: static std::map<std::string, std::set<std::string>> parse_invariants_to_check(...); updated read_case and run_yaml_test_case functions.
src/ebpf_yaml.hpp Added member variable: std::map<std::string, std::set<std::string>> invariants_to_check to TestCase struct.
test-data/add.yaml Added new test cases: check concrete state and check concrete state - negative with preconditions and invariants to check for register states.

Assessment against linked issues

Objective Addressed Explanation
Expose API to compare abstract state to concrete state (728)
Validate pre-invariants against constraints at specified labels (728)

Possibly related PRs

Suggested reviewers

  • elazarg
  • dthaler

🐇 In the meadow, where changes bloom,
New functions sprout, dispelling gloom.
Constraints and labels, all in a row,
With YAML tests, our knowledge will grow!
Hops of progress, a leap so bright,
In the code's embrace, we find delight! 🌼


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 3427faa and 286a914.

📒 Files selected for processing (8)
  • src/asm_parse.hpp (1 hunks)
  • src/asm_syntax.hpp (1 hunks)
  • src/config.hpp (1 hunks)
  • src/crab_verifier.cpp (5 hunks)
  • src/crab_verifier.hpp (1 hunks)
  • src/ebpf_yaml.cpp (6 hunks)
  • src/ebpf_yaml.hpp (1 hunks)
  • test-data/add.yaml (1 hunks)
🧰 Additional context used
📓 Learnings (1)
src/ebpf_yaml.cpp (1)
Learnt from: dthaler
PR: vbpf/ebpf-verifier#721
File: src/ebpf_yaml.cpp:327-329
Timestamp: 2024-10-11T20:51:53.396Z
Learning: In `src/ebpf_yaml.cpp`, avoid suggesting changes to error handling code in the `run_conformance_test_case` function if the code is not modified in the current PR.
🪛 GitHub Check: validate-yaml
test-data/add.yaml

[warning] 196-196:
196:6 [comments] too few spaces before comment


[warning] 200-200:
200:6 [comments] too few spaces before comment


[warning] 204-204:
204:7 [comments] too few spaces before comment


[warning] 196-196:
196:6 [comments] too few spaces before comment


[warning] 200-200:
200:6 [comments] too few spaces before comment


[warning] 204-204:
204:7 [comments] too few spaces before comment

🔇 Additional comments (9)
src/asm_parse.hpp (1)

6-6: LGTM: New header inclusions are appropriate.

The added headers (, "crab/interval.hpp", and "crab/linear_constraint.hpp") are necessary for the new function declaration and align with the PR objectives of implementing constraint checking functionality.

Also applies to: 9-10

src/config.hpp (1)

24-24: Verify integration of the new option

While the addition of store_pre_invariants is correct, we should ensure its proper integration:

  1. Verify that ebpf_verifier_default_options (declared at the end of this file) is updated to include a default value for store_pre_invariants.
  2. Check that thread_local_options is properly initialized with this new option where relevant in the codebase.

To verify the integration, run the following script:

This script will help ensure that the new option is properly integrated into the codebase.

src/crab_verifier.hpp (1)

29-42: LGTM! Function declaration aligns well with PR objectives.

The new function ebpf_check_constraints_at_label is well-declared and documented. It accurately implements the PR objective of allowing users to compare a set of constraints against a model at a specific label. The function signature and documentation provide clear information about its purpose, parameters, and return value.

test-data/add.yaml (3)

182-182: LGTM: Pre-conditions are well-defined.

The pre-conditions for registers r0, r1, and r2 are clearly specified and consistent, initializing them as numbers with a value of 0. This sets up a good starting point for the test case.


184-190: LGTM: Code section is concise and relevant.

The code section contains a series of simple arithmetic operations and register assignments. This is well-suited for testing the eBPF verifier's ability to track concrete register states, which aligns with the PR objectives.


179-219: Overall, the new test case is well-designed and aligns with PR objectives.

The "check concrete state" test case is a valuable addition to the test suite. It effectively demonstrates the ability to check constraints at specific labels and verify concrete state invariants, which aligns perfectly with the PR objectives.

The test case is well-structured with clear pre-conditions, a concise code section, comprehensive invariants-to-check, and thorough post-conditions. The minor formatting and duplication issues mentioned earlier can be easily addressed to further improve the quality of the test case.

Great job on implementing this test case to validate the new ebpf_check_constraints_at_label functionality!

🧰 Tools
🪛 GitHub Check: validate-yaml

[warning] 196-196:
196:6 [comments] too few spaces before comment


[warning] 200-200:
200:6 [comments] too few spaces before comment


[warning] 204-204:
204:7 [comments] too few spaces before comment


[warning] 196-196:
196:6 [comments] too few spaces before comment


[warning] 200-200:
200:6 [comments] too few spaces before comment


[warning] 204-204:
204:7 [comments] too few spaces before comment

src/crab_verifier.cpp (2)

287-287: Validate label string before constructing label_t

The construction of label_t label(label_string); assumes that label_string is a valid label. Consider adding validation to ensure that label_string corresponds to a valid label in the program, and handle any exceptions that may be thrown if the label is invalid.

You can verify whether the label exists by checking against the available labels in the save_pre_invariants. If the label is invalid, provide a clear error message to the user.


305-309: Handle bottom abstract invariant appropriately

If the from_inv (abstract invariant) is bottom, it means the program state at that label is unreachable.

Ensure that this situation is expected in your use case. If reaching a bottom invariant is unexpected, you might need to investigate why the invariant is bottom at this label.

src/ebpf_yaml.cpp (1)

262-262: Verify the necessity of setting 'store_pre_invariants' to true

Setting test_case.options.store_pre_invariants to true unconditionally may have unintended side effects or performance implications. Ensure that this option needs to be enabled for all test cases, or consider making it configurable per test case.

src/asm_parse.hpp Show resolved Hide resolved
src/config.hpp Outdated Show resolved Hide resolved
src/ebpf_yaml.hpp Outdated Show resolved Hide resolved
test-data/add.yaml Outdated Show resolved Hide resolved
test-data/add.yaml Outdated Show resolved Hide resolved
src/crab_verifier.cpp Outdated Show resolved Hide resolved
src/crab_verifier.cpp Outdated Show resolved Hide resolved
src/crab_verifier.cpp Outdated Show resolved Hide resolved
src/ebpf_yaml.cpp Outdated Show resolved Hide resolved
src/ebpf_yaml.cpp Outdated Show resolved Hide resolved
@coveralls
Copy link

coveralls commented Oct 14, 2024

Coverage Status

coverage: 90.551% (+0.2%) from 90.312%
when pulling fc5d74a on Alan-Jowett:issue728
into 3656437 on vbpf:main.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 286a914 and 0f6520e.

📒 Files selected for processing (8)
  • src/asm_parse.hpp (1 hunks)
  • src/asm_syntax.hpp (1 hunks)
  • src/config.hpp (1 hunks)
  • src/crab_verifier.cpp (5 hunks)
  • src/crab_verifier.hpp (1 hunks)
  • src/ebpf_yaml.cpp (6 hunks)
  • src/ebpf_yaml.hpp (1 hunks)
  • test-data/add.yaml (1 hunks)
🧰 Additional context used
🔇 Additional comments (11)
src/asm_parse.hpp (3)

6-6: LGTM: New header inclusions are appropriate.

The added headers (, "crab/interval.hpp", "crab/linear_constraint.hpp") are necessary for the new function declaration and align with the PR objectives.

Also applies to: 9-10


14-21: Excellent addition of function documentation.

The detailed comment above the parse_linear_constraints function provides clear information about its purpose, parameters, and return value. This addition addresses the suggestion from the previous review and enhances code readability and maintainability.


22-23: LGTM: New function declaration is well-structured and aligns with PR objectives.

The parse_linear_constraints function declaration is appropriately designed with efficient parameter passing (const and reference) and a clear return type. The function signature matches the description in the comment and supports the new functionality for constraint checking.

src/config.hpp (1)

24-25: LGTM: New option for storing pre-invariants added with clear documentation

The addition of bool store_pre_invariants; to the ebpf_verifier_options_t structure is appropriate and aligns with the PR objectives. The comment explaining its purpose is clear and concise, which improves code readability and maintainability.

src/ebpf_yaml.hpp (1)

17-17: LGTM! Documentation suggestion addressed.

The new member variable invariants_to_check is well-defined and appropriately documented. The inline comment clearly explains its purpose, addressing the suggestion from the previous review.

This addition aligns with the PR objectives, enabling the checking of constraints at specific labels. The use of std::map<std::string, std::set<std::string>> is a good choice for efficient lookup and prevention of duplicate invariants.

test-data/add.yaml (1)

178-261: Excellent addition of comprehensive test cases.

The two new test cases, "check concrete state" and "check concrete state - negative", are excellent additions to the test suite. They provide thorough coverage of the new functionality for checking constraints at specific labels, including both positive and negative scenarios. These test cases align perfectly with the PR objectives and significantly enhance the overall quality of the test suite.

Great job on improving the test coverage and ensuring the robustness of the new feature!

src/ebpf_yaml.cpp (5)

102-102: LGTM

The addition of the invariants_to_check member to RawTestCase is appropriate and integrates well with the existing code. This allows for the storage of invariants to check from the YAML test cases.


130-142: Function 'parse_invariants_to_check' is correctly implemented

The function properly checks for the existence of 'invariants-to-check' before accessing it, preventing potential exceptions. It correctly parses the invariants and constructs the required map for further processing.


151-151: Assignment of 'invariants_to_check' in 'parse_case'

The field invariants_to_check in RawTestCase is correctly initialized using the parse_invariants_to_check(case_node) function. This ensures that the invariants are properly loaded from the YAML configuration.


223-225: Proper propagation of 'invariants_to_check' in 'read_case'

The invariants_to_check member is correctly assigned from raw_case to the new TestCase, ensuring that the invariants are available for verification during test execution.


266-267: Setting 'store_pre_invariants' to 'true'

Setting test_case.options.store_pre_invariants to true ensures that pre-invariants are stored. This is necessary for checking invariants at specific labels during the test case execution.

src/crab_verifier.hpp Outdated Show resolved Hide resolved
test-data/add.yaml Outdated Show resolved Hide resolved
src/crab_verifier.cpp Show resolved Hide resolved
src/crab_verifier.cpp Show resolved Hide resolved
src/ebpf_yaml.cpp Outdated Show resolved Hide resolved
@Alan-Jowett Alan-Jowett force-pushed the issue728 branch 3 times, most recently from 8a8a8fd to 73a9473 Compare October 14, 2024 22:05
@Alan-Jowett
Copy link
Contributor Author

The PR was rebased. @coderabbitai full review

Copy link

coderabbitai bot commented Oct 14, 2024

@Alan-Jowett: Understood, I will perform a full review of the updated PR.

✅ Actions performed

Full review triggered.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 0f6520e and 8ecfe92.

📒 Files selected for processing (9)
  • src/asm_parse.hpp (1 hunks)
  • src/asm_syntax.hpp (1 hunks)
  • src/config.cpp (1 hunks)
  • src/config.hpp (1 hunks)
  • src/crab_verifier.cpp (5 hunks)
  • src/crab_verifier.hpp (1 hunks)
  • src/ebpf_yaml.cpp (6 hunks)
  • src/ebpf_yaml.hpp (1 hunks)
  • test-data/add.yaml (1 hunks)
🧰 Additional context used
📓 Learnings (2)
src/ebpf_yaml.cpp (1)
Learnt from: dthaler
PR: vbpf/ebpf-verifier#721
File: src/ebpf_yaml.cpp:327-329
Timestamp: 2024-10-11T20:51:53.396Z
Learning: In `src/ebpf_yaml.cpp`, avoid suggesting changes to error handling code in the `run_conformance_test_case` function if the code is not modified in the current PR.
test-data/add.yaml (1)
Learnt from: Alan-Jowett
PR: vbpf/ebpf-verifier#729
File: test-data/add.yaml:220-261
Timestamp: 2024-10-14T21:52:47.957Z
Learning: In `test-data/add.yaml`, error messages in test cases should match the style used by the verifier by including the label number at the beginning of the message, e.g., `"3: message"`.
🔇 Additional comments (8)
src/asm_parse.hpp (2)

6-6: LGTM: New includes are appropriate and well-organized.

The added includes for <vector>, "crab/interval.hpp", and "crab/linear_constraint.hpp" are necessary for the new function declaration. The ordering of includes follows good practices, with standard library headers preceding project-specific ones.

Also applies to: 9-10


14-23: LGTM: Well-documented function declaration aligns with PR objectives.

The new parse_linear_constraints function declaration is well-structured and consistent with the existing code style. The added documentation provides a clear description of the function's purpose, parameters, and return value, which addresses the previous review comments and enhances code readability and maintainability.

src/config.hpp (1)

24-25: LGTM: New option for storing pre-invariants added with clear documentation

The addition of bool store_pre_invariants; to the ebpf_verifier_options_t structure is appropriate and aligns with the PR objectives. The inline comment clearly explains the purpose of this new option, which enhances code readability and maintainability.

src/ebpf_yaml.hpp (1)

17-17: LGTM! Documentation added as suggested.

The new member variable invariants_to_check has been added to the TestCase struct with appropriate documentation. This addition aligns well with the PR objectives, allowing for the storage and checking of invariants at specific labels. The implementation uses suitable data structures (std::map and std::set) for efficient lookup and prevention of duplicate invariants.

The inline comment provides a clear explanation of the member's purpose, addressing the suggestion from the previous review.

test-data/add.yaml (1)

180-261: Overall: Excellent addition of test cases for constraint checking.

The two new test cases, "check concrete state" and "check concrete state - negative", are valuable additions to the test suite. They effectively cover the functionality of checking constraints at specific labels, which aligns perfectly with the PR objectives. The positive and negative test scenarios provide comprehensive coverage for the new ebpf_check_constraints_at_label API function.

These test cases will help ensure the reliability and correctness of the constraint checking feature, contributing to the overall robustness of the eBPF verifier.

src/asm_syntax.hpp (1)

26-47: Well-implemented constructor and appropriate exception handling

The new label_t constructor correctly parses the input std::string_view to initialize from, to, and stack_frame_prefix. Exception handling for std::invalid_argument and std::out_of_range is properly implemented, ensuring robustness against invalid inputs.

src/crab_verifier.cpp (1)

165-166: Confirm thread-local storage usage for save_pre_invariants

The introduction of static thread_local std::optional<crab::invariant_table_t> save_pre_invariants ensures that pre-invariants are stored per thread. If the verifier operates in a multi-threaded environment, this approach is appropriate. However, please verify that all accesses to save_pre_invariants are thread-safe and that its use aligns with the intended concurrency model.

src/ebpf_yaml.cpp (1)

130-142: Function parse_invariants_to_check handles YAML parsing correctly

The parse_invariants_to_check function effectively parses the "invariants-to-check" node from the YAML file and appropriately handles the case when the node is not defined, ensuring robustness against missing data.

src/config.cpp Outdated Show resolved Hide resolved
src/crab_verifier.hpp Outdated Show resolved Hide resolved
src/crab_verifier.cpp Show resolved Hide resolved
src/crab_verifier.cpp Outdated Show resolved Hide resolved
src/crab_verifier.cpp Outdated Show resolved Hide resolved
src/ebpf_yaml.cpp Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 8ecfe92 and 924ad43.

📒 Files selected for processing (9)
  • src/asm_parse.hpp (1 hunks)
  • src/asm_syntax.hpp (1 hunks)
  • src/config.cpp (1 hunks)
  • src/config.hpp (1 hunks)
  • src/crab_verifier.cpp (5 hunks)
  • src/crab_verifier.hpp (1 hunks)
  • src/ebpf_yaml.cpp (6 hunks)
  • src/ebpf_yaml.hpp (1 hunks)
  • test-data/add.yaml (1 hunks)
🧰 Additional context used
📓 Learnings (2)
src/ebpf_yaml.cpp (1)
Learnt from: dthaler
PR: vbpf/ebpf-verifier#721
File: src/ebpf_yaml.cpp:327-329
Timestamp: 2024-10-11T20:51:53.396Z
Learning: In `src/ebpf_yaml.cpp`, avoid suggesting changes to error handling code in the `run_conformance_test_case` function if the code is not modified in the current PR.
test-data/add.yaml (1)
Learnt from: Alan-Jowett
PR: vbpf/ebpf-verifier#729
File: test-data/add.yaml:220-261
Timestamp: 2024-10-14T21:52:47.957Z
Learning: In `test-data/add.yaml`, error messages in test cases should match the style used by the verifier by including the label number at the beginning of the message, e.g., `"3: message"`.
🔇 Additional comments (17)
src/config.cpp (1)

17-17: LGTM! Consider adding documentation for the new option.

The addition of .store_pre_invariants = false is consistent with the PR objectives and provides a sensible default value. This new option enhances the configurability of the verifier, allowing users to opt-in to storing pre-invariants when needed.

As previously suggested, consider adding a comment above this line or in the header file to briefly explain the purpose and usage of this new option. This would improve the maintainability of the code and help future developers understand its significance.

src/asm_parse.hpp (2)

6-6: LGTM: New includes are appropriate and well-ordered.

The addition of these headers is necessary for the new parse_linear_constraints function. The ordering follows the convention of placing standard library headers before project-specific ones.

Also applies to: 9-10


14-23: LGTM: Well-documented and properly declared new function.

The parse_linear_constraints function declaration is well-structured and consistent with the existing code style. The added comment provides a clear description of the function's purpose, parameters, and return value, which enhances code readability and maintainability.

src/config.hpp (1)

24-25: LGTM: Well-documented addition of store_pre_invariants option

The new store_pre_invariants boolean member has been added to the ebpf_verifier_options_t structure with a clear and informative comment. This addition aligns well with the PR objectives and supports the new functionality for storing pre-invariants.

The comment above the new member effectively explains its purpose, addressing the suggestion from the previous review. The naming is consistent with other boolean options in the structure, maintaining good code readability.

src/ebpf_yaml.hpp (1)

17-17: LGTM. Documentation added as suggested.

The new member variable invariants_to_check has been added to the TestCase struct with appropriate documentation. This addition aligns well with the PR objectives and the requirements from the linked issue #728.

The use of std::map<std::string, std::set<std::string>> is a good choice for efficient lookup of invariants by label and prevention of duplicate invariants for each label.

src/crab_verifier.hpp (1)

30-43: LGTM! Documentation enhancement implemented.

The function declaration and documentation for ebpf_check_constraints_at_label are well-defined and align with the PR objectives. The previous suggestion to explain "abstract verifier constraints" has been implemented, improving the clarity of the documentation.

src/crab_verifier.cpp (7)

20-20: Appropriate inclusion of 'asm_parse.hpp' header

The inclusion of #include "asm_parse.hpp" is necessary to expose the function for parsing linear constraints. This ensures that the parsing functionality is accessible within this source file.


165-166: Declaration of thread-local variable 'save_pre_invariants'

The addition of the thread-local variable save_pre_invariants correctly initializes it to std::nullopt. This variable is used to store pre-invariants when the store_pre_invariants option is enabled.


178-180: Conditional storage of pre-invariants in 'get_ebpf_report'

The code appropriately checks if store_pre_invariants is enabled before storing pre_invariants in save_pre_invariants. This ensures that pre-invariants are only stored when required.


230-232: Conditional storage of pre-invariants in 'ebpf_analyze_program_for_test'

Similarly, the function ebpf_analyze_program_for_test now stores pre_invariants in save_pre_invariants when store_pre_invariants is enabled. This maintains consistency across functions that perform analysis.


279-280: Clearing thread-local state in 'ebpf_verifier_clear_thread_local_state'

Resetting save_pre_invariants to std::nullopt ensures that the thread-local state is properly cleared when the verifier's state is reset. This prevents any potential leakage of pre-invariants between verifications.


282-326: Implementation of 'ebpf_check_constraints_at_label' function

The new ebpf_check_constraints_at_label function provides the ability to compare a set of constraints against the stored pre-invariants at a specific label. The function includes detailed error messages to aid in debugging when discrepancies are found.


328-337: Implementation of 'ebpf_get_invariants_at_label' function

The addition of ebpf_get_invariants_at_label allows retrieval of the stored invariants at a given label. The function checks if pre-invariants are available and returns them appropriately.

src/ebpf_yaml.cpp (4)

102-102: Addition of 'invariants_to_check' to 'RawTestCase' Structure

The new member invariants_to_check is appropriately added to the RawTestCase struct to store invariants for testing purposes.


151-151: Initializing 'invariants_to_check' in 'parse_case' Function

The parse_case function correctly initializes the invariants_to_check member by invoking parse_invariants_to_check(case_node).


223-225: Transferring 'invariants_to_check' to 'TestCase'

The read_case function appropriately transfers the invariants_to_check member from RawTestCase to TestCase, ensuring that the invariants are available for test execution.


266-269: Setting 'store_pre_invariants' Based on Invariants to Check

The logic correctly sets test_case.options.store_pre_invariants to true when test_case.invariants_to_check is not empty, enabling the storage of pre-invariants necessary for invariant checks.

src/ebpf_yaml.hpp Show resolved Hide resolved
src/crab_verifier.hpp Show resolved Hide resolved
test-data/add.yaml Outdated Show resolved Hide resolved
test-data/add.yaml Outdated Show resolved Hide resolved
src/asm_syntax.hpp Show resolved Hide resolved
src/ebpf_yaml.cpp Outdated Show resolved Hide resolved
src/ebpf_yaml.cpp Outdated Show resolved Hide resolved
Signed-off-by: Alan Jowett <[email protected]>
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 924ad43 and fc5d74a.

📒 Files selected for processing (5)
  • src/config.cpp (1 hunks)
  • src/crab_verifier.cpp (5 hunks)
  • src/crab_verifier.hpp (1 hunks)
  • src/ebpf_yaml.cpp (6 hunks)
  • test-data/add.yaml (1 hunks)
🧰 Additional context used
📓 Learnings (3)
src/crab_verifier.cpp (1)
Learnt from: Alan-Jowett
PR: vbpf/ebpf-verifier#729
File: src/crab_verifier.cpp:323-326
Timestamp: 2024-10-16T04:59:11.616Z
Learning: In `src/crab_verifier.cpp`, all exceptions thrown in the function `ebpf_check_constraints_at_label` derive from `std::exception`, making a catch-all handler unnecessary.
src/crab_verifier.hpp (1)
Learnt from: Alan-Jowett
PR: vbpf/ebpf-verifier#729
File: src/crab_verifier.hpp:44-51
Timestamp: 2024-10-16T05:00:24.900Z
Learning: In `src/crab_verifier.hpp`, adding usage examples to the documentation of `ebpf_get_invariants_at_label` is not necessary.
src/ebpf_yaml.cpp (1)
Learnt from: Alan-Jowett
PR: vbpf/ebpf-verifier#729
File: src/ebpf_yaml.hpp:17-17
Timestamp: 2024-10-16T04:59:56.598Z
Learning: Remember that the `invariants_to_check` member in `TestCase` is properly populated and used in the YAML parsing logic in `src/ebpf_yaml.cpp`.
🪛 GitHub Check: validate-yaml
test-data/add.yaml

[warning] 192-192:
192:6 [comments] too few spaces before comment


[warning] 196-196:
196:6 [comments] too few spaces before comment


[warning] 200-200:
200:6 [comments] too few spaces before comment


[warning] 204-204:
204:7 [comments] too few spaces before comment


[warning] 232-232:
232:6 [comments] too few spaces before comment


[warning] 236-236:
236:6 [comments] too few spaces before comment


[warning] 240-240:
240:6 [comments] too few spaces before comment


[warning] 244-244:
244:7 [comments] too few spaces before comment


[warning] 192-192:
192:6 [comments] too few spaces before comment


[warning] 196-196:
196:6 [comments] too few spaces before comment

🔇 Additional comments (13)
src/config.cpp (1)

17-17: LGTM! The new option is well-implemented and documented.

The addition of .store_pre_invariants = false with an explanatory comment is consistent with the PR objectives. It provides a sensible default value and clear guidance on its usage, enhancing the configurability of the verifier while maintaining backward compatibility.

src/crab_verifier.hpp (3)

30-45: LGTM! Function aligns well with PR objectives.

The ebpf_check_constraints_at_label function declaration and its documentation are well-defined and directly address the PR objectives. The function provides a way to compare concrete constraints against abstract verifier constraints at a specific label, which is a key requirement outlined in the linked issue #728.

The documentation is comprehensive and includes a clear explanation of abstract constraints, which enhances usability. The function signature accurately reflects its purpose and the described behavior.


46-53: LGTM! Function complements the PR objectives well.

The ebpf_get_invariants_at_label function declaration and its documentation are well-defined and support the PR objectives. This function provides a mechanism to retrieve invariants at a specific label, which is crucial for comparing abstract and concrete states as outlined in the linked issue #728.

The documentation clearly states the function's purpose and requirements, and the exception handling is properly documented. The function signature accurately reflects its intended behavior.


29-53: Overall, excellent additions to the codebase.

The new functions ebpf_check_constraints_at_label and ebpf_get_invariants_at_label are well-implemented and documented. They directly address the requirements outlined in the linked issue #728 and align perfectly with the PR objectives. These additions will significantly enhance the ability to compare abstract and concrete states in the BPF verification process.

The code is clean, well-structured, and maintains the existing coding style of the file. Great job on these implementations!

test-data/add.yaml (2)

180-219: LGTM: Well-structured test case for concrete state checking.

The "check concrete state" test case is well-designed and aligns perfectly with the PR objective of checking constraints at specific labels. The pre-conditions, code execution, invariants-to-check, and post-conditions are all correctly defined and provide a thorough validation of the register states throughout the execution.

🧰 Tools
🪛 GitHub Check: validate-yaml

[warning] 192-192:
192:6 [comments] too few spaces before comment


[warning] 196-196:
196:6 [comments] too few spaces before comment


[warning] 200-200:
200:6 [comments] too few spaces before comment


[warning] 204-204:
204:7 [comments] too few spaces before comment


[warning] 192-192:
192:6 [comments] too few spaces before comment


[warning] 196-196:
196:6 [comments] too few spaces before comment


220-261: LGTM: Well-designed negative test case for constraint checking.

The "check concrete state - negative" test case is well-structured and serves as an excellent negative test for the constraint checking functionality. It intentionally includes an incorrect invariant to trigger a mismatch between concrete and abstract invariants, which aligns with the PR objectives.

The error message format correctly includes the label number at the beginning, which is consistent with the verifier's error reporting style.

🧰 Tools
🪛 GitHub Check: validate-yaml

[warning] 232-232:
232:6 [comments] too few spaces before comment


[warning] 236-236:
236:6 [comments] too few spaces before comment


[warning] 240-240:
240:6 [comments] too few spaces before comment


[warning] 244-244:
244:7 [comments] too few spaces before comment

src/crab_verifier.cpp (6)

165-172: Modularize invariant saving logic for maintainability

The addition of save_invariants_if_needed improves code maintainability by encapsulating the logic for saving pre-invariants. This adheres to the DRY principle and makes future changes easier.


184-184: Ensure pre-invariants are saved consistently

By calling save_invariants_if_needed(pre_invariants);, the function ensures that pre-invariants are stored when necessary. This is crucial for functions that rely on these invariants later in the verification process.


234-234: Consistent use of invariant saving function

The invocation of save_invariants_if_needed(pre_invariants); here ensures that pre-invariants are consistently saved across different analysis functions, promoting code consistency.


281-282: Properly clear thread-local state

Resetting saved_pre_invariants to std::nullopt in ebpf_verifier_clear_thread_local_state ensures that the thread-local state is clean for subsequent analyses. This prevents potential issues from residual data.


284-328: Robust exception handling during constraint checking

The ebpf_check_constraints_at_label function is wrapped in a try-catch block to handle exceptions derived from std::exception. This ensures that any errors during constraint checking are caught and reported gracefully.


330-341: Intentional exception propagation for label parsing

In ebpf_get_invariants_at_label, constructing label_t l = label_t(label); may throw an exception if the label is invalid. The comment indicates that this is intentional so that the caller can handle malformed labels appropriately.

src/ebpf_yaml.cpp (1)

266-269: Ensure side effects of setting 'store_pre_invariants' are intended

Setting test_case.options.store_pre_invariants to true when invariants_to_check is not empty allows pre-invariants to be stored for later checks. Verify that enabling this option does not unintentionally affect other parts of the verifier or introduce performance overhead.

To assess any potential side effects, consider reviewing where store_pre_invariants is used in the codebase:

✅ Verification successful

Side Effects of Setting store_pre_invariants Confirmed

After reviewing all usages of store_pre_invariants, enabling this option is intentional and does not introduce unintended side effects or significant performance overhead. The flag is used in a controlled manner within the verifier, ensuring that activating it supports necessary functionalities without negatively impacting other parts of the system.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Find all usages of 'store_pre_invariants' in the codebase.

# Expected Result: Identify code that relies on 'store_pre_invariants' to ensure enabling it is safe.
rg --type cpp 'store_pre_invariants'

Length of output: 832

test-data/add.yaml Show resolved Hide resolved
src/ebpf_yaml.cpp Show resolved Hide resolved
src/ebpf_yaml.cpp Show resolved Hide resolved
@Alan-Jowett
Copy link
Contributor Author

@elazarg is this the best option for allowing the caller to check the current concrete state of the vm against the abstract state the verifier has?

I had considered just passing in the register values, but that doesn't really work well as there is additional context that the VM has that can be used to filter the assertions (things like if the register value is pointing to context, packet, shared, or stack memory) that the verifier would otherwise not know. This becomes an issue as the concrete values for pointers tend to be > 2GB which trips some assertions in the verifier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose API to compare abstract state to concrete state
2 participants