Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sqlite schema #6648

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Conversation

arzonus
Copy link
Contributor

@arzonus arzonus commented Jan 28, 2025

What changed?
sqlite schema has been added. 99.9% is a mysql schema. There is a few differences:

  • All versioned schemas were removed due to not having sqlite plugin before. Only 0.1 schemas are left, but they were replaced by schemas described in cadence/schema.sql and visibility/schema.sql, so they're identical.
  • domain_metadata table's id type has been changed to INTEGER. SQLite doesn't support AUTOINCREMENT for BIGINT.

sqlite plugin has been registered, so should be appeared in all services. However, the methods are not tested yet, it will be done in future. Without adding, schema setup wouldn't have been able to be tested.

Why?
SQLite storage will be used in the future as an in-memory replacement for persistence storage. The diff is a part of the project In-Memory Storage.

Previous PRs:

How did you test it?
Unit tests are passed.

There is a new sqlite package in tools/sql. The package contains only one test SetupSchema that calls SetupFromConfig func to setup schema to in-memory running sqlite db. It tests that all databases schemas (cadence and visibility) can be setup.

Unfortunately, I couldn't reuse tests from clitest package that were supposed to be used for this case. These tests are subject to be refactored as they were created only for mysql.

At the same time, I couldn't reuse tools/common/schema/test/ package, as it also calls sql-tool that requires to having config.SQL filled as it's done for Postgres or MySQL. If we need to support SQLite, we will need to refactor the package. Also, the tests don't support other SQL plugins than MySQL, so it also needs to be added.

I decided to not focus on this problem, as most of these test cases covering schema updating that it is not critical for SQLite, as it will be used as in-memory storage (or with temporary database files).

We should focus on this problem later, when we will work on the Local Cadence project.

Potential risks

Release notes

Documentation Changes

Detailed Description
No need, as only there are no break changes.

Impact Analysis

  • Backward Compatibility: [Analysis of backward compatibility]
  • Forward Compatibility: [Analysis of forward compatibility]

Testing Plan

  • Unit Tests: [Do we have unit test covering the change?]
  • Persistence Tests: [If the change is related to a data type which is persisted, do we have persistence tests covering the change?]
  • Integration Tests: [Do we have integration test covering the change?]
  • Compatibility Tests: [Have we done tests to test the backward and forward compatibility?]

Rollout Plan

  • What is the rollout plan?
  • Does the order of deployment matter?
  • Is it safe to rollback? Does the order of rollback matter?
  • Is there a kill switch to mitigate the impact immediately?

@arzonus arzonus force-pushed the add-sqlite-schema branch 2 times, most recently from d421186 to 2b79327 Compare February 6, 2025 08:57
@arzonus arzonus changed the title [WIP] Add sqlite schema Add sqlite schema Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant