fix(connector): Use ConnectorError::InvalidConnectorConfig for an invalid CoinbaseConnectorMeta

[bkontur]

Type of Change

• Bugfix
• New feature
• Enhancement
• Refactoring
• Dependency updates
• Documentation
• CI/CD

Description

I've removed to_connector_meta_from_secret_with_required_field, which uses MissingRequiredField as an error (I also didn't like the unnecessary initialization of the MissingRequiredField before we knew if it would be needed). Instead, I added a TryFrom implementation for CoinbaseConnectorMeta, which uses the requested ConnectorError::InvalidConnectorConfig. Additionally, I included CoinbaseConnectorMeta::try_from in the validate_auth_and_metadata_type as suggested.

Migration

-- The migration is for adding an empty `pricing_type` attribute for `CoinbaseConnectorMeta` where it is missing.
UPDATE merchant_connector_account
SET metadata = jsonb_insert(
    metadata,
    '{pricing_type}',
    '""',
    true
)
WHERE connector_name = 'coinbase'
    AND NOT metadata ? 'pricing_type';

Additional Changes

• This PR modifies the API contract
• This PR modifies the database schema
• This PR modifies application configuration/environment variables

Motivation and Context

This PR addresses the bug from the issue: #2899.

How did you test it?

Run unit test:

cargo test --release  --package router --lib connector::coinbase::transformers::tests::coinbase_payments_request_try_from_works -- --exact

Migration script test

https://onecompiler.com/postgresql/3zz2tpkad

Test script with test data and run migration:

CREATE TABLE merchant_connector_account (
    id SERIAL PRIMARY KEY,
    connector_name VARCHAR(255) NOT NULL
);
ALTER TABLE merchant_connector_account ADD COLUMN metadata JSONB DEFAULT NULL;

-- insert test data
INSERT INTO merchant_connector_account (
        id,
        connector_name,
        metadata
    )
VALUES
  (1, 'coinbase', '{ "pricing_type": "fixed-rate" }'),
  (2, 'coinbase', '{ "whatever_attribute": "abc" }'),
  (3, 'coinbase', '{}');

-- Before
select * from merchant_connector_account;

-- Run migration:
-- The migration is for adding an empty `pricing_type` attribute for `CoinbaseConnectorMeta` where it is missing.
UPDATE merchant_connector_account
SET metadata = jsonb_insert(
    metadata,
    '{pricing_type}',
    '""',
    true
)
WHERE connector_name = 'coinbase'
    AND NOT metadata ? 'pricing_type';
    
-- After
select * from merchant_connector_account;

Output:

Output:

CREATE TABLE
ALTER TABLE
INSERT 0 3
 id | connector_name |            metadata            
----+----------------+--------------------------------
  1 | coinbase       | {"pricing_type": "fixed-rate"}
  2 | coinbase       | {"whatever_attribute": "abc"}
  3 | coinbase       | {}
(3 rows)

UPDATE 2
 id | connector_name |                     metadata                      
----+----------------+---------------------------------------------------
  1 | coinbase       | {"pricing_type": "fixed-rate"}
  2 | coinbase       | {"pricing_type": "", "whatever_attribute": "abc"}
  3 | coinbase       | {"pricing_type": ""}
(3 rows)

Run locally:

Invalid metadata:

Empty metadata:

Correct metadata:

Checklist

• I formatted the code cargo +nightly fmt --all
• I addressed lints thrown by cargo clippy
• I reviewed the submitted code
• I added unit tests for my changes where possible
• I added a CHANGELOG entry if applicable

[srujanchikke]

Please attach screenshots of testing if possible. refer #2746 to know how to test this issue.

[srujanchikke]

This changes will impact previous merchants who are using coinbase, This PR needs data migrations. Could you please refer #2746 description for more info.

[bkontur]

Issue#2899 does not mention any data migrations.
I've checked that #2746, and in that case, the migration involved replacing terminalId with terminal_id due to the removal of #[serde(rename_all = "camelCase")] here.

However, my issue is different. If I understand correctly, this issue is about empty JSON for CoinbaseConnectorMeta or the absence of the pricing_type attribute in the JSON, and the goal is to add validations to prevent storing invalid CoinbaseConnectorMeta.

So, I am not sure about the migration here. If empty JSON or JSON without pricing_type is stored, how should the migration to the correct CoinbaseConnectorMeta be handled? What value should be set for pricing_type?

[srujanchikke]

No worries, We will take this up as these are data migrations. FYI, I was talking about existing merchants, we need to add pricing_type in metadata and set it to empty string. In that case, when merchant tries to make payment connector throws error and they are expected to update the metadata.

[bkontur]

@srujanchikke aha, ok, thank you, make senses now, so I prepared migration script and simple test demonstration that it works, please see PR description.

[bkontur]

Please attach screenshots of testing if possible. refer #2746 to know how to test this issue.

Ok, I will try the next week

[srujanchikke]

Also need to make changes where ever CoinbaseConnectorMeta is used

hyperswitch/crates/router/src/connector/coinbase/transformers.rs

Line 263 in ecf51b5

let connector_meta: CoinbaseConnectorMeta =

let metadata = item.router_data.get_connector_meta()?; let connector_meta: CoinbaseConnectorMeta = metadata.parse_value(“CoinbaseConnectorMeta”) .change_context(errors::ConnectorError::InvalidConnectorConfig { config: “Merchant connector account metadata”, })?;

[srujanchikke]

No worries, We will take this up as these are data migrations. FYI, I was talking about existing merchants, we need to add pricing_type in metadata and set it to empty string. In that case, when merchant tries to make payment connector throws error and they are expected to update the metadata.

[bkontur]

Please attach screenshots of testing if possible. refer #2746 to know how to test this issue.

Ok, I will try the next week

@srujanchikke testing done with local docker compose --file docker-compose-development.yml up -d.
I like the prepared postman stuff :), please, see the description with attached screenshots.

[bkontur]

@srujanchikke I think ,I've addressed all your review comments, added migration script (with simple test for sql) and added screenshots from postman testing, anything else?

[srujanchikke]

Hi @bkontur ,
Apologies for the delay, Given some minor comments, Please resolve it.