Separate query issues for unsupported datatypes and PK/UK on complex datatypes #2361

ShivanshGahlot · 2025-02-25T18:46:08Z

Describe the changes in this pull request

Added separate issues for all the different unsupported datatypes in Query Issues. Earlier they were all reported under the same issue. Doing this so that all the datatypes have separate items in the reports and callhome payload.
Added separate issues for PK and UK on all the different unsupported data types in Query Issues. Same reason as above.
Fixes: In assess-migration/analyze-schema payload, create separate items for "unsupported datatypes" #2252, Define separate Issue for each case related to datatype(unsupported datatype, index on complex datatype) #2231

Describe if there are any user-facing changes

Unsupported datatypes analyze schema report:

Callhome data for unsupported datatypes:

Schema Report for UK on complex datatypes:

Schema Report for PK on complex datatypes:

Callhome data for PK on complex datatypes:

How was this pull request tested?

Manually

Does your PR have changes that can cause upgrade issues?

Component	Breaking changes?
MetaDB	No
Name registry json	No
Data File Descriptor Json	No
Export Snapshot Status Json	No
Import Data State	No
Export Status Json	No
Data .sql files of tables	No
Export and import data queue	No
Schema Dump	No
AssessmentDB	No
Sizing DB	No
Migration Assessment Report Json	No
Callhome Json	No
YugabyteD Tables	No
TargetDB Metadata Tables	No

…ssues in analyze schema

…ration using queryissues in analyze schema

…ration during FF and FB using queryissues in analyze schema

…ind Migration Caveats issues

priyanshi-yb · 2025-03-06T13:05:35Z

yb-voyager/cmd/analyzeSchema.go

+	queryissue.POINT_DATATYPE,
+	queryissue.LINE_DATATYPE,
+	queryissue.LSEG_DATATYPE,
+	queryissue.BOX_DATATYPE,
+	queryissue.PATH_DATATYPE,
+	queryissue.POLYGON_DATATYPE,
+	queryissue.CIRCLE_DATATYPE,


lets have a list of these in queryissue like unsupportedDatatypeIssuesInLiveMigration and unsupportedDatatypeIssuesInLiveMigrationWithFFOrFB so that we these list in a single place

priyanshi-yb · 2025-03-06T13:06:50Z

yb-voyager/cmd/analyzeSchema.go

+		queryissue.PK_UK_ON_TXID_SNAPSHOT_DATATYPE,
+		queryissue.PK_UK_ON_PGLSN_DATATYPE,
+		queryissue.PK_UK_ON_ARRAY_DATATYPE,
+		queryissue.PK_UK_ON_USER_DEFINED_DATATYPE,


Similarly for these have a list pkOrUkOnComplexDatatypesIssues

priyanshi-yb · 2025-03-06T13:08:48Z

yb-voyager/cmd/assessMigrationCommand.go

+	log.Infof("fetching unsupported features for PK/UK on complex datatypes...")
+	unsupportedFeatures := make([]UnsupportedFeature, 0)
+
+	unsupportedFeatures = append(unsupportedFeatures, getUnsupportedFeaturesFromSchemaAnalysisReport(queryissue.PK_UK_ON_CITEXT_DATATYPE_ISSUE_NAME, "", queryissue.PK_UK_ON_CITEXT_DATATYPE, schemaAnalysisReport, false))


once you have that list, you can use that list here directly to loop over and reduce the number of lines here.

priyanshi-yb · 2025-03-06T13:22:06Z

yb-voyager/src/query/queryissue/detectors_ddl.go

@@ -222,24 +225,299 @@ func (d *TableIssueDetector) DetectIssues(obj queryparser.DDLObject) ([]QueryIss
 	return issues, nil
 }

+func reportPKOrUniqueConstraintOnUnsupportedDatatypesIssue(objType string, objName string, typeName string, constraintName string, issues *[]QueryIssue) {
+	switch typeName {


can you see if we can merge this function with already existing reportIndexOnComplexDatatypes function as the types are same in both the cases?

priyanshi-yb

LGTM with some comments.

priyanshi-yb · 2025-03-07T13:02:43Z

yb-voyager/src/query/queryissue/detectors_ddl.go

@@ -410,183 +612,473 @@ func (d *IndexIssueDetector) DetectIssues(obj queryparser.DDLObject) ([]QueryIss
 	return issues, nil
 }

-func reportIndexOnComplexDatatypesIssue(objType string, objName string, typeName string) QueryIssue {
+func reportIndexOnComplexDatatypesIssue(objType string, objName string, typeName string, isPkorUk bool, constraintName string) QueryIssue {


Change function name to reportIndexOrConstraintIssuesOnComplexDatatypes

priyanshi-yb · 2025-03-07T13:11:59Z

yb-voyager/cmd/analyzeSchema.go

@@ -623,18 +623,71 @@ var MigrationCaveatsIssues = []string{
 	queryissue.ALTER_TABLE_ADD_PK_ON_PARTITIONED_TABLE,
 	queryissue.FOREIGN_TABLE,
 	queryissue.POLICY_WITH_ROLES,
-	queryissue.UNSUPPORTED_DATATYPE_LIVE_MIGRATION,
-	queryissue.UNSUPPORTED_DATATYPE_LIVE_MIGRATION_WITH_FF_FB,
+}


Can you put this migrationCaveatsIssues inside the convertIssueInstanceToAnalyzeIssue , to avoid any usage of it outside that function as its not complete now.

priyanshi-yb · 2025-03-07T13:12:21Z

yb-voyager/cmd/analyzeSchema.go

-	queryissue.UNSUPPORTED_DATATYPE_LIVE_MIGRATION_WITH_FF_FB,
+}
+
+var UnsupportedDatatypesInLiveMigrationIssues = []string{


put these lists in the queryissue package.

priyanshi-yb · 2025-03-07T13:16:22Z

yb-voyager/src/query/queryissue/detectors_ddl.go

+		if !isPkorUk {
+			queryIssue = NewIndexOnCitextDatatypeIssue(
+				objType,
+				objName,
+				"",
+			)
+		} else {
+			queryIssue = NewPrimaryOrUniqueConstraintOnCitextDatatypeIssue(
+				objType,
+				objName,
+				"",
+				typeName,
+				constraintName,
+			)


you can use lo.Ternary for this.

queryissue = lo.Ternary(isPkorUk, NewPrimaryOrUniqueConstraintOnCitextDatatypeIssue(objType,objName,"",typeName,constraintName),NewIndexOnCitextDatatypeIssue(objType,objName,""))

ShivanshGahlot added 13 commits March 4, 2025 12:16

Reporting individual issues for pg unuspported datatypes using queryi…

9ff632f

…ssues in analyze schema

Reporting individual issues for pg unuspported datatypes for live mig…

c94ec0d

…ration using queryissues in analyze schema

Reporting individual issues for pg unuspported datatypes for live mig…

62db777

…ration during FF and FB using queryissues in analyze schema

Fixing tests

029f1a7

Fixing tests

3eddd68

Fixing tests

b8aed40

Fixed assessment test using string.EqualFold instead of Contains to f…

cf1198b

…ind Migration Caveats issues

Fixing analyze schema tests

ec4a024

Fixing analyze schema tests

209c791

Created separate issues for all the complex datatypes with PK or UK

901690e

Removed comments and debugging lines

ded3ff6

Fixing tests

ac1c661

Fixing tests

377c8cc

ShivanshGahlot requested review from priyanshi-yb, makalaaneesh and sanyamsinghal March 4, 2025 12:29

ShivanshGahlot changed the title ~~Shivansh/separate issues for datatypes~~ Separate query issues for unsupported datatypes and PK/UK on complex datatypes Mar 4, 2025

ShivanshGahlot marked this pull request as ready for review March 4, 2025 12:29

Fixing HTML report tests

44b4ef6

ShivanshGahlot force-pushed the shivansh/separate-issues-for-datatypes branch from 1921abc to 44b4ef6 Compare March 5, 2025 12:03

ShivanshGahlot added 3 commits March 5, 2025 13:27

Adding tests for PK adn UK on complex in issues ddl tests

9569092

Adding tests for PK adn UK on complex in issues ddl tests

d397165

Adding tests for PK adn UK on complex in issues ddl tests

c5f0a60

priyanshi-yb reviewed Mar 6, 2025

View reviewed changes

ShivanshGahlot added 2 commits March 7, 2025 11:03

Added suggested changes

67fd55b

Added suggested changes

97f0471

ShivanshGahlot requested a review from priyanshi-yb March 7, 2025 11:30

priyanshi-yb approved these changes Mar 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Separate query issues for unsupported datatypes and PK/UK on complex datatypes #2361

Separate query issues for unsupported datatypes and PK/UK on complex datatypes #2361

ShivanshGahlot commented Feb 25, 2025 •

edited

Loading

priyanshi-yb Mar 6, 2025

ShivanshGahlot Mar 7, 2025

priyanshi-yb Mar 6, 2025

ShivanshGahlot Mar 7, 2025

priyanshi-yb Mar 6, 2025

ShivanshGahlot Mar 7, 2025

priyanshi-yb Mar 6, 2025

ShivanshGahlot Mar 7, 2025

priyanshi-yb left a comment

priyanshi-yb Mar 7, 2025

priyanshi-yb Mar 7, 2025

priyanshi-yb Mar 7, 2025

priyanshi-yb Mar 7, 2025 •

edited

Loading

Separate query issues for unsupported datatypes and PK/UK on complex datatypes #2361

Are you sure you want to change the base?

Separate query issues for unsupported datatypes and PK/UK on complex datatypes #2361

Conversation

ShivanshGahlot commented Feb 25, 2025 • edited Loading

Describe the changes in this pull request

Describe if there are any user-facing changes

How was this pull request tested?

Does your PR have changes that can cause upgrade issues?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

priyanshi-yb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

priyanshi-yb Mar 7, 2025 • edited Loading

Choose a reason for hiding this comment

ShivanshGahlot commented Feb 25, 2025 •

edited

Loading

priyanshi-yb Mar 7, 2025 •

edited

Loading