Reporting a separate issue for all the different unsupported datatypes for the issue index on complex datatypes in schema report and assessment report #2288

ShivanshGahlot · 2025-02-04T12:16:29Z

Describe the changes in this pull request

Earlier indexes on all the complex datatypes were reported under a single issue type: INDEX ON COMPLEX DATATYPES
Added a separate issue for all such complex datatypes and removed the original issue.
Reporting the issue in both the analyze schema report and the assessment report.
Fixed issue: Define separate Issue for each case related to datatype(unsupported datatype, index on complex datatype) #2231

Describe if there are any user-facing changes

Assessment Report will now have separate issues for all the complex datatypes:

Schema analysis report will have separate issues for all the complex datatypes:

How was this pull request tested?

Manually

Does your PR have changes that can cause upgrade issues?

Component	Breaking changes?
MetaDB	No
Name registry json	No
Data File Descriptor Json	No
Export Snapshot Status Json	No
Import Data State	No
Export Status Json	No
Data .sql files of tables	No
Export and import data queue	No
Schema Dump	No
AssessmentDB	No
Sizing DB	No
Migration Assessment Report Json	No
Callhome Json	No
YugabyteD Tables	No
TargetDB Metadata Tables	No

…ypes

makalaaneesh · 2025-02-19T05:47:10Z

yb-voyager/cmd/analyzeSchema.go

@@ -1241,6 +1241,59 @@ var includeObjectNameInCallhomePayloadForIssueTypes = []string{
 	UNSUPPORTED_EXTENSION_ISSUE_TYPE,
 }

+// analyze issue reasons to modify the reason before sending to callhome as will have sensitive information


I don't think we're using this anywhere...
@sanyamsinghal to confirm.

Yeah seems like a miss during merge from my side. Removed it now

makalaaneesh · 2025-02-19T05:47:31Z

yb-voyager/cmd/analyzeSchema.go

+}
+
+// analyze issue reasons to send the object names for to callhome
+var reasonsToSendObjectNameToCallhome = []string{


same for this. Not using anywhere. Perhaps a miss during merge?

Yep. Removed it.

makalaaneesh · 2025-02-19T05:50:13Z

yb-voyager/cmd/assessMigrationCommand.go

-		}
-	}
-	return indexesOnComplexTypesFeature
+	log.Infof("fetching unsupported features for Index on complex datatypes...")


I believe this is just for backwards compatibility (to support yugabyted). RIght?

yeah, and also we are appending the assessment issue in this function getUnsupportedFeaturesFromSchemaAnalysisReport itself, so we need to call this for all various issue types

makalaaneesh · 2025-02-19T05:52:32Z

yb-voyager/src/query/queryissue/issues_ddl.go

-func NewIndexOnComplexDatatypesIssue(objectType string, objectName string, sqlStatement string, typeName string) QueryIssue {
-	issue := indexOnComplexDatatypesIssue
-	issue.Description = fmt.Sprintf(issue.Description, typeName)
+func NewIndexOnArrayDatatypeIssue(objectType string, objectName string, sqlStatement string) QueryIssue {


let's add tests for each of these types in issues_ddl_test.go to assert that creating an index actually fails on YB.

Added tests

priyanshi-yb · 2025-02-20T08:23:19Z

yb-voyager/src/query/queryissue/constants.go

+	INDEX_ON_POLYGON_DATATYPE_ISSUE_SUGGESTION       = "Refer to the docs link for the workaround"
+	INDEX_ON_TXID_SNAPSHOT_DATATYPE_ISSUE_SUGGESTION = "Refer to the docs link for the workaround"
+	INDEX_ON_ARRAY_DATATYPE_ISSUE_SUGGESTION         = "Refer to the docs link for the workaround"
+	INDEX_ON_USER_DEFINED_DATATYPE_ISSUE_SUGGESTION  = "Refer to the docs link for the workaround"


It is okay to have a single suggestion constant for this as it is same for all

priyanshi-yb · 2025-02-20T08:28:21Z

yb-voyager/src/query/queryissue/constants.go

+	INDEX_ON_POLYGON_DATATYPE_ISSUE_DESCRIPTION       = "Indexes on columns with polygon data type are not yet supported in YugabyteDB."
+	INDEX_ON_TXID_SNAPSHOT_DATATYPE_ISSUE_DESCRIPTION = "Indexes on columns with txid_snapshot data type are not yet supported in YugabyteDB."
+	INDEX_ON_ARRAY_DATATYPE_ISSUE_DESCRIPTION         = "Indexes on columns with array data type are not yet supported in YugabyteDB."
+	INDEX_ON_USER_DEFINED_DATATYPE_ISSUE_DESCRIPTION  = "Indexes on columns with user defined data types are not yet supported in YugabyteDB."


For user-defined type, we can mention the type name in the description as in the description field we can have dynamic data or the schema information in that.

Hey we weren't mentioning the UDT type name before also I guess.

priyanshi-yb · 2025-02-20T08:28:48Z

yb-voyager/src/query/queryissue/detectors_ddl.go

@@ -399,10 +399,10 @@ func (d *IndexIssueDetector) DetectIssues(obj queryparser.DDLObject) ([]QueryIss
 				if !ok {
 					continue
 				}
-				issues = append(issues, NewIndexOnComplexDatatypesIssue(
+				fmt.Printf("I was here with %s", typeName)


remove comment

priyanshi-yb · 2025-02-20T08:30:14Z

yb-voyager/src/query/queryissue/detectors_ddl.go

 						obj.GetObjectType(),
 						index.GetObjectName(),
 						"",
-						reportTypeName,


we can still send this type name to the NewIndexOnUserDefinedTypeIssue and put this detail in the description

Hey here we were sending "user_defined_type" as reportTypeName in case of UDT. We were not really sending any details about the UDT.

priyanshi-yb

LGTM!
Verify once with the local callhome setup if the issues are coming up properly with specific types and without sensitive information.

priyanshi-yb · 2025-02-24T11:26:51Z

yb-voyager/src/query/queryissue/issues_ddl_test.go

+		}
+
+		_, err = conn.Exec(ctx, testCase.sql)
+		fmt.Println("Query executed: ", testCase.sql)


remove print

priyanshi-yb · 2025-02-24T11:29:26Z

yb-voyager/cmd/assessMigrationCommand.go

-		}
-	}
-	return indexesOnComplexTypesFeature
+	log.Infof("fetching unsupported features for Index on complex datatypes...")


yeah, and also we are appending the assessment issue in this function getUnsupportedFeaturesFromSchemaAnalysisReport itself, so we need to call this for all various issue types

priyanshi-yb · 2025-02-24T11:33:06Z

migtests/tests/pg/omnibus/expected_files/expectedAssessmentReport.json

@@ -655,9 +655,9 @@
 		{


there are a couple of other schemas as well mgi and rna that are not in GH, so please verify those as well once

…s for the issue index on complex datatypes in schema report and assessment report (#2288) - Earlier indexes on all the complex datatypes were reported under a single issue type: INDEX ON COMPLEX DATATYPES - Added a separate issue for all such complex datatypes and removed the original issue. - Reporting the issue in both the analyze schema report and the assessment report. - Fixed issue: Define separate Issue for each case related to datatype(unsupported datatype, index on complex datatype) #2231

ShivanshGahlot requested review from priyanshi-yb, sanyamsinghal and makalaaneesh February 4, 2025 13:13

ShivanshGahlot added 7 commits February 18, 2025 10:58

Added index on array datatype issue

61a1c53

Added index on UDT type issue

514dbfe

Added index on complex datatypes issues for all the unsupported datat…

00aa471

…ypes

Added index on complex datatypes issues for all the unsupported datat…

60323b6

…ypes

Fixed schema analysis report test

30d62be

Fixed assessment report test

7bdf3be

Fixed go tests

931918f

ShivanshGahlot force-pushed the shivansh/assess-migration-payload branch from fa91bcd to 931918f Compare February 18, 2025 11:03

ShivanshGahlot added 7 commits February 18, 2025 11:06

Fixing merging changes

ba0f487

Fixing merging changes

540d9d9

Fixing merging changes

5e4b3df

Fixing merging changes

2b03111

Fixing merging changes

db991ea

Fixing merging changes

18157de

Fixing merging changes

cfd7ca9

makalaaneesh reviewed Feb 19, 2025

View reviewed changes

ShivanshGahlot added 4 commits February 19, 2025 12:16

Making suggested changes

d3add31

Making suggested changes

924d963

Making suggested changes

8dad4b1

Making suggested changes

44c9a49

priyanshi-yb reviewed Feb 20, 2025

View reviewed changes

ShivanshGahlot added 2 commits February 21, 2025 06:28

Making suggested changes

1f83063

Making suggested changes

e3db043

ShivanshGahlot requested review from makalaaneesh and priyanshi-yb February 23, 2025 11:50

ShivanshGahlot added 2 commits February 23, 2025 12:20

Test fixes

898b22d

Fixed test cases

4016658

Fixed test cases

d6eae74

priyanshi-yb approved these changes Feb 24, 2025

View reviewed changes

ShivanshGahlot merged commit ec1cb4f into main Feb 25, 2025
66 checks passed

ShivanshGahlot deleted the shivansh/assess-migration-payload branch February 25, 2025 05:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reporting a separate issue for all the different unsupported datatypes for the issue index on complex datatypes in schema report and assessment report #2288

Reporting a separate issue for all the different unsupported datatypes for the issue index on complex datatypes in schema report and assessment report #2288

ShivanshGahlot commented Feb 4, 2025 •

edited

Loading

makalaaneesh Feb 19, 2025

ShivanshGahlot Feb 19, 2025

makalaaneesh Feb 19, 2025

ShivanshGahlot Feb 19, 2025

makalaaneesh Feb 19, 2025

priyanshi-yb Feb 24, 2025

makalaaneesh Feb 19, 2025

ShivanshGahlot Feb 23, 2025

priyanshi-yb Feb 20, 2025

ShivanshGahlot Feb 23, 2025

priyanshi-yb Feb 20, 2025

ShivanshGahlot Feb 23, 2025

priyanshi-yb Feb 20, 2025

ShivanshGahlot Feb 23, 2025

priyanshi-yb Feb 20, 2025

ShivanshGahlot Feb 23, 2025

priyanshi-yb left a comment

priyanshi-yb Feb 24, 2025

priyanshi-yb Feb 24, 2025

priyanshi-yb Feb 24, 2025

Reporting a separate issue for all the different unsupported datatypes for the issue index on complex datatypes in schema report and assessment report #2288

Reporting a separate issue for all the different unsupported datatypes for the issue index on complex datatypes in schema report and assessment report #2288

Conversation

ShivanshGahlot commented Feb 4, 2025 • edited Loading

Describe the changes in this pull request

Describe if there are any user-facing changes

How was this pull request tested?

Does your PR have changes that can cause upgrade issues?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

priyanshi-yb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShivanshGahlot commented Feb 4, 2025 •

edited

Loading