Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AzureDevOpsPipelineProcessorOptions throws a serverPath null API exception for the build pipelines #970

Open
IstuntmanI opened this issue Jul 20, 2021 · 6 comments

Comments

@IstuntmanI
Copy link

Describe your issue:

When running the pipelines processor with the default settings, an error similar to Value cannot be null. Parameter name: definition.Repository.Mappings.Mapping.ServerPath appears. I just started using this tool and it seems to me that this processor is in some kind of alpha stage? Hopefully it isn't deprecated or anything.

Source Details

  • Source Version: Azure DevOps Services
  • Source Language: Eng

Target Details

I'm using this tool to move a project from an organization to another one, so:

  • Target Version: Azure DevOps Services
  • Target Language: Eng

I have completed the following:

Simply running the tool with this default processor:

{
	"$type": "AzureDevOpsPipelineProcessorOptions",
	"Enabled": true,
	"MigrateBuildPipelines": true,
	"MigrateReleasePipelines": true,
	"MigrateTaskGroups": true,
	"MigrateVariableGroups": true,
	"MigrateServiceConnections": true,
	"BuildPipelines": null,
	"ReleasePipelines": null,
	"ProcessorEnrichers": null,
	"SourceName": "Source",
	"TargetName": "Target"
}

Attempted fix

I noticed that in the https://dev.azure.com/{org}/{proj}/_apis/build/definitions/{id}/ URL we have this information:

"repository":{
  "properties":{
	 "cleanOptions":"0",
	 "tfvcMapping":"{\"mappings\":[{\"serverPath\":\"$/{proj}/{branch}\",\"mappingType\":\"map\",\"localPath\":\"\\\\test\"}]}",
	 "labelSources":"0",
	 "labelSourcesFormat":"$(build.buildNumber)"
  },
  "id":"$/",
  "type":"TfsVersionControl",
  "name":"{proj}",
  "url":"https://dev.azure.com/{org}/",
  "defaultBranch":"$/{proj}/{branch}",
  "rootFolder":"$/{proj}",
  "clean":"false",
  "checkoutSubmodules":false
},

Then I took a look in the BuildDefinitions.cs file and I noticed that the accessor for the tfvcMapping string field is missing from the Properties class (which is used in the Repository class, which is used in the BuildDefinition class), so I created a derived PropertiesBuildDefinition class (in case that its current form is still needed for some reason):

public partial class PropertiesBuildDefinition : Properties
{
    public string TfvcMapping { get; set; }
}
public partial class Repository
{
    public PropertiesBuildDefinition Properties { get; set; }
    // ...
}
}

This will simply write the server and local paths just like they were before, in the old project. This probably means that it may throw another exception if the project to migrate to has another name. I'm not creating a pull request for this reason and also because I'm not sure if this is fine for other Azure DevOps/TFS source versions.

@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days

@tomfrenzel
Copy link
Collaborator

The project name would have to be mapped to the target one. Shouldn't be a Big Deal.
But, I'm not sure whether Azure DevOps or TFS versions which don't use TfvcMapping will throw an exception when trying to HTTP POST a Pipeline with that Filed (even if it's empty). I'll have to validate that.

@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days

@tomfrenzel
Copy link
Collaborator

I'll look into that the next weeks

@github-actions github-actions bot closed this as completed Mar 1, 2022
@backasch
Copy link

backasch commented Sep 19, 2024

Hi @MrHinsh, @tomfrenzel

any updates on this topic, I received the same error also with version 16.0.3.
We are using Azure DevOps Server 2022.

Used configuration:

{
    "MigrationTools": {
      "Version": "16.0",
      "Endpoints": {
      "Source_AdoEndpoint": {
        "EndpointType": "AzureDevOpsEndpoint",
        "Organisation": "https://tfsurl.com/tfs/SourceCollection/",
        "Project": "PROJECT",
        "AuthenticationMode": "AccessToken",
        "AccessToken": "MyToken",
        "ReflectedWorkItemIdField": null,
        "EndpointEnrichers": null
      },
      "Target_AdoEndpoint": {
        "EndpointType": "AzureDevOpsEndpoint",
        "Organisation": "https://tfsurl.com/tfs/TargetCollection/",
        "Project": "PROJECT",
        "AuthenticationMode": "AccessToken",
        "AccessToken": "MyToken",
        "ReflectedWorkItemIdField": null,
        "EndpointEnrichers": null
      }
    },
      "Processors": [
        {
          "ProcessorType": "AzureDevOpsPipelineProcessor",
          "BuildPipelines": "",
          "Enabled": "True",
          "MigrateBuildPipelines": "True",
          "MigrateReleasePipelines": "True",
          "MigrateServiceConnections": "True",
          "MigrateTaskGroups": "True",
          "MigrateVariableGroups": "True",
          "ReleasePipelines": "",
          "RepositoryNameMaps":  {
            "PROJECT": "PROJECT"
          },
          "SourceName": "Source_AdoEndpoint",
          "TargetName": "Target_AdoEndpoint"
        }
      ]
    }
  }

Error Message:

POST https://tfsurl.com/tfs/TargetCollection//PROJECT/_apis/build/definitions/
{"$id":"1","innerException":null,"message":"Value cannot be null.\r\nParameter name: definition.Repository.Mappings.Mapping.ServerPath","typeName":"System.ArgumentNullException, mscorlib","typeKey":"ArgumentNullException","errorCode":0,"eventId":0}

I need to add the RepositoryNameMaps, otherwise the migration tool faults with an NRE.
Both projects have the same name and also the repository names are equal.

[16.0.3] Error while running AzureDevOpsPipelineProcessor
System.NullReferenceException: Object reference not set to an instance of an object.
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.MapRepositoriesInBuidDefinition(IEnumerable`1 sourceRepositories, IEnumerable`1 targetRepositories, BuildDefinition definitionToBeMigrated) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 386
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.<CreateBuildPipelinesAsync>d__17.MoveNext() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 304
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.<MigratePipelinesAsync>d__9.MoveNext() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 86
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.InternalExecute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 40
   at MigrationTools.Processors.Infrastructure.Processor.Execute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools\Processors\Infrastructure\Processor.cs:line 98

Kind Regards
Martin

@MrHinsh MrHinsh reopened this Sep 19, 2024
@xci-mrt
Copy link

xci-mrt commented Sep 25, 2024

Having same issue


[13:41:16 INF] [16.0.4-Preview.5] Beginning run of 1 processors
[13:41:16 INF] [16.0.4-Preview.5] Processor: AzureDevOpsPipelineProcessor
[13:41:16 INF] [16.0.4-Preview.5] Migration Context Start: AzureDevOpsPipelineProcessor
[13:41:16 INF] [16.0.4-Preview.5] Processor::InternalExecute::Start
[13:41:16 INF] [16.0.4-Preview.5] Processor::EnsureConfigured
[13:41:16 INF] [16.0.4-Preview.5] ProcessorEnricherContainer::ProcessorExecutionBegin
[13:41:16 INF] [16.0.4-Preview.5] Processing Service Connections..
[13:41:17 INF] [16.0.4-Preview.5] 0 of 1 source ServiceConnection(s) are going to be migrated..
[13:41:17 INF] [16.0.4-Preview.5] 0 of 0 ServiceConnection(s) got migrated..
[13:41:17 INF] [16.0.4-Preview.5] Processing Variablegroups..
[13:41:17 INF] [16.0.4-Preview.5] 0 of 2 source VariableGroups(s) are going to be migrated..
[13:41:17 INF] [16.0.4-Preview.5] 0 of 0 VariableGroups(s) got migrated..
[13:41:17 INF] [16.0.4-Preview.5] Processing Taskgroups..
[13:41:17 INF] [16.0.4-Preview.5] 0 of 0 source TaskGroup(s) are going to be migrated..
[13:41:17 INF] [16.0.4-Preview.5] 0 of 0 TaskGroup(s) got migrated..
[13:41:17 INF] [16.0.4-Preview.5] Processing Build Pipelines..
[13:41:17 INF] [16.0.4-Preview.5] Querying definitions in the project: devsw
[13:41:17 INF] [16.0.4-Preview.5] Configured BuildDefinition definitions: All
[13:41:20 INF] [16.0.4-Preview.5] Querying definitions in the project: devsw
[13:41:20 INF] [16.0.4-Preview.5] Configured BuildDefinition definitions: All
[13:41:20 INF] [16.0.4-Preview.5] 117 of 123 source BuildDefinition(s) are going to be migrated..
Error logged to Elmah.io!
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!
[13:41:20 FTL] [16.0.4-Preview.5] Error while running AzureDevOpsPipelineProcessor
System.NullReferenceException: Object reference not set to an instance of an object.
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.MapRepositoriesInBuidDefinition(IEnumerable`1 sourceRepositories, IEnumerable`1 targetRepositories, BuildDefinition definitionToBeMigrated) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 386
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.<CreateBuildPipelinesAsync>d__17.MoveNext() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 304
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.<MigratePipelinesAsync>d__9.MoveNext() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 86
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at MigrationTools.Processors.AzureDevOpsPipelineProcessor.InternalExecute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.AzureDevops.Rest\Processors\AzureDevOpsPipelineProcessor.cs:line 40
   at MigrationTools.Processors.Infrastructure.Processor.Execute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools\Processors\Infrastructure\Processor.cs:line 98
[13:41:20 INF] [16.0.4-Preview.5] AzureDevOpsPipelineProcessor completed in 00:00:00
[13:41:20 ERR] [16.0.4-Preview.5] AzureDevOpsPipelineProcessor The Processor MigrationEngine entered the failed state...stopping run
[13:41:20 INF] [16.0.4-Preview.5] Command ExecuteMigrationCommand completed in 00:00:05.6353979

config:


{
  "Serilog": {
    "MinimumLevel": "Debug"
  },
  "MigrationTools": {
    "Version": "16.0",
    "Endpoints": {
      "Source": {
        "EndpointType": "AzureDevOpsEndpoint",
        "Organisation": "http://tfs01.dev.local:8080/tfs/dev%20DefaultCollection/",
        "Project": "devsw",
        "AuthenticationMode": "AccessToken",
        "AccessToken": "xxxxxx",
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      },
      "Target": {
        "EndpointType": "AzureDevOpsEndpoint",
        "Organisation": "http://tfs01.dev.local:8080/tfs/dev%20Migration%20Performance%20Test/",
        "Project": "devsw",
        "AuthenticationMode": "AccessToken",
        "AccessToken": "xxxxx",
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      }
    },
    "Processors": [
      {
        "ProcessorType": "AzureDevOpsPipelineProcessor",
        "Enabled": true,
        "MigrateBuildPipelines": true,
        "MigrateReleasePipelines": false,
        "MigrateTaskGroups": false,
        "MigrateVariableGroups": false,
        "MigrateServiceConnections": false,
        "BuildPipelines": true,
        "ReleasePipelines": true,
        "RepositoryNameMaps": true,
        "SourceName": "Source",
        "TargetName": "Target"
      }
    ],
    "CommonTools": {
      "TfsChangeSetMappingTool": {
        "Enabled": true,
        "ChangeSetMappingFile": ""
      },
      "TfsGitRepositoryTool": {
        "Enabled": true,
        "Mappings": null
      },
      "FieldMappingTool": {
        "Enabled": true,
        "FieldMaps": []
      }
    }
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants