Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modern Configuration Builder (v16.x) Part 1 #2241

Merged
merged 229 commits into from
Aug 29, 2024
Merged

Conversation

MrHinsh
Copy link
Member

@MrHinsh MrHinsh commented Aug 4, 2024

The new config will support:

  • appsettings.json - With defaults for many settings here rather than in code.
  • Environment Settings - Enables the ability to override settings with, for example, MigratonTools:Target:PersonalAccessToken as an environment variable.
  • Console Overides - Enables any setting to be passed via the console.

These changes are a prelude to creating a configuration command that allows users to change the settings and reduce the failure from manual changes. Will also allow greater flexibility and use of the Options Pattern as much as possible.

This will trigger a bump to 16.0 as its a major config change and the code will adapt the new configs to the old model for legacy support..

Todo List:

…n management

✨ (config): introduce appsettings.json for centralized configuration
📝 (docs): update MigrationTools.xml with new version and branch information

Adding `appsettings.json` to the solution items ensures that configuration settings are centralized and easily manageable. This file includes logging levels and migration tool settings, which enhances the flexibility and maintainability of the application. The `MigrationTools.xml` documentation is updated to reflect the latest version and branch information, ensuring that the documentation is current and accurate.

♻️ (MigrationToolHost.cs): refactor configuration and logging setup for better readability and maintainability

Refactor the configuration and logging setup to improve code readability and maintainability. The changes include:
- Adding missing using directives for `System.Configuration` and `NuGet.Protocol.Plugins`.
- Removing unnecessary whitespace.
- Aligning the indentation for better readability.
- Changing the way `EngineConfiguration` options are configured to use a more concise and readable approach with `AddOptions<EngineConfiguration>().Configure`.
- Ensuring consistent formatting and spacing throughout the file.

These changes make the code easier to read and maintain, reducing the likelihood of errors and improving overall code quality.
… add new field mappings, and enhance logging

📝 (MigrationTools.xml): update Git metadata for documentation
✨ (launchSettings.json): add new launch profile for executing without config

The project names in the configuration are updated to "myProjectName" for consistency. The authentication mode is changed to "Prompt" to enhance security. New field mappings are added to support various field transformations and enrichments. The logging level is set to "Information" for better clarity. Git metadata in the documentation is updated to reflect the latest commit details. A new launch profile is added to allow execution without configuration, improving flexibility in running the project.

🔧 (MigrationToolHost.cs): add support for environment variables in configuration
💡 (MigrationToolHost.cs): add spacing for better readability

Add support for environment variables to allow configuration via environment settings, enhancing flexibility and deployment options. Improve code readability by adding spacing around variable declarations and method calls.
…nfigurations

- Add `configuration3.json` to support new migration scenarios.
- Update `appsettings.json` to include `WorkItemFieldMapping`.
- Enable `WorkItemMigrationConfig` in `configuration.json`.
- Update `MigrationTools.sln` to include `configuration3.json`.
- Update `MigrationTools.xml` to reflect the latest commit details.
- Add `ConfigurationSectionName` constant in `TfsAttachmentEnricherOptions`.
- Modify `TfsNodeStructure` constructor to use `IOptions<TfsNodeStructureOptions>`.

These changes enhance the flexibility and configurability of the migration tools, allowing for more detailed and specific migration scenarios. The updates to the solution and configuration files ensure that new features are properly integrated and can be easily managed.

✨ (ProcessorEnrichers): add configuration section names for various options
🔧 (ServiceCollectionExtensions): bind configuration sections to options
📝 (launchSettings.json): add new launch configuration for configuration3.json
📝 (csproj): include configuration3.json in the project

Adding `ConfigurationSectionName` constants to various options classes improves the maintainability and readability of the code by centralizing the configuration section names. Binding these configuration sections in `ServiceCollectionExtensions` ensures that the options are correctly configured from the appsettings. Including `configuration3.json` and updating `launchSettings.json` allows for easier testing and execution of different configurations.

✨ (MigrationTools): add support for new configuration format and command line arguments

Add support for a new configuration format to improve flexibility and future-proofing. Introduce command line argument parsing to enhance configurability. Implement a version check to ensure compatibility with older configurations, providing a warning for deprecated formats. Add utility methods to dynamically load processor configurations and enrichers from the new format. Remove the workaround for the query SOAP bug as it is no longer needed.
…xtensions.cs, MigrationToolHost.cs): refactor endpoint configuration and processor binding

Refactor endpoint configuration methods to use a more descriptive naming convention, changing `AddEndpoint` to `AddMigrationToolsEndpoint` and `ConfigureEndpoint` to `ConfigureMigrationToolsEndpoint`. This improves code readability and clarity.

Simplify processor binding logic in `MigrationToolHost.cs` by removing redundant code and using a more concise approach. This reduces complexity and potential errors in processor configuration.

♻️ (EndpointFactoryServiceCollectionExtensions.cs): rename methods to improve clarity and consistency
🔧 (ServiceCollectionExtensions.cs): update method calls to match renamed methods
💡 (WorkItemMigrationContext.cs): add spacing for better readability

Renaming methods in `EndpointFactoryServiceCollectionExtensions.cs` to `AddMigrationToolsEndpoint` and `AddMigrationToolsEndPoints` improves clarity by making it explicit that these methods are related to Migration Tools. This change enhances code readability and maintainability. The corresponding method calls in `ServiceCollectionExtensions.cs` are updated to reflect these changes. Additionally, a minor formatting improvement is made in `WorkItemMigrationContext.cs` for better readability.
Refactor the configuration structure to improve clarity and maintainability. The changes include:
- Moving and renaming configuration sections for better organization.
- Updating the configuration keys to reflect the new structure.
- Simplifying the retrieval of configuration values in the code.
- Removing redundant and outdated configuration sections.

These changes aim to make the configuration files more intuitive and easier to manage, reducing the risk of errors and improving the overall maintainability of the project.

✨ (MigrationTools): add AppDomainExtensions and TypeExtensions for better type handling and configuration

Introduce `AppDomainExtensions` to fetch types from assemblies related to MigrationTools. Add `TypeExtensions` to filter types by interface, configuration section name, and name string. Update `ConfigurationSectionExtensions` to utilize these new extensions for improved configuration handling.

These changes enhance the flexibility and maintainability of the configuration system by modularizing type retrieval and filtering logic, making it easier to extend and debug.
…Exist" processor type

🔧 (MigrationToolHost.cs): update configuration loading logic for processors and enrichers
♻️ (ConfigurationSectionExtensions.cs): refactor configuration section extension methods for better error handling and logging

The new processor configuration for "DoesNotExist" is added to support additional processing types. The configuration loading logic in `MigrationToolHost.cs` is updated to use a more flexible and robust method for loading processor and enricher configurations. The refactoring in `ConfigurationSectionExtensions.cs` improves error handling and logging, making it easier to diagnose configuration issues.
… source and target options

Improve the readability and maintainability of the code by refactoring the configuration section parsing for the `Source` and `Target` options. This change ensures that the `Source` and `Target` options are correctly populated from the configuration, enhancing the flexibility and correctness of the migration tool's configuration handling.
The "DoesNotExist" processor type is removed because it is not a valid processor and could cause errors or unexpected behavior in the application. This cleanup ensures that only valid and functional processors are configured, improving the reliability and maintainability of the configuration file.
📝 (docs): update MigrationTools.xml with new commit metadata
🔧 (csproj): ensure appsettings.json is always copied to output directory

The new Test settings in appsettings.json allow for additional configuration options under CommonEnrichers. The MigrationTools.xml documentation is updated to reflect the latest commit metadata, ensuring accurate reference information. The csproj file is modified to always copy appsettings.json to the output directory, ensuring the configuration file is available during runtime.
Remove the "Test" settings under "CommonEnrichers" and redundant "FieldMaps" entries under "TfsFieldMappings". This cleanup reduces clutter and potential confusion in the configuration file, ensuring only relevant settings are maintained.
@github-actions github-actions bot added enhancement documentation This is an item for a Documentation enhancement build-action Edits to the build system enhancement-classic enhancement-modern feature labels Aug 4, 2024
…lease

The next-version is updated from 15.0.1 to 16.0.0 to reflect significant changes or new features that warrant a major version increment. This ensures proper versioning and helps in tracking the evolution of the project.
@MrHinsh MrHinsh marked this pull request as draft August 4, 2024 12:51
…onToolHost.cs): integrate Serilog for enhanced logging and update configuration

Add Serilog configuration to appsettings.json and configuration3.json to improve logging capabilities. Update the logging levels for Microsoft and Microsoft.Hosting.Lifetime to Debug for more detailed logs. Refactor MigrationToolHost.cs to use Serilog for logging, including setting up log levels and output templates. This change enhances the application's logging capabilities, making it easier to debug and monitor the application. Additionally, update the version in configuration.json to 16.0 to reflect the new configuration changes.
…agement

♻️ (MigrationToolHost.cs): refactor logging setup and add method to retrieve all services
📝 (MigrationTools.xml): update auto-generated documentation with new commit details

The logging configuration in `appsettings.json` is updated to include `rollingInterval` and `shared` properties, which improve log file management by creating new log files daily and allowing shared access. The `MigrationToolHost.cs` file is refactored to streamline the logging setup and includes a new method to retrieve all services from the service provider, enhancing code readability and maintainability. The `MigrationTools.xml` documentation is updated to reflect the latest commit details, ensuring accurate and up-to-date information.
…nt settings

♻️ (MigrationToolHost.cs): refactor logging setup to include separate log files for verbose and error logs

The logging configuration in `appsettings.json` is updated to include `Microsoft.Hosting.Lifetime` at the `Debug` level for better debugging information. Redundant logging settings are removed to streamline the configuration.

In `MigrationToolHost.cs`, the logging setup is refactored to create separate log files for verbose and error logs, improving log management and readability. The logs are now stored in a dynamically created path, enhancing the flexibility of the logging system.
♻️ (MigrationToolHost.cs): refactor logging setup for better maintainability
📝 (MigrationTools.xml): update generated documentation with new commit details

The redundant logging configuration in `appsettings.json` is removed to simplify the settings file. The logging setup in `MigrationToolHost.cs` is refactored to centralize the output template and add missing enrichers for better log context. The generated documentation in `MigrationTools.xml` is updated to reflect the latest commit details.
… to Verbose

♻️ (MigrationToolHost.cs): remove redundant log level restriction in Serilog configuration

Changing the log level for `Microsoft.Hosting.Lifetime` to `Verbose` provides more detailed logging information, which can be useful for debugging and monitoring. The redundant `restrictedToMinimumLevel` parameter is removed from the Serilog configuration to simplify the logging setup and ensure all log levels are captured as intended.
♻️ (MigrationToolHost.cs): refactor configuration version handling and logging
📝 (MigrationTools.xml): update generated documentation with new commit details

The Serilog minimum level is updated to "Information" to reduce log verbosity. The configuration version handling in `MigrationToolHost.cs` is refactored to use a new `MigrationConfigVersion` enum, improving code readability and maintainability. The generated documentation is updated to reflect the latest commit details.
…tra commands support

Add detailed descriptions and examples for the `execute` and `init` commands to improve user understanding. Introduce an optional `extraCommands` parameter to `CreateDefaultBuilder` to allow additional commands to be configured. This enhances the flexibility and usability of the migration tool.

Additionally, minor refactoring and cleanup are performed to improve code readability and maintainability.
…ration file creation and editing

🔧 (MigrationTools.sln): include appsettings.json in the solution items
♻️ (launchSettings.json): reorganize and rename profiles for better clarity
📝 (MigrationTools.xml): update XML documentation with new commit details

The MigrationConfigCommand is introduced to facilitate the creation and editing of configuration files, improving the flexibility and usability of the tool. The appsettings.json file is added to the solution items for better configuration management. The launchSettings.json file is reorganized and profiles are renamed for better clarity and usability. The XML documentation is updated to reflect the latest commit details, ensuring accurate and up-to-date information.
♻️ (MigrationConfigCommand): change telemetry event name to "MigrationConfigCommand" for better clarity

🐛 (MigrationConfigCommand): throw exception if config file already exists to prevent unsupported edits

💡 (MigrationToolHost): uncomment and update service configurations for better clarity

The MigrationTools.xml file is updated to reflect the latest commit details, ensuring documentation accuracy. The telemetry event name in MigrationConfigCommand is changed to "MigrationConfigCommand" for better clarity. An exception is now thrown if the configuration file already exists, preventing unsupported edits. Service configurations in MigrationToolHost are uncommented and updated for better clarity and maintainability.
…new commit details

Update the MigrationTools.xml file to reflect the latest commit details, including the new commit hash, date, and number of commits. This ensures the documentation is up-to-date with the current state of the repository.

♻️ (tests): refactor TfsWorkItemEndPointTests to improve test setup

Refactor the TfsWorkItemEndPointTests to streamline the creation of TfsWorkItemEndpoint instances. The new GetTfsWorkItemEndPoint method centralizes the setup of dependencies and options, making the tests more maintainable and reducing redundancy. This change improves the clarity and maintainability of the test code.

✅ (tests): add dynamic source and target names in TfsProcessorTests

♻️ (infrastructure): refactor endpoint retrieval in Processor.cs

The changes introduce dynamic source and target names in `TfsProcessorTests` to improve flexibility and readability. This allows the tests to be more adaptable to different configurations.

In `Processor.cs`, the endpoint retrieval logic is refactored into a new `GetEndpoint` method. This improves code readability and error handling by centralizing the logic for fetching endpoints and handling potential exceptions.
@MrHinsh MrHinsh self-assigned this Aug 27, 2024
@MrHinsh MrHinsh marked this pull request as ready for review August 27, 2024 18:35
@MrHinsh MrHinsh merged commit 8a0f516 into main Aug 29, 2024
14 checks passed
@MrHinsh MrHinsh deleted the feature/configuration-builder branch August 29, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build-action Edits to the build system documentation This is an item for a Documentation enhancement enhancement enhancement-classic enhancement-modern feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant