Add Audit Logging To SecreteManager.exe #4183

davfost · 2024-09-27T21:47:52Z

Add Audit Logging To SecreteManager.exe

Adds the OpenTelemetry.Audit.Geneva package and dependencies
Following logging recommendations from Geneva team
Added non-volatile logging operations to secret update and create operations
Adds new command line options for setting the target service ID to used when performing audit logging

Part Of GitHub Issue https://github.com/dotnet/arcade/issues/13217

…e the secrete manager command line operations to allow for passing in a service tree id associated with the service execuring change operations.

Changing variable names to be more consistent with use Altering some comments

src/SecretManager/Microsoft.DncEng.SecretManager/Program.cs

…l and base command settings when not using the command line.

mmitche · 2024-10-03T22:36:12Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/InfoCommand.cs

    {
        _console = console;
    }

    public override Task RunAsync(CancellationToken cancellationToken)
    {
+        // Provides a curtisy warning message if the ServiceTreeId option is set to a empty guid


Suggested change

// Provides a curtisy warning message if the ServiceTreeId option is set to a empty guid

// Provides a courtesy warning message if the ServiceTreeId option is set to a empty guid

mmitche · 2024-10-04T15:19:16Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/ProjectBaseCommand.cs

+            {
+                {"servicetreeid=", "The service tree ID (must be a valid GUID id from aka.ms/servicetree)", id =>
+                    {
+                        if (Guid.TryParse(id, out var guid))


Since someone took the time to specify a service tree ID here, I would just error out if it's not a valid GUID.

mmitche · 2024-10-04T16:01:16Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+            var auditFactory = AuditLoggerFactory.Create(options =>
+            {
+                // We use ETW as the destination for the audit logs becsue the application is not gurenteed to run on windows
+                options.Destination = AuditLogDestination.ETW;


ETW is available on Linux?

+1. We also don't typically capture ETW data. Where does audit information ultimately end up?

mmitche · 2024-10-04T16:23:15Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/ValidateCommand.cs

@@ -1,32 +1,33 @@
 using System.Threading;
 using System.Threading.Tasks;
 using Microsoft.DncEng.CommandLineLib;
+using Microsoft.VisualStudio.Services.Common;


garath · 2024-10-07T22:03:09Z

src/SecretManager/Microsoft.DncEng.SecretManager/StorageTypes/AzureKeyVault.cs

-        var createdSecret = await client.SetSecretAsync(name, value.Value ?? "");
-        var properties = createdSecret.Value.Properties;
-        foreach (var (k, v) in value.Tags)
+        // The default audit state should alwasy be failure and overwritten with success at the end of the operation.


Suggested change

// The default audit state should alwasy be failure and overwritten with success at the end of the operation.

// The default audit state should always be failure and overwritten with success at the end of the operation.

garath · 2024-10-07T22:04:52Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+            // The hope is that app insights will also catch the base exception for debugging.
+            catch (Exception ex)
+            {
+                Console.WriteLine($"Failed to add audit log for secret update!: <{ex.Message}>");


Will the audit message always be safe for public view in a console?

garath · 2024-10-07T22:07:23Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+    /// </summary>
+    public class SecurityAuditLogger
+    {
+        private ILogger ControlPanelLogger;


Suggested change

private ILogger ControlPanelLogger;

private ILogger ControlPlaneLogger;

I think that's what you meant?

garath · 2024-10-07T22:09:24Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+            {
+                LogSecretAction(OperationType.Update, operationName, credentialProvider, secretName, secretStoreType, secretLocation, result, resultMessage);
+            }
+            // Audit logging is a 'volitile' operation meaning it can throw exceptions if logging fails.


Suggested change

// Audit logging is a 'volitile' operation meaning it can throw exceptions if logging fails.

// Audit logging is a 'volatile' operation meaning it can throw exceptions if logging fails.

garath · 2024-10-07T22:10:28Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+                return;
+            }
+
+            // The token applicatoin id of the client running the assembly.


Suggested change

// The token applicatoin id of the client running the assembly.

// The token application id of the client running the assembly.

garath · 2024-10-07T22:11:01Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+            // NOTE: The user identity here should be something 'dynamic'.
+            // If you are hard coding this value you should question if this Audit Log is useful
+            // as it is likly redundant to lower level permission change logging that is already occuring.


Should these comments be here? They look like copy-paste from an example?

garath · 2024-10-07T22:29:24Z

NuGet.config

 <configuration>
  <packageSources>
    <clear />
+    <add key="AzureGenevaMonitoring" value="https://msblox.pkgs.visualstudio.com/_packaging/AzureGenevaMonitoring/nuget/v3/index.json" />


What permissions are required to access this feed? Will there be issues with CI and individual users getting access?

There is currently an issue with this feed and it blocked the PR build from completing. I have been discussing it with Matt Mitchell to see how it can be resolved. As you observed this feed requires external permissions that are not granted so the package restore is failing.

The basic issue is that the Audit team says we have to use the version of the package that comes from 'this' feed and not the public version of the package.

I need to investigate if the 'public' feed version of the package has the same class and methods defined. If so, we can do something where the 'internal feed' is different than the feed used in the 'public' processes.

I also have open questions to the Audit team to verify if the package from 'this' feed CAN be exposed publicly. If so we may be able to just manually move target versions into one of the feeds we manage directly.

Matt said these two options have been used in the past to deal with other packages that have similar issues.

Good good, Matt should have good ideas here.

Just to say it explicitly, do not merge this PR until there's a good solution for public CI and for local dev.

garath · 2024-10-08T00:11:13Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+        /// </summary>
+        public SecurityAuditLogger(Guid serviceTreeId)
+        {
+            var auditFactory = AuditLoggerFactory.Create(options =>


What does this logger do if the Service ID is Guid.Empty? Does it still try to log or is that enough to make it silent?

garath · 2024-10-08T00:12:07Z

src/SecretManager/Microsoft.DncEng.SecretManager.Tests/SynchronizeCommandTests.cs

@@ -26,6 +24,11 @@ private async Task TestCommand(DateTimeOffset now, string manifestText, string l
        var cancellationToken = cts.Token;

        var services = new ServiceCollection();
+        // Dependency injection instruction needed to support properties used for Geneval Logging operations
+        services.AddSingleton(new GlobalCommand());
+        services.AddSingleton(new SecurityAuditLogger(Guid.Empty));


Since these are test classes, would it be better to make a do-nothing version of SecurityAuditLogger? (I'm not sure what SecurityAuditLogger does.)

garath · 2024-10-08T00:13:51Z

src/SecretManager/Microsoft.DncEng.SecretManager.Tests/SynchronizeCommandTests.cs

+        services.AddSingleton(new GlobalCommand());
+        services.AddSingleton(new SecurityAuditLogger(Guid.Empty));
+
+        // Original dependency injection instructions


I don't thin kthis comment (or the previous comment) is necessary. The code shouldn't have a sense of "human time". That's what version control handles. The code should always just be "this is what it is right now".

garath · 2024-10-08T00:18:34Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/ValidateAllCommand.cs

 using Microsoft.DncEng.CommandLineLib;
 using Microsoft.DncEng.SecretManager.StorageTypes;
+using Microsoft.VisualStudio.Services.Common;


What's bringing this in?

garath · 2024-10-08T00:18:53Z

src/SecretManager/Microsoft.DncEng.SecretManager/ITokenCredentialProvider.cs

@@ -5,5 +5,15 @@ namespace Microsoft.DncEng.SecretManager;

 public interface ITokenCredentialProvider
 {
+    /// <summary>
+    /// The applicatoin ID for the credential provider.


Suggested change

/// The applicatoin ID for the credential provider.

/// The application ID for the credential provider.

garath · 2024-10-08T00:22:37Z

Directory.Packages.props

@@ -57,15 +57,15 @@
    <PackageVersion Include="Microsoft.DotNet.SwaggerGenerator.MSBuild" Version="$(MicrosoftDotNetSwaggerGeneratorMSBuildVersion)" />
    <PackageVersion Include="Microsoft.DotNet.VersionTools" Version="$(MicrosoftDotNetVersionToolsVersion)" />
    <PackageVersion Include="Microsoft.DotNet.Web.Authentication" Version="$(MicrosoftDotNetWebAuthenticationVersion)" />
-    <PackageVersion Include="Microsoft.Extensions.Configuration" Version="6.0.1" />
-    <PackageVersion Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
+    <PackageVersion Include="Microsoft.Extensions.Configuration" Version="8.0.0" />


I recognize that these updates are safe because they all implement netstandard2.0, but I'm wondering what motivates the bump in this PR?

garath · 2024-10-08T00:32:03Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/ProjectBaseCommand.cs

+    {
+
+        /// <summary>
+        /// Indictes if the global option for 'quiet' is set


Suggested change

/// Indictes if the global option for 'quiet' is set

/// Indicates if the global option for 'quiet' is set

garath · 2024-10-08T00:32:38Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/ProjectBaseCommand.cs

+
+        /// <summary>
+        /// Provides the ServiceTreeId set with global options
+        /// The ID is a goid and is set to Guid.Empty if not set


Suggested change

/// The ID is a goid and is set to Guid.Empty if not set

/// The ID is a guid and is set to Guid.Empty if not set

garath · 2024-10-08T00:33:14Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/ProjectBaseCommand.cs

+        }
+
+        /// <summary>
+        /// Overides the GetOptions method from the base class to add a cusotm option for the ServiceTreeId


Suggested change

/// Overides the GetOptions method from the base class to add a cusotm option for the ServiceTreeId

/// Overrides the GetOptions method from the base class to add a custom option for the ServiceTreeId

garath · 2024-10-08T00:35:40Z

src/SecretManager/Microsoft.DncEng.SecretManager/Commands/ValidateAllCommand.cs

@@ -45,6 +47,9 @@ public override bool AreRequiredOptionsSet()

    public override async Task RunAsync(CancellationToken cancellationToken)
    {
+        // Provides a curtisy warning message if the ServiceTreeId option is set to a empty guid


Suggested change

// Provides a curtisy warning message if the ServiceTreeId option is set to a empty guid

// Provides a courtesy warning message if the ServiceTreeId option is set to an empty guid

garath · 2024-10-08T00:49:29Z

src/SecretManager/Microsoft.DncEng.SecretManager/SecurityAuditLogger.cs

+            var ipAddress = host.AddressList.FirstOrDefault(ip => ip.AddressFamily == AddressFamily.InterNetwork);
+            if (ipAddress == null)
+            {
+                throw new Exception("No network adapters with an IPv4 address in the system!");


garath · 2024-10-08T00:52:57Z

src/SecretManager/Microsoft.DncEng.SecretManager/StorageTypes/AzureKeyVault.cs

+            _auditLogger.LogSecretUpdate(
+                credentialProvider: _tokenCredentialProvider,
+                secretName: name,
+                secretStoreType: "AzureKeyVault",


Suggested change

secretStoreType: "AzureKeyVault",

secretStoreType: nameof(AzureKeyVault),

Compiler fanciness.

garath

Marking to hold at least until the questions about how CI will handle the new packaging are answered.

davfost added 7 commits September 26, 2024 16:37

Initial change to add secuirty audit logging capabilites and to updat…

05b8a68

…e the secrete manager command line operations to allow for passing in a service tree id associated with the service execuring change operations.

Resolving merge conflicts with main

cd9b948

Resolving merge conflict with main

7134128

Merge remote-tracking branch 'origin/main' into dev/davfost/20240710

e661780

Adding logging for Key creation in Azure Key vaults

87c81eb

Changing variable names to be more consistent with use Altering some comments

Merge remote-tracking branch 'origin/main' into dev/davfost/20240710

7f74cda

Removing temporary test project from solution.

80629f8

davfost requested review from garath, dougbu, mmitche, riarenas, epananth and missymessa September 27, 2024 21:47

davfost self-assigned this Sep 27, 2024

dougbu reviewed Sep 27, 2024

View reviewed changes

src/SecretManager/Microsoft.DncEng.SecretManager/Program.cs Outdated Show resolved Hide resolved

davfost added 3 commits September 27, 2024 16:14

Resorting using statements per PR comments

f71bee0

Update to address unit test failure and add capabilities to set globa…

cc6d25e

…l and base command settings when not using the command line.

Merge remote-tracking branch 'origin/main' into dev/davfost/20240710

db2f714

mmitche reviewed Oct 3, 2024

View reviewed changes

mmitche reviewed Oct 4, 2024

View reviewed changes

Merge remote-tracking branch 'origin/main' into dev/davfost/20240710

6676fbe

garath reviewed Oct 7, 2024

View reviewed changes

garath reviewed Oct 8, 2024

View reviewed changes

garath requested changes Oct 8, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Audit Logging To SecreteManager.exe #4183

Add Audit Logging To SecreteManager.exe #4183

davfost commented Sep 27, 2024 •

edited

Loading

mmitche Oct 3, 2024

mmitche Oct 4, 2024

mmitche Oct 4, 2024

garath Oct 8, 2024

mmitche Oct 4, 2024

garath Oct 7, 2024

garath Oct 7, 2024

garath Oct 7, 2024

garath Oct 7, 2024

garath Oct 7, 2024

garath Oct 7, 2024

garath Oct 7, 2024

davfost Oct 7, 2024

garath Oct 8, 2024

garath Oct 8, 2024 •

edited

Loading

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath Oct 8, 2024

garath left a comment

	// Provides a curtisy warning message if the ServiceTreeId option is set to a empty guid
	// Provides a courtesy warning message if the ServiceTreeId option is set to a empty guid

	// The default audit state should alwasy be failure and overwritten with success at the end of the operation.
	// The default audit state should always be failure and overwritten with success at the end of the operation.

	private ILogger ControlPanelLogger;
	private ILogger ControlPlaneLogger;

	// Audit logging is a 'volitile' operation meaning it can throw exceptions if logging fails.
	// Audit logging is a 'volatile' operation meaning it can throw exceptions if logging fails.

	// The token applicatoin id of the client running the assembly.
	// The token application id of the client running the assembly.

	/// The applicatoin ID for the credential provider.
	/// The application ID for the credential provider.

	/// Indictes if the global option for 'quiet' is set
	/// Indicates if the global option for 'quiet' is set

	/// The ID is a goid and is set to Guid.Empty if not set
	/// The ID is a guid and is set to Guid.Empty if not set

	/// Overides the GetOptions method from the base class to add a cusotm option for the ServiceTreeId
	/// Overrides the GetOptions method from the base class to add a custom option for the ServiceTreeId

	secretStoreType: "AzureKeyVault",
	secretStoreType: nameof(AzureKeyVault),

Add Audit Logging To SecreteManager.exe #4183

Are you sure you want to change the base?

Add Audit Logging To SecreteManager.exe #4183

Conversation

davfost commented Sep 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

garath Oct 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

garath left a comment

Choose a reason for hiding this comment

davfost commented Sep 27, 2024 •

edited

Loading

garath Oct 8, 2024 •

edited

Loading