Adding Credential Input Source Export for Exporting Credentials using awxkit

[neevnuv]

SUMMARY

When exporting credentials using awxkit, if the credential had input sources, they would not be exported.
This PR will add the ability to export those values and import them afterwards. #14738

ISSUE TYPE

• New or Enhanced Feature

COMPONENT NAME

• Collection
• CLI

AWX VERSION

23.6.0

ADDITIONAL INFORMATION

Relates to the issue #14738 

[neevnuv]

Hello, I finished working on this PR.
Would love for a review, I am not sure if to keep as a draft or not.

[jbradberry]

@neevnuv Ok, here's what I think. After sitting down and experimenting with an input source for a bit, I see that the related endpoint off of api/v2/credential/<id>/ is a bit odd and won't fit into the framework of DEPENDENT_EXPORT without refactoring how that all works or introducing another edge case like we did with workflow approvals.

Instead, I think you should just add CredentalInputSource to the EXPORTABLE_RESOURCES list, and give the page type a proper NATURAL_KEY. This information ought to tell you what the natural key should be: https://github.com/ansible/awx/blob/devel/awx/main/models/credential/__init__.py#L1243

Yes, this has the downside that users will need to remember to explicitly export their credential input sources (unless they are exporting everything), but I think it will be simpler all around.

[neevnuv]

As of now, I made it possible to access the credential_input_sources using:

awx export --credential_input_sources

and allow the credentials that use input sources to export them when using:

awx export --credential

Now I am researching on payload functions and how they are used.

[jbradberry]

You probably don't want this here.

[neevnuv]

won't this allow to export the credential input sources when imoprting a credential with an input source?

[jbradberry]

No. As I explained before, DEPENDENT_EXPORT doesn't quite work with how this relationship is being exposed in the API. You'll have to leave this out, and in order to correctly export a user will need to do both --credentials and --credential_input_sources.

[fosterseth]

I keep getting this when importing

Traceback (most recent call last):
  File "/home/sbf/awx/awxkit/awxkit/cli/__init__.py", line 25, in run
    cli.parse_resource()
  File "/home/sbf/awx/awxkit/awxkit/cli/client.py", line 152, in parse_resource
    self.resource = parse_resource(self, skip_deprecated=skip_deprecated)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sbf/awx/awxkit/awxkit/cli/resource.py", line 222, in parse_resource
    response = command.handle(client, parser)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sbf/awx/awxkit/awxkit/cli/resource.py", line 145, in handle
    client.v2.import_assets(data)
  File "/home/sbf/awx/awxkit/awxkit/api/pages/api.py", line 422, in import_assets
    for resource in self._dependent_resources():
  File "/home/sbf/awx/awxkit/awxkit/api/pages/api.py", line 267, in _dependent_resources
    for page_cls in itertools.chain(*has_create.page_creation_order(*data_pages)):
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sbf/awx/awxkit/awxkit/api/mixins/has_create.py", line 96, in page_creation_order
    actual_order = separate_async_optionals(order)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sbf/awx/awxkit/awxkit/api/mixins/has_create.py", line 68, in separate_async_optionals
    if dependency in compared.optional_dependencies:
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: type object 'CredentialInputSource' has no attribute 'optional_dependencies'

any idea what is going on?

[jbradberry]

@neevnuv The above traceback is because you don't have the HasCreate mixin class as one of the bases for your new page type. Please include everything in api/mixins/ that seems relevant for this type of API endpoint.

[neevnuv]

Thanks for the feedback, I added only HasCreate.

Reasons:
HasInstanceGroups - credential_input_sources don't have instanceGroups.
HasNotifications - credential_input_sources don't have notifications.
HasStatus - credential_input_sources don't have status.
HasSurvey - credential_input_sources don't have survies.
HasVariables - credential_input_sources don't have inventories variables.
HasCopy - doesn't have an option to copy.

[jbradberry]

cc @fosterseth

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[lukewestervelt]

@fosterseth @jbradberry @djyasin

I'm currently encountering the same issue (#14738) with TSS SecureVault secrets. AWX instance credentials are setup to pull username/passwords from TSS SecureVault instead of storing in plaintext/encrypted in the username/password fields.

Was going through export/import AWX process, and attempting to inject secrets back into credentials during the import process. Attempted to add the TSS JSON inside of the inputs block like so:

      "inputs": {
        "password": {
          "secret_field": "password",
          "secret_id": "#####"
        },
        "username": {
          "secret_field": "username",
          "secret_id": "#####"
        }
      },

But met with this error message:
ERROR:awxkit.api.pages.api:/api/v2/credentials/ "GitHub PAT": Bad Request (400) received

After diving into the AWX instance with credentials objects that are setup correctly, inputs and input_sources block are missing even before export/import is attempted:

// using awx-manage shell_plus for below- https://github.com/ansible/awx/blob/989a4387df6cf802a0f00c0251669ff6116889c0/docs/credentials/extract_credentials.md?plain=1#L23C1-L23C105

print(Credential.objects.get(name="GitHub PAT").inputs)
{}
print(Credential.objects.get(name="GitHub PAT").input_sources)
main.CredentialInputSource.None

Seems to me that this PR would possibly address this issue. Are there plans to merge this? Any other guidence would be much appreciated, thanks!

[jbradberry]

@lukewestervelt I'm afraid I'm no longer on this team and have no further say in what happens on this project. Hopefully @fosterseth will be able to make that call. My recollection was that this PR is ready or close to it.