Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix edge cases such as disable istio-inject for Tekton compiler #119

Merged
merged 3 commits into from
Apr 25, 2020

Conversation

Tomcli
Copy link
Member

@Tomcli Tomcli commented Apr 24, 2020

Which issue is resolved by this Pull Request:
Resolves #114

Description of your changes:

  • Add new annotation to disable istio-inject
  • Remove literal string conversion for pyyaml since there could be conflicts with yaml 1.1 and 1.2 on some edge cases. If we want this feature back then we have to use ruamel.yaml. We are using pyyaml right now because we need to be consistent with the kfp package.
  • Add headers to golden test yaml
  • Fix minor syntax bugs

Environment tested:

  • Python Version (use python --version): 3.6.4
  • Tekton Version (use tkn version): 0.11.3
  • Kubernetes Version (use kubectl version):
  • OS (e.g. from /etc/os-release):

@kubeflow-bot
Copy link

This change is Reviewable

Copy link
Member

@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Tomcli -- I have a few questions :-)

@@ -236,6 +237,23 @@ def _test_pipeline_workflow(self,
compiled = list(yaml.safe_load_all(f))
if GENERATE_GOLDEN_YAML:
with open(golden_yaml_file, 'w') as f:
f.write(textwrap.dedent("""\
Copy link
Member

@ckadner ckadner Apr 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good idea to add the license header! Can you move this text into a variable at the top of the file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -36,18 +36,6 @@
from .. import tekton_api_version


def _literal_str_representer(dumper, data):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot the history of this, can you quickly explain why we had to add this and why it's no longer needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to add literal style to make the script part looks nicer. But there's some edge cases with pyyaml when using it on integer since it's designed with yaml 1.1

Here's the definition of literal style in yaml
https://yaml.org/spec/1.2/spec.html#id2795688

'name': 'copy-results',
'script': '#!/bin/sh\nset -exo pipefail\n'
}
volume_mount_step_template = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aren't those still being used in the code below?

# initial base_step and volume setup
base_step = {
'image': 'busybox',
'name': 'copy-results',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still see the copy-results step in the compiled YAML and in a comment, does that come from somewhere else?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved this into a different function, but somehow it get put back after the merge. We no longer need this over here because it is initiated at the beginning of this function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, thanks :-)

if GENERATE_GOLDEN_YAML:
with open(golden_yaml_file, 'w') as f:
f.write(textwrap.dedent("""\
license_header = textwrap.dedent("""\
Copy link
Member

@ckadner ckadner Apr 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant move license_header to the top of the compiler_tests.py file :-) ... since Andrew will have to use that in his PR as well and it blows up the code too much here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh you mean set it as global variable?

@@ -27,6 +27,24 @@
# in order to generate new "golden" YAML files
GENERATE_GOLDEN_YAML = False

# License header for Kubeflow project
LICENSE_HEADER = textwrap.dedent("""\
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, thanks :-)

@ckadner
Copy link
Member

ckadner commented Apr 25, 2020

/lgtm

@ckadner
Copy link
Member

ckadner commented Apr 25, 2020

/approve

@k8s-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ckadner

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 4f6a564 into kubeflow:master Apr 25, 2020
k8s-ci-robot pushed a commit that referenced this pull request Apr 27, 2020
* Add unit tests

* Update condition.yaml

* Update test_kfp_samples.sh for new test_util.py

* Add results logging and refactor

* Refactor and add licenses

* Add license

* Add __main__ to introduced testdata

* Adjust new testdata for #119

* Add and edit doc string

* Fix golden yaml
@Tomcli Tomcli deleted the disable-istio branch May 20, 2020 17:20
HumairAK referenced this pull request in red-hat-data-services/data-science-pipelines-tekton May 16, 2023
…error

Bugfix: Benign modified DSPA object error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Docs for deploying tekton pipeline on kfp user namespaces.
4 participants