Deployment errors in hello and nginx_openstack examples

[Furqan-Iqbal]

library/nginx/server/types.yaml:37:17: [PropertyDefinition] Invalid keys: value.

Hello example error
opera.error.ParseError: [PropertyDefinition] Invalid keys: value.

I will be thankful if someone help me in this regards.

[philippemerle]

This problem was resolved by #224.
You should certainly update your local xOpera installation.

[Furqan-Iqbal]

when you solved this. I just insatalled xopera today. secondly the SaaS login keaklock is also not working

[philippemerle]

when you solved this.

Nov 8, see 7018382

I just insatalled xopera today

Are you installed xopera with pip?
If yes, the xopera pip module should not be up-to-date, please wait for the next pip release or use xopera from the source code directly.

the SaaS login keaklock is also not working

You should contact the SaaS management.

[anzoman]

Hi @Furqan-Iqbal and thanks for opening this discussion.

So, I can see that you have tried to use our xOpera TOSCA orchestrator. And yes, the hello-world example gives and error regardless of the opera version you use. This is because we've been updating our xOpera examples (thanks to @philippemerle) in the last few weeks. We had to update them due to some remaining TOSCA v1.3 inconsistencies and it looks like that some of these issues also uncovered some bugs in our xOpera TOSCA parser. But no worries, we'll fix them as soon as possible and meanwhile you can browse older versions of xOpera examples (take a look here: xopera-opera and here: xopera-examples) that ca be deployed with the latest opera version 0.6.7.

And regarding the xOpera SaaS - I've seen you've already contacted us on our e-mail. We will respond to you shortly.

[anzoman]

Hi again @Furqan-Iqbal this issue has been fixed in 7018382 like @philippemerle stated.

So, until we release a new production opera version, you can try to use our latest development version from TestPyPI.

You can install it by running:

pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ opera==0.6.8.dev8

[anzoman]

This is now fully resolved in version 0.6.8.

[Furqan-Iqbal]

Hi Anze Luzar, I have a question about TOSCA could you forward your email address or I already send you request on LinkedIn. Best regards Furqan Iqbal

…

On Mon, Nov 15, 2021 at 6:46 PM Anže Luzar ***@***.***> wrote: Hi again @Furqan-Iqbal <https://github.com/Furqan-Iqbal> this issue has been fixed in 7018382 <7018382> like @philippemerle <https://github.com/philippemerle> stated. So, until we release a new production <https://pypi.org/project/opera/#history> opera version, you can try to use our latest development <https://test.pypi.org/project/opera/#history> version from TestPyPI. You can install it by running: pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ opera==0.6.8.dev8 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#228 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMKPLKEDD2YMHKC4THGUXK3UMFBPHANCNFSM5H66ETQQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[anzoman]

Hi @Furqan-Iqbal, we've already answered you from [email protected], so I think there's not need to elaborate this discussion.

[Furqan-Iqbal]

In my thesis I want to deploy Ansible with TOSCA. I found that xOpera will be the best solution for me. However, I have a few questions about that. In your video "How can the TOSCA practitioners benefit from xOpera project" you deployed the hello world example. How can a user access this in the localhost. Secondly , how to download or upload the CSAR file to xOpera. I am asking about xOpera installed in a python environment. 
It will be good if we can do a short meeting. I will be thankful to you. 

[anzoman]

Hi @Furqan-Iqbal I think that you will be just fine if you follow xOpera CLI documentation or README and deploy the hello-world example you mentioned. The extracted TOSCA CSARs and more examples can be found in xopera-examples repository and are also explained more in detail here in our docs.

[cankarm]

Thanks @anzoman . I see that I replied similar content to the email also. @Furqan-Iqbal, hopefully you will now be able to deploy something.

[Furqan-Iqbal]

Hi Xlab-Si/Xopera-Opera Team, I deployed the hello-world example and it made a text file in the temp folder. I want to access this hello world message by localhost. I tried a lot and it gives me an error message when I use localhost ip address or port etc. I will be thankful if someone helps me in this regard. Best regards Furqan Iqbal

…

On Mon, Dec 20, 2021 at 2:15 PM Mati(ja)c Cankar ***@***.***> wrote: Thanks @anzoman <https://github.com/anzoman> . I see that I replied similar content to the email also. @Furqan-Iqbal <https://github.com/Furqan-Iqbal>, hopefully you will now be able to deploy something. — Reply to this email directly, view it on GitHub <#228 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMKPLKEMSR3P26IUKFWMGUDUR4T5VANCNFSM5H66ETQQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[anzoman]

Hi @Furqan-Iqbal perhaps I don't fully understand your question, but you can access the result of that hello example right from your temporary folder (look for /tmp/playing-opera/hello/) and you don't have to change anything in the example because it already runs on localhost by default when you deploy it. Just do the following:

# clone the repository
(.venv) $ git clone [email protected]:xlab-si/xopera-opera.git
(.venv) $ cd xopera-opera/examples/hello
# deploy the example
(.venv) $ opera deploy service.yaml
[Worker_0]   Deploying my-workstation_0
[Worker_0]   Deployment of my-workstation_0 complete
[Worker_0]   Deploying hello_0
[Worker_0]     Executing create on hello_0
[Worker_0]   Deployment of hello_0 complete
# observe the outputs
(.venv) examples/hello$ ls -lh /tmp/playing-opera/hello/
total 0
-rw-rw-rw- 1 user user 0 Dec 10 16:02 hello.txt
(.venv) examples/hello$ cat /tmp/playing-opera/hello/hello.txt
default-marker

For more info you can follow our xOpera documentation or README.

More examples can be found in xopera-examples repository and are also explained in our docs.

I would also suggest that you take a look at the following videos that show how xOpera (and other tools) work in action:

• xOpera tools overview (from TOSCA webinar): https://youtu.be/NZLYWB9uxjk
• xOpera CLI: https://youtu.be/cb1efi3wnpw
• TPS with CLI: https://youtu.be/28eTwojw5ac
• TPS with Eclipse Che: https://youtu.be/vCjfZ4Iue0E
• xOpera SaaS with GUI: https://youtu.be/T4XviKWLc-A
• xOpera SaaS with Eclipse Che: https://youtu.be/SIiLOe5dSqc

[Furqan-Iqbal]

I know it create the file and i can access it by cat command. But I want to give the localhost ip 127.0.0.1 or my cloud vm ip and port and to access it outside the cLI. is it possible ?if yes how ? I tried but it always give me an error when i put port number or IP address. Nothing about this is mentioned in the documentation.

[anzoman]

So, xOpera allows you to connect and orchestrate things on a remote VM. To connect to you will need to use the OPERA_SSH_USER and OPERA_SSH_IDENTITY_FILE environment variables (continue here for more info). Also, FYI the workstation node from hello example is actually not needed and we'll probably delete it in the future

I also attach some additional info (from the similar issues in the past) below if you want to read it:

When you deploy a TOSCA service template with opera the orchestrator uses its Ansible executor and therefore prepares some kind of Ansible inventory so you don't have to prepare it yourself. If you are orchestrating on a localhost you are relieved of all connection problems but the things get a little more interesting if you want to execute the deployment on a remote VM. Opera and Ansible are by default using SSH protocol so if you are are able to access your remote machine via ssh user@your_ip that means that xOpera will be able to connect. But there's one additional tiny detail. To connect to a remote VM and deploy your TOSCA CSARs as a certain user you should set OPERA_SSH_USER environment variable before because otherwise opera will try to connect to a VM as a default user which is currently set to centos. So for example if you want to connect as a root user you can try to run the deployment with OPERA_SSH_USER=root opera deploy -i inputs.yml service.yaml and you are good to go. Also when connecting SSH keys can be a problem. Configuring "passwordless" SSH access will solve your problems here (but it's possible that you will also need to update your ~/.ssh/config file if you have too many keys). If you have to many SSH keys and you don't want to modify ~/.ssh/config you can use OPERA_SSH_IDENTITY_FILE env var and point it to your private SSH key.

The OPERA_SSH_USER currently offers just one ssh connection for all nodes. If you want to connect to multiple ssh hosts, you can do this distinction within Ansible playbooks where you can set some Ansible variables and also some TOSCA attributes (public_address and private_address) like in this example here, where we create a new Ansible inventory for the ssh connection to the previously created AWS EC2 (this node is then also used as a host for other nodes). If you want to connect multiple nodes to the same host/VM via ssh you will have to make sure that your remote VM is defined as a separate node which is then connected to all the nodes as a host (with HostedOn TOSCA requirement) like here. You can see that the vm node (where we set private_address and public_address attributes) is a host for nginx node and then the host IP is also applied to this node.

In the future we plan to reorganize or remove the aforementioned env vars for easier orchestration (maybe we'll start using some kind of opera config file that could be placed in .opera/config).

[Furqan-Iqbal]

Thanks for the detail replay. I tried it but wasn't successful maybe it is due to Hello-world example is creating the file in temp directory and I am not able to access it in the local host.

I will be very thankful if someone give me few minutes on call to discuss it.

[Furqan-Iqbal]

Dear Xopera Team, I am deploying the hello world program on the VM. When I deploy it shows the message deployed but there is no file in the VM. However, when I undeploy it, it gives me an error message. I have set up a public key, private key and test the connection and also send the files from localhost to VM. There is no permission denied problem. But in the xopera the error message is permission is denied. Please help me in this regard. Here is the logfile of the undeploy command. (venv) ***@***.***:~/PycharmProjects/testing/xopera-opera/examples/hello$ OPERA_SSH_USER=ubuntu opera undeploy The instance model already exists. Use --resume/-r option to continue current undeployment process. (venv) ***@***.***:~/PycharmProjects/testing/xopera-opera/examples/hello$ OPERA_SSH_USER=ubuntu opera undeploy -r The resume undeploy option might have unexpected consequences on the already undeployed blueprint. Do you want to continue? (Y/n): y [Worker_0] Undeploying hello_0 [Worker_0] Executing delete on hello_0 [Worker_0] ------------ { "custom_stats": {}, "global_custom_stats": {}, "plays": [ { "play": { "duration": { "end": "2022-01-04T22:31:36.495355Z", "start": "2022-01-04T22:31:36.425969Z" }, "id": "f356307d-9a3c-4ad7-311e-000000000006", "name": "all" }, "tasks": [ { "hosts": { "opera": { "action": "file", "changed": false, "msg": "Failed to connect to the host via ssh: ***@***.***: Permission denied (publickey,password).", "unreachable": true } }, "task": { "duration": { "end": "2022-01-04T22:31:36.495355Z", "start": "2022-01-04T22:31:36.430824Z" }, "id": "f356307d-9a3c-4ad7-311e-000000000008", "name": "Remove the location" } } ] } ], "stats": { "opera": { "changed": 0, "failures": 0, "ignored": 0, "ok": 0, "rescued": 0, "skipped": 0, "unreachable": 1 } } } [Worker_0] ------------ [Worker_0] ============ Traceback (most recent call last): File "/home/furqan/PycharmProjects/testing/venv/bin/opera", line 8, in <module> sys.exit(main()) File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/cli.py", line 61, in main return args.func(args) File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/commands/undeploy.py", line 75, in _parser_callback undeploy(storage, args.verbose, args.workers) File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/commands/undeploy.py", line 111, in undeploy topology.undeploy(verbose_mode, workdir, num_workers) File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/instance/topology.py", line 69, in undeploy do_undeploy = executor.wait_results() File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/threading/node_executor.py", line 33, in wait_results errors = self.process_results(results) File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/threading/node_executor.py", line 57, in process_results future.result() File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__get_result() File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/instance/node.py", line 163, in undeploy self.run_operation(OperationHost.HOST, StandardInterfaceOperation.shorthand_name(), File "/home/furqan/PycharmProjects/testing/venv/lib/python3.8/site-packages/opera/instance/base.py", line 85, in run_operation raise OperationError("Failed") opera.error.OperationError: Failed Best regards Furqan Iqbal

…

On Tue, Dec 28, 2021 at 12:39 PM Anže Luzar ***@***.***> wrote: So, xOpera allows you to connect and orchestrate things on a remote VM. To connect to you will need to use the OPERA_SSH_USER and OPERA_SSH_IDENTITY_FILE environment variables (continue here <https://xlab-si.github.io/xopera-docs/cli.html#secrets-and-environment-variables> for more info). Also, FYI the workstation node from hello example is actually not needed and we'll probably delete it in the future I also attach some additional info (from the similar issues in the past) below if you want to read it: When you deploy a TOSCA service template with opera the orchestrator uses its Ansible executor and therefore prepares some kind of Ansible inventory so you don't have to prepare it yourself. If you are orchestrating on a localhost you are relieved of all connection problems but the things get a little more interesting if you want to execute the deployment on a remote VM. Opera and Ansible are by default using SSH protocol so if you are are able to access your remote machine via ssh ***@***.***_ip that means that xOpera will be able to connect. But there's one additional tiny detail. To connect to a remote VM and deploy your TOSCA CSARs as a certain user you should set OPERA_SSH_USER environment variable before because otherwise opera will try to connect to a VM as a default user which is currently set to centos. So for example if you want to connect as a root user you can try to run the deployment with OPERA_SSH_USER=root opera deploy -i inputs.yml service.yaml and you are good to go. Also when connecting SSH keys can be a problem. Configuring "passwordless" SSH access will solve your problems here (but it's possible that you will also need to update your ~/.ssh/config file if you have too many keys). If you have to many SSH keys and you don't want to modify ~/.ssh/config you can use OPERA_SSH_IDENTITY_FILE env var and point it to your private SSH key. The OPERA_SSH_USER currently offers just one ssh connection for all nodes. If you want to connect to multiple ssh hosts, you can do this distinction within Ansible playbooks where you can set some Ansible variables and also some TOSCA attributes (public_address and private_address) like in this example here <https://github.com/xlab-si/xopera-examples/blob/master/cloud/aws/thumbnail-generator-with-vm/modules/ec2_vm/playbooks/create.yaml#L33-L61>, where we create a new Ansible inventory for the ssh connection to the previously created AWS EC2 (this node is then also used as a host for other nodes). If you want to connect multiple nodes to the same host/VM via ssh you will have to make sure that your remote VM is defined as a separate node which is then connected to all the nodes as a host (with HostedOn TOSCA requirement) like here <https://github.com/xlab-si/xopera-opera/blob/master/examples/nginx_openstack/service.yaml>. You can see that the vm node (where we set private_address and public_address attributes) is a host for nginx node and then the host IP is also applied to this node. In the future we plan to reorganize or remove the aforementioned env vars for easier orchestration (maybe we'll start using some kind of opera config file that could be placed in .opera/config). — Reply to this email directly, view it on GitHub <#228 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMKPLKCG4TJQDQM4RHN7HWDUTGOYBANCNFSM5H66ETQQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[anzoman]

Deployed status does not mean that the instance is really there because xOpera will not magically check if the file exists, it just means that the deployment process went through. And you can always delete .opera storage folder or use -rf switches and redeploy.

Also from your output Failed to connect to the host via ssh: ***@***.***: Permission denied (publickey,password).", it seems that you have some problems with SSH connection. Check first if you can ssh just with ssh ubuntu@<your-ip> without password and without having to specify the private key file with -i. If not you will have to either use OPERA_SSH_IDENTITY_FILE env var or edit ~/.ssh/config file to be able to connect.

[Furqan-Iqbal]

Thanks for your reply. As I mentioned before, I have checked and ssh ***@***.*** address id worked. I have used OPERA_SSH_IDENTITY_FILE as an environment variable but the same result. In pycharm i tested the connection in ssh configuration and also worked. I will be really thankful if you can give me 10 minutes and I can show you the live program. I also attached the screenshots of playbook and service.yaml files Best regards Furqan Iqbal

…

On Wed, Jan 5, 2022 at 11:05 AM Anže Luzar ***@***.***> wrote: Deployed status does not mean that the instance is really there because xOpera will not magically check if the file exists, it just means that the deployment process went through. And you can always delete .opera storage folder or use -rf switches and redeploy. Also from your output Failed to connect to the host via ssh: ***@***.***: Permission denied (publickey,password).", it seems that you have some problems with SSH connection. Check first if you can ssh just with ssh ubuntu@<your-ip> without password and without having to specify the private key file with -i. If not you will have to either use OPERA_SSH_IDENTITY_FILE env var or edit ~/.ssh/config file to be able to connect. — Reply to this email directly, view it on GitHub <#228 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMKPLKBPB3PWCO7ZH2AHQ6TUUQJXPANCNFSM5H66ETQQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[anzoman]

@Furqan-Iqbal can you paste the TOSCA CSAR (archive) you're using here?

[Furqan-Iqbal]

Sorry what do you mean by CSAR archive here. I am not able to see any CSAR file in the xopera example repository
here is the TOSCA Service.yaml file:

tosca_definitions_version: tosca_simple_yaml_1_3

metadata:
  template_name: "hello-world"
  template_author: "XLAB"
  template_version: "1.0"

node_types:
  hello_type:
    derived_from: tosca.nodes.SoftwareComponent
    interfaces:
      Standard:
        inputs:
          marker:
            value: { get_input: marker }
            type: string
        operations:
          create: playbooks/create.yaml
          delete: playbooks/delete.yaml

topology_template:
  inputs:
    marker:
      type: string
      default: default-marker

  node_templates:
    my-workstation:
      type: tosca.nodes.Compute
      attributes:
        private_address: <my-vm-ip>
        public_address: <my-vm-ip>

    hello:
      type: hello_type
      requirements:
        - host: my-workstation

Create Ansible playbook :

---
- hosts: <my-vm-ip>
  gather_facts: false

  tasks:
    - name: Make the location
      file:
        path: /home/student/ansible/hello
        recurse: true
        state: directory

    - name: Ansible was here
      copy:
        dest: /home/student/ansible/hello/hello.txt
        content: "{{ marker }}"

[anzoman]

@Furqan-Iqbal TOSCA CSAR = zip archive with service template and other TOSCA modules and usually with TOSCA.meta file.

Your files look okay, but try to set hosts: all in your Ansible playbooks like in the original example.

[Furqan-Iqbal]

Hi, I already tried with hosts: all same errors. I am using the xopera github repository and I didn't find any TOSCA.meta file. Although I made a zip file of my project. Find attached to an email the file. It will be good if I can show it to you live. Best regards Furqan Iqbal

…

On Fri, Jan 7, 2022 at 2:14 PM Anže Luzar ***@***.***> wrote: @Furqan-Iqbal <https://github.com/Furqan-Iqbal> TOSCA CSAR = zip archive with service template and other TOSCA modules and usually with TOSCA.meta file. Your files look okay, but try to set hosts: all in your Ansible playbooks like in the original example. — Reply to this email directly, view it on GitHub <#228 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMKPLKB2HTFYSVUPUNSSCG3UU3RJXANCNFSM5H66ETQQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[cankarm]

Hi @Furqan-Iqbal

you can take CSAR from the template library (https://template-library-xopera.xlab.si/services). There are .csar files or zip-s. For each CSAR you can change the extension (suffix) to zip, and you will be able to unpack it. In more complex CSARs, the TOSCA.meta is there to explain which is the initial yaml ->where there is not only a single service.yaml

[anzoman]

@Furqan-Iqbal I don't see the zip file. Can you attach it here (but erase your IPs and any personal data)?.

[Furqan-Iqbal]

Yes, but here is all the stuff and everything is ok as you told me but why the hello-world not running on VM. All the things are the same according to your example. Please help me

…

On Fri, Jan 7, 2022 at 3:44 PM Mati(ja)c Cankar ***@***.***> wrote: Hi @Furqan-Iqbal <https://github.com/Furqan-Iqbal> you can take CSAR from the template library ( https://template-library-xopera.xlab.si/services). There are .csar files or zip-s. For each CSAR you can change the extension (suffix) to zip, and you will be able to unpack it. In more complex CSARs, the TOSCA.meta is there to explain which is the initial yaml ->where there is not only a single service.yaml — Reply to this email directly, view it on GitHub <#228 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMKPLKD57VRXNJGODPBLOLDUU335LANCNFSM5H66ETQQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>