Failure cases handling

[pigmej]

Currently we support only success path, we may need to support failure path too.

Exact scope of this issue is TDB.

[nkwangleiGIT]

hi pigmej, do we have some plan or proposal for this requirement? we want to use AppController for workflow tasks (k8s jobs), thanks.

[pigmej]

@nkwangleiGIT Currently @gluke77 is doing research about that, how we can support that etc. We will have document for that somewhere in the middle of this week.

@nkwangleiGIT Could you describe your use case? It would be cool to include it in @gluke77 research around that area.

[zen]

Basically, we have an idea to let user decide what to do with failures, and provide him with several options. What we plan for first iteration of this feature is:

• retires with timeouts (configurable)
• rollback
• tear down on failure, e.g. entire graph will be destroyed after failure

In the future we plan to support more sophisticated scenarios, when you can attach custom graphs on different failures. This would allow more elegant tear down or alternative graph execution.

Anyway, it would be great to see your use cases @nkwangleiGIT , we could consider them in our design

[nkwangleiGIT]

@pigmej @zen thanks so much for your reply.
About my use case, here are some basic info:

1. We're trying to create build-flow(CI) using k8s Jobs, so it's quite like the approach in your demo:
   https://youtu.be/BXRToNV4Rdw?t=178
   Thanks again for the great demo.
2. Basically, it'll do similar flow like shippable but using k8s Jobs for each build step.
   http://docsv2.readthedocs.io/en/latest/config.html#test-and-code-coverage-visualization
3. So we need to handle the failure case of each step, including timeout, retry times, remove the latter steps after the failure point. Or even use job as the decision gateway, and make decision based on the output of job(low priority for now)
4. For long term, it'll be great if we can use k8s to do workflow or bpm.
   BTW, we also expect AppController to support Deployment, so we can use the Dependency for application deployment and visualize these scenarios.

We'd like to contribute this project once we get familiar with this great tool, it'll be also great if you can send me some more documents to get start besides the ones in this repository, thanks!

[gluke77]

@nkwangleiGIT Ptal #98. If you think that the document does not cover all your usecases, pls let me know.

[nkwangleiGIT]

@gluke77 it's sufficient for our current requirement, when we can get the 1st version implementation about this?
BTW, how we should handle the events about failure? or we have to watch related k8s API for this?
Thanks

[gluke77]

@nkwangleiGIT As for implementation ETA, well the only answer that I could give is "we want it rather sooner then later". As for externally available events -- we don't have any plans yet.

[pigmej]

@nkwangleiGIT or maybe you meant more or less some post deployment events ? And handling these errors? If yes then it's also being worked on currently.

[nkwangleiGIT]

@gluke77 got it, I'll keep a close eye on this issue and AppController project
@pigmej yes, events to add custom actions or integrate with other service will be great to simplify the work
Thanks all

[pigmej]

General handling landed, specific cases moved to next milestone.

[gluke77]

So far retries (w/o timeouts) #174 and 'on-error' conditional subgraphs #184 are implemented. Timeout support #199 is in progress. 'Ignore this' and 'ignore with children' will follow shortly.

[gluke77]

'Ignore this" implemented #205

[pigmej]

Tear down moved to: #210
Rollback moved to: #209

Other items are merged.