Use official parser

[jirikuncar]

Is there an interest in switching to the official Gherkin parser: https://github.com/cucumber/cucumber/tree/master/gherkin/python?

I would be willing to prepare a PR.

Cheers!

[The-Compiler]

As far as I know, @olegpidsadnyi started some work on this (or at least planned to) - see e.g. #366 (comment)

[anniepapa]

@The-Compiler @olegpidsadnyi Do you have any idea when will be the next release of Pytest-bdd? I was asked twice already from company guild meeting about this. Very much looking forward to it. 😄

[youtux]

I actually abandoned the idea (for the moment) of using the parser from cucumber. I am trying instead to rewrite the parser.
You can see the progress in the rewrite-parser branch.

[elchupanebrej]

@youtux could you please provide some insights into why rewriting was selected over using the official parser?

Maybe it's possible to create some intermediate adapter layer(or model) to make possible support a few different parsers?
Seems that pytest-bdd Gherkin dialect and official Gherkin have some independent features. pytest-bdd has much less community than Gherkin so it is vitally necessary to support official features and just after that support extensions. Seems that it's needed to give some integration space inside. So next evolution steps could be:

• Separate intermediate adapter model (and prove that legacy parser could work with it)
• Develop adapter model to connect model provided by the official parser and intermediate model
• Extend official parser to support features provided by pytest-bdd
• Extend pytest-bdd run model to support features from the official Gherkin dialect through the adapter model

[TBBle]

It looks like this has become the blocker for Datatables, i.e. #366 (comment), but the rewrite-parser branch also stalled in September 2020, over twelve months ago.

Is there still seen to be a way forward in this direction (replacing the parser, either a rewrite or integrating the Gherkin python parser), or do we need to resuscitate #366 against current pytest-bdd master in order to advance that support?

(Or perhaps point people at external addons like sttable although I don't know how well-maintained that is, as I haven't actually tried it.)

[theObserver1]

I dont mind about the parser but i want pretty data tables in my CI reports :) . We use sttable as a workaround and it works perfectly except its not supported by CI reporting tools. Recommend sttable is officially incorporated into pytest-bdd as a quick win so downsteam tools can also be updated. Its really nothing more than a standard Python dict and list parsing so should be fairly maintenance free.

[youtux]

For those interested, lately I've been working on a new parser in the branch https://github.com/pytest-dev/pytest-bdd/tree/tatsu-parser.
It is more strict and it adheres to the Gherkin specifications. It parses data tables and step docstrings, although they still have to be implemented in the runner.

It is almost complete, it needs polishing around error reporting and documenting the changes.

[elchupanebrej]

I have integrated the official parser into pytest bdd, it still needs some polishing, but it gives a possibility to integrate with tools from cucumber.io

[jugaadi]

Any plans to support Markdown with Gherkin ?

[nhuray]

@elchupanebrej Any plans to reintegrate the work you did in pytest-bdd-ng into pytest-bdd ?

[youtux]

To be clear, @elchupanebrej is not part of the pytest-bdd team. So no, there is no plan to integrate the changes of pytest-bdd-ng into pytest-bdd.

[judgeaxl]

Since the pytest-bdd-ng work is not being considered, how's progress on the local attempts?

[youtux]

I haven't had any time in the last period to work on this unfortunately.

[blaisep]

Hi @youtux , et al.

TL;DR: A path toward full cucumber support

I contribute to the upstream open-cucumber project and I'm one of the few pythonistas there.
I have good news and bad news.

Since Pytest-BDD has drifted behind Cucumber core and the cucumber team doesn't build a python implementation, there are several components (like the message parser and the cucumber expressions) which require a big lift to reproduce in the current state of pytest-bdd.

Bad news:

The current "macro-economic conditions" have reduced the number of resources in the open cucumber team. We don't anticipate being able to produce a python implementation. We would like to contribute to pytest-bdd and have that be the reference cucumber implementation. (We already spoke to Jens and he decided that behave is not a suitable candidate. When I spoke with @elchupanebrej a few months ago, he said he was not able to determine how to contribute to pytest-dev and he's not sure that he can support pytest-bdd-ng on his own.

Good news

Some of us want to improve the python support I have several people interested in refactoring Pytest-BDD so that it is in sync with upstream. In fact, I can get assistance from the upstream team with acceptance testing.

Next Steps

Ideally, we would like to learn how to contribute to pytest-dev so that we can refactor the work @elchupanebrej has done and also keep up with new upstream developments.

see also:
https://github.com/pytest-dev/pytest-bdd/milestone/1
#180
https://github.com/pytest-dev/pytest-bdd/tree/rewrite-parser
#306 (comment)
#596

• @h-joshi , @PatrickMassot , @theObserver1 , @BryanEddy5 , @aBulgakoff , @nayana-prashanth , @TBBle , @jirikuncar , @okken

[The-BDD-Coach]

@elchupanebrej has already added the official Cucumber Gherkin parser in his fork of Pytest-BDD; it already supports data tables, multiple Examples tables, 'Rule'...

You can see a comparison in the tutorial: https://thebddcoach.com/post/a-quick-introduction-to-pytest-bdd-ng-for-people-who-are-already-familiar-with-cucumber-or-behave/

If we can merge the two projects it would be a big win for everyone.

[youtux]

Hi @blaisep, I talked also to @olegpidsadnyi and this sounds like a great opportunity to join effort and improve pytest-bdd to get all the features of cucumber and become the reference implementation for python!

I started some time ago rewriting the parser in order to enable all the "new" features (datatables, i18n support, ...), but unfortunately I never had enough time to complete it.

At the time, I didn't want to rely on the gherkin-official parser library as it didn't seem to have an API specification, nor to define the lifecycle of supported python versions.

But if we could have the support of the cucumber team and make sure that our requirements and specs are aligned, then I'm all up to use the official parser library!

That being said, I don't have too much time currently to work on integrating the parser, but if you or anybody would like to work on it, I'll be glad to give directions and review code to get it merged in pytest-bdd.

How does that sound?

[The-BDD-Coach]

@youtux, @elchupanebrej has already integrated the official parser into Pytest-BDD-NG, so it shouldn't be difficult to merge those changes into Pytest-BDD.

[jsa34]

This is now done in #699. Should we close this issue, @youtux ?