Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix TypeParser + TypeLexer #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

instabledesign
Copy link

Hi,
This gonna to fix 1 wrong doctrine/lexer usage.
The wrong usage was located in the TypeLexer

    protected function getCatchablePatterns()
    {
        return [
            '\'(?:[^\']|\'\')*\'',
            '([a-z0-9\\\\]+)', // <----- You must not use parenthesis here
        ];
    }

You must not use group capture (parenthesis) because this regex was use in preg_split function
For example with array<key=int, value=string>
if you use ([a-z0-9\\\\]+) you get 15 tokens in the lexer

Array
(
    [0] => Array
        (
            [value] => array
            [type] => 2
            [position] => 0
        )
    [1] => Array
        (
            [value] => array
            [type] => 2
            [position] => 0
        )
    [2] => Array
        (
            [value] => <
            [type] => 5
            [position] => 5
        )
    [3] => Array
        (
            [value] => key
            [type] => 2
            [position] => 6
        )
    [4] => Array
        (
            [value] => key
            [type] => 2
            [position] => 6
        )
    [5] => Array
        (
            [value] => =
            [type] => 7
            [position] => 9
        )
    [6] => Array
        (
            [value] => int
            [type] => 2
            [position] => 10
        )
    [7] => Array
        (
            [value] => int
            [type] => 2
            [position] => 10
        )
    [8] => Array
        (
            [value] => ,
            [type] => 6
            [position] => 13
        )
    [9] => Array
        (
            [value] => value
            [type] => 2
            [position] => 15
        )
    [10] => Array
        (
            [value] => value
            [type] => 2
            [position] => 15
        )
    [11] => Array
        (
            [value] => =
            [type] => 7
            [position] => 20
        )
    [12] => Array
        (
            [value] => string
            [type] => 2
            [position] => 21
        )
    [13] => Array
        (
            [value] => string
            [type] => 2
            [position] => 21
        )
    [14] => Array
        (
            [value] => >
            [type] => 4
            [position] => 27
        )
)

and if you use [a-z0-9\\\\]+ you get 10 tokens in the lexer

Array
(
    [0] => Array
        (
            [value] => array
            [type] => 2
            [position] => 0
        )
    [1] => Array
        (
            [value] => <
            [type] => 5
            [position] => 5
        )
    [2] => Array
        (
            [value] => key
            [type] => 2
            [position] => 6
        )
    [3] => Array
        (
            [value] => =
            [type] => 7
            [position] => 9
        )
    [4] => Array
        (
            [value] => int
            [type] => 2
            [position] => 10
        )
    [5] => Array
        (
            [value] => ,
            [type] => 6
            [position] => 13
        )
    [6] => Array
        (
            [value] => value
            [type] => 2
            [position] => 15
        )
    [7] => Array
        (
            [value] => =
            [type] => 7
            [position] => 20
        )
    [8] => Array
        (
            [value] => string
            [type] => 2
            [position] => 21
        )
    [9] => Array
        (
            [value] => >
            [type] => 4
            [position] => 27
        )
)

so i fix the TypeParser::walk() function in order to drop the foreach (currently was a workaround to skip the token doublon capture by the regex parenthesis)

I'm not happy to set a lot of ->walk() call everywhere in the TypeParser but they are needed.

This PR is related to doctrine/lexer#12 (comment)
I've also tested with https://github.com/doctrine/lexer/pull/12/files#diff-3945e835e8d3a0ad8409023030a9db04R262

@instabledesign
Copy link
Author

Tests fail look unrelated.

@instabledesign
Copy link
Author

@egeloen can you check plz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant