Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[red-knot] complete expression type inference #12701

Open
7 of 21 tasks
carljm opened this issue Aug 5, 2024 · 6 comments
Open
7 of 21 tasks

[red-knot] complete expression type inference #12701

carljm opened this issue Aug 5, 2024 · 6 comments
Labels
help wanted Contributions especially welcome red-knot Multi-file analysis & type inference

Comments

@carljm
Copy link
Contributor

carljm commented Aug 5, 2024

  • number literal expression
  • string literal expression
  • bytes literal expression
  • f-string expression
  • ellipsis literal expression
  • generator expression (blocked on generics)
  • list comprehension (blocked on generics)
  • dict comprehension (blocked on generics)
  • set comprehension (blocked on generics)
  • lambda expression (blocked on Callable type)
  • call expression
  • starred expression (should be done as part of args/kwargs support)
  • yield expression (part of async)
  • yield-from expression (part of async)
  • await expression (part of async)
  • unary expression
  • binary expression
  • boolean expression
  • [red-knot] Compare expressions #13618
  • [red-knot] Infer subscript expression types #13689
  • slice expression
@carljm carljm added the red-knot Multi-file analysis & type inference label Aug 5, 2024
carljm pushed a commit that referenced this issue Aug 22, 2024
## Summary

This PR adds the `bytes` type to red-knot:
- Added the `bytes` type
- Added support for bytes literals
- Support for the `+` operator

Improves on #12701 

Big TODO on supporting and normalizing r-prefixed bytestrings
(`rb"hello\n"`)

## Test Plan

Added a test for a bytes literals, concatenation, and corner values
carljm pushed a commit that referenced this issue Aug 26, 2024
## Summary

Introduce a `StringLiteralType` with corresponding `Display` type and a
relatively basic test that the resulting representation is as expected.

Note: we currently always allocate for `StringLiteral` types. This may
end up being a perf issue later, at which point we may want to look at
other ways of representing `value` here, i.e. with some kind of smarter
string structure which can reuse types. That is most likely to show up
with e.g. concatenation.

Contributes to #12701.

## Test Plan

Added a test for individual strings with both single and double quotes
as well as concatenated strings with both forms.
carljm pushed a commit that referenced this issue Aug 26, 2024
# Summary

Add support for the first unary operator: negating integer literals. The
resulting type is another integer literal, with the value being the
negated value of the literal. All other types continue to return
`Type::Unknown` for the present, but this is designed to make it easy to
extend easily with other combinations of operator and operand.

Contributes to #12701.

## Test Plan

Add tests with basic negation, including of very large integers and
double negation.
carljm added a commit that referenced this issue Aug 30, 2024
## Summary

- Introduce methods for inferring annotation and type expressions.
- Correctly infer explicit return types from functions where they are
simple names that can be resolved in scope.

Contributes to #12701 by way of helping unlock call expressions (this
does not remotely finish that, as it stands, but it gets us moving that
direction).

## Test Plan

Added a test for function return types which use the name form of an
annotation expression, since this is aiming toward call expressions.
When we extend this to working for other annotation and type expression
positions, we should add explicit tests for those as well.

---------

Co-authored-by: Alex Waygood <[email protected]>
Co-authored-by: Carl Meyer <[email protected]>
@carljm carljm added the help wanted Contributions especially welcome label Sep 8, 2024
@mdbernard
Copy link
Contributor

Could you please clarify why some of these are currently blocked, and what actions would be needed to unblock them?

@carljm
Copy link
Contributor Author

carljm commented Sep 16, 2024

Could you please clarify why some of these are currently blocked, and what actions would be needed to unblock them?

Hi! Thanks for the nudge. Some of these are actually no longer blocked, because we already did the blocking thing! I've updated the list accordingly.

Some items are blocked by settling on a representation for generic types and handling of type variables in inference. I'm hoping to get to this in the next few weeks.

Some others aren't really blocked but are part of a larger project (for example, adding support for async Python and awaitables.)

If there are any particular items you're interested in working on, let me know and I'm happy to discuss in more detail.

@TomerBin
Copy link
Contributor

Boolean expression can now be checked off ✅

carljm added a commit that referenced this issue Sep 27, 2024
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Implement inference for `f-string`, contributes to #12701.

### First Implementation

When looking at the way `mypy` handles things, I noticed the following:
- No variables (e.g. `f"hello"`) ⇒ `LiteralString`
- Any variable (e.g. `f"number {1}"`) ⇒ `str`

My first commit (1ba5d0f) implements
exactly this logic, except that we deal with string literals just like
`infer_string_literal_expression` (if below `MAX_STRING_LITERAL_SIZE`,
show `Literal["exact string"]`)

### Second Implementation

My second commit (90326ce) pushes
things a bit further to handle cases where the expression within the
`f-string` are all literal values (string representation known at static
time).

Here's an example of when this could happen in code:
```python
BASE_URL = "https://httpbin.org"
VERSION = "v1"
endpoint = f"{BASE_URL}/{VERSION}/post"  # Literal["https://httpbin.org/v1/post"]
```
As this can be sightly more costly (additional allocations), I don't
know if we want this feature.

## Test Plan

- Added a test `fstring_expression` covering all cases I can think of

---------

Co-authored-by: Carl Meyer <[email protected]>
charliermarsh added a commit that referenced this issue Sep 30, 2024
## Summary

Just trying to familiarize myself with the general patterns, testing,
etc.

Part of #12701.
carljm added a commit that referenced this issue Oct 4, 2024
## Summary

Implements the comparison operator for `[Type::IntLiteral]` and
`[Type::BooleanLiteral]` (as an artifact of special handling of `True` and
`False` in python).
Sets the framework to implement more comparison for types known at
static time (e.g. `BooleanLiteral`, `StringLiteral`), allowing us to only
implement cases of the triplet `<left> Type`, `<right> Type`, `CmpOp`.
Contributes to #12701 (without checking off an item yet).

## Test Plan

- Added a test for the comparison of literals that should include most
cases of note.
- Added a test for the comparison of int instances

Please note that the cases do not cover 100% of the branches as there
are many and the current testing strategy with variables make this
fairly confusing once we have too many in one test.

---------

Co-authored-by: Carl Meyer <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
@cake-monotone
Copy link
Contributor

Would it be okay if I worked on the lambda expression? It doesn't seem to have any major blocking issues, and I think it would be a good starting point for my first contribution to red-knot. Please let me know if you have any other suggestions.

@carljm
Copy link
Contributor Author

carljm commented Oct 7, 2024

Would it be okay if I worked on the lambda expression? It doesn't seem to have any major blocking issues, and I think it would be a good starting point for my first contribution to red-knot. Please let me know if you have any other suggestions.

Thanks for your interest in contributing! Lambda expression inference would require adding a general Callable type, which we don't have yet, and is quite complex to implement; probably not a great first contribution. I think probably a good starter task would be fleshing out some missing support in unary/binary/compare expressions; see also #13618

@cake-monotone
Copy link
Contributor

Thanks for your interest in contributing! Lambda expression inference would require adding a general Callable type, which we don't have yet, and is quite complex to implement; probably not a great first contribution. I think probably a good starter task would be fleshing out some missing support in unary/binary/compare expressions; see also #13618

Thank you for the helpful explanation! I’ll follow your suggestion and try working on the comparison task first :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Contributions especially welcome red-knot Multi-file analysis & type inference
Projects
None yet
Development

No branches or pull requests

4 participants