Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libexpr: experimental pipe operators #11131

Merged
merged 1 commit into from
Jul 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions doc/manual/rl-next/pipe-operators.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
synopsis: "Add `pipe-operators` experimental feature"
prs:
- 11131
---

This is a draft implementation of [RFC 0148](https://github.com/NixOS/rfcs/pull/148).

The `pipe-operators` experimental feature adds [`<|` and `|>` operators][pipe operators] to the Nix language.
*a* `|>` *b* is equivalent to the function application *b* *a*, and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: "a into b", should we give it a name like that? Could be nice.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷 The other direction is a little harder to name, I think? I don't know if this is important.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"into" follows nicely from the more explicit "a piped into b", though I'm not sure about the other direction (<|).

Maybe that could be read as "from", as in a <| b is "a piped from b"; although I'd personally just read it backwards as "b piped into a".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"taking"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or ‘of’?

This seems like tutorial stuff, not reference stuff; these names would be entirely non-normative and not necessary for understanding the language from first principles. So does it belong in the reference documentation, or in release notes, or both or neither?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or ‘of’?

This seems like tutorial stuff, not reference stuff; [...] entirely non-normative

Certainly. It was just a

thought

I figured it'd be good to talk about this.


I think I like these two, because they're short and literal, with a word per symbol, in order.

  • |> pipe into
  • <| back pipe

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Late to the party, but FWIW in the current Lix draft I call them pipe left / pipe right in the code and tend towards calling them pipe (forward) and pipe backward in the documentation. (The forward being implicit and sometimes omitted)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to add some more ideas into the pot, a plumber would usually talk about pipes "feeding" or "supplying" and then "draining" into the wastewater pipes.

E.g:

  • "this tap is fed from the cold water supply"
  • "the sink drains down a 50mm pipe"
  • "the bath is supplied hot water from pipe A"
  • "pipe B supplies the bath with cold water"
  • "pipe C feeds the shower with cold water"
  • "the shower is fed hot water by pipe D"

I don't think this will sound natural in the context of programming, but I thought it was worth reminiscing on my background in construction 😅

I agree "Pipe left/backward" and "pipe right/forward/into" should be fairly natural, simple and obvious

*a* `<|` *b* is equivalent to the function application *a* *b*.

For example:

```
nix-repl> 1 |> builtins.add 2 |> builtins.mul 3
9
nix-repl> builtins.add 1 <| builtins.mul 2 <| 3
7
```

`<|` and `|>` are right and left associative, respectively, and have lower precedence than any other operator.
These properties may change in future releases.
rhendric marked this conversation as resolved.
Show resolved Hide resolved

See [the RFC](https://github.com/NixOS/rfcs/pull/148) for more examples and rationale.
Copy link
Member

@roberth roberth Jul 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Channeling @fricklerhandwerk, the reference docs should be at least as complete as this.
EDIT (except for the temporal parts)


[pipe operators]: @docroot@/language/operators.md#pipe-operators
37 changes: 36 additions & 1 deletion doc/manual/src/language/operators.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,17 @@
| Logical conjunction (`AND`) | *bool* `&&` *bool* | left | 12 |
| Logical disjunction (`OR`) | *bool* <code>\|\|</code> *bool* | left | 13 |
| [Logical implication] | *bool* `->` *bool* | right | 14 |
| [Pipe operator] (experimental) | *expr* `\|>` *func* | left | 15 |
| [Pipe operator] (experimental) | *func* `<\|` *expr* | right | 15 |

[string]: ./types.md#type-string
[path]: ./types.md#type-path
[number]: ./types.md#type-float <!-- TODO(@rhendric, #10970): rationalize this -->
[number]: ./types.md#type-float
[list]: ./types.md#list
[attribute set]: ./types.md#attribute-set

<!-- TODO(@rhendric, #10970): ^ rationalize number -> int/float -->

## Attribute selection

> **Syntax**
Expand Down Expand Up @@ -182,3 +186,34 @@ Equivalent to `!`*b1* `||` *b2*.

[Logical implication]: #logical-implication

## Pipe operators

- *a* `|>` *b* is equivalent to *b* *a*
- *a* `<|` *b* is equivalent to *a* *b*

> **Example**
>
> ```
> nix-repl> 1 |> builtins.add 2 |> builtins.mul 3
> 9
>
> nix-repl> builtins.add 1 <| builtins.mul 2 <| 3
> 7
> ```
> **Warning**
>
> This syntax is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md)
> and may change in future releases.
>
> To use this syntax, make sure the
> [`pipe-operators` experimental feature](@docroot@/contributing/experimental-features.md#xp-feature-pipe-operators)
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = pipe-operators
> ```
[Pipe operator]: #pipe-operators
14 changes: 14 additions & 0 deletions src/libexpr/lexer.l
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@ static StringToken unescapeStr(SymbolTable & symbols, char * s, size_t length)
return {result, size_t(t - result)};
}

static void requireExperimentalFeature(const ExperimentalFeature & feature, const Pos & pos)
{
if (!experimentalFeatureSettings.isEnabled(feature))
throw ParseError(ErrorInfo{
.msg = HintFmt("experimental Nix feature '%1%' is disabled; add '--extra-experimental-features %1%' to enable it", showExperimentalFeature(feature)),
.pos = pos,
});
}

}

Expand Down Expand Up @@ -119,6 +127,12 @@ or { return OR_KW; }
\-\> { return IMPL; }
\/\/ { return UPDATE; }
\+\+ { return CONCAT; }
\<\| { requireExperimentalFeature(Xp::PipeOperators, state->positions[CUR_POS]);
return PIPE_FROM;
}
\|\> { requireExperimentalFeature(Xp::PipeOperators, state->positions[CUR_POS]);
return PIPE_INTO;
}

{ID} { yylval->id = {yytext, (size_t) yyleng}; return ID; }
{INT} { errno = 0;
Expand Down
30 changes: 23 additions & 7 deletions src/libexpr/parser.y
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,14 @@ static void setDocPosition(const LexerState & lexerState, ExprLambda * lambda, P
}
}

static Expr * makeCall(PosIdx pos, Expr * fn, Expr * arg) {
if (auto e2 = dynamic_cast<ExprCall *>(fn)) {
e2->args.push_back(arg);
return fn;
}
return new ExprCall(pos, fn, {arg});
}


%}

Expand All @@ -123,6 +131,7 @@ static void setDocPosition(const LexerState & lexerState, ExprLambda * lambda, P

%type <e> start expr expr_function expr_if expr_op
%type <e> expr_select expr_simple expr_app
%type <e> expr_pipe_from expr_pipe_into
%type <list> expr_list
%type <attrs> binds
%type <formals> formals
Expand All @@ -140,6 +149,7 @@ static void setDocPosition(const LexerState & lexerState, ExprLambda * lambda, P
%token <path> PATH HPATH SPATH PATH_END
%token <uri> URI
%token IF THEN ELSE ASSERT WITH LET IN_KW REC INHERIT EQ NEQ AND OR IMPL OR_KW
%token PIPE_FROM PIPE_INTO /* <| and |> */
%token DOLLAR_CURLY /* == ${ */
%token IND_STRING_OPEN IND_STRING_CLOSE
%token ELLIPSIS
Expand Down Expand Up @@ -206,9 +216,21 @@ expr_function

expr_if
: IF expr THEN expr ELSE expr { $$ = new ExprIf(CUR_POS, $2, $4, $6); }
| expr_pipe_from
| expr_pipe_into
| expr_op
;

expr_pipe_from
: expr_op PIPE_FROM expr_pipe_from { $$ = makeCall(state->at(@2), $1, $3); }
| expr_op PIPE_FROM expr_op { $$ = makeCall(state->at(@2), $1, $3); }
;

expr_pipe_into
: expr_pipe_into PIPE_INTO expr_op { $$ = makeCall(state->at(@2), $3, $1); }
| expr_op PIPE_INTO expr_op { $$ = makeCall(state->at(@2), $3, $1); }
;

expr_op
: '!' expr_op %prec NOT { $$ = new ExprOpNot($2); }
| '-' expr_op %prec NEGATE { $$ = new ExprCall(CUR_POS, new ExprVar(state->s.sub), {new ExprInt(0), $2}); }
Expand All @@ -233,13 +255,7 @@ expr_op
;

expr_app
: expr_app expr_select {
if (auto e2 = dynamic_cast<ExprCall *>($1)) {
e2->args.push_back($2);
$$ = $1;
} else
$$ = new ExprCall(CUR_POS, $1, {$2});
}
: expr_app expr_select { $$ = makeCall(CUR_POS, $1, $2); }
| expr_select
;

Expand Down
10 changes: 9 additions & 1 deletion src/libutil/experimental-features.cc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ struct ExperimentalFeatureDetails
* feature, we either have no issue at all if few features are not added
* at the end of the list, or a proper merge conflict if they are.
*/
constexpr size_t numXpFeatures = 1 + static_cast<size_t>(Xp::VerifiedFetches);
constexpr size_t numXpFeatures = 1 + static_cast<size_t>(Xp::PipeOperators);

constexpr std::array<ExperimentalFeatureDetails, numXpFeatures> xpFeatureDetails = {{
{
Expand Down Expand Up @@ -294,6 +294,14 @@ constexpr std::array<ExperimentalFeatureDetails, numXpFeatures> xpFeatureDetails
)",
.trackingUrl = "https://github.com/NixOS/nix/milestone/48",
},
{
.tag = Xp::PipeOperators,
.name = "pipe-operators",
.description = R"(
Add `|>` and `<|` operators to the Nix language.
)",
.trackingUrl = "https://github.com/NixOS/nix/milestone/55",
},
}};

static_assert(
Expand Down
1 change: 1 addition & 0 deletions src/libutil/experimental-features.hh
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ enum struct ExperimentalFeature
ConfigurableImpureEnv,
MountedSSHStore,
VerifiedFetches,
PipeOperators,
};

/**
Expand Down
5 changes: 5 additions & 0 deletions tests/functional/lang/eval-fail-pipe-operators.err.exp
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
error: experimental Nix feature 'pipe-operators' is disabled; add '--extra-experimental-features pipe-operators' to enable it
at /pwd/lang/eval-fail-pipe-operators.nix:1:3:
1| 1 |> 2
| ^
2|
1 change: 1 addition & 0 deletions tests/functional/lang/eval-fail-pipe-operators.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1 |> 2
3 changes: 3 additions & 0 deletions tests/unit/libexpr/main.cc
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ int main (int argc, char **argv) {
setEnv("_NIX_TEST_NO_SANDBOX", "1");
#endif

// For pipe operator tests in trivial.cc
experimentalFeatureSettings.set("experimental-features", "pipe-operators");

roberth marked this conversation as resolved.
Show resolved Hide resolved
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
54 changes: 54 additions & 0 deletions tests/unit/libexpr/trivial.cc
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,60 @@ namespace nix {
ASSERT_THAT(v, IsIntEq(15));
}

TEST_F(TrivialExpressionTest, forwardPipe) {
auto v = eval("1 |> builtins.add 2 |> builtins.mul 3");
ASSERT_THAT(v, IsIntEq(9));
}

TEST_F(TrivialExpressionTest, backwardPipe) {
auto v = eval("builtins.add 1 <| builtins.mul 2 <| 3");
ASSERT_THAT(v, IsIntEq(7));
}

TEST_F(TrivialExpressionTest, forwardPipeEvaluationOrder) {
auto v = eval("1 |> null |> (x: 2)");
ASSERT_THAT(v, IsIntEq(2));
}

TEST_F(TrivialExpressionTest, backwardPipeEvaluationOrder) {
auto v = eval("(x: 1) <| null <| 2");
ASSERT_THAT(v, IsIntEq(1));
}

TEST_F(TrivialExpressionTest, differentPipeOperatorsDoNotAssociate) {
ASSERT_THROW(eval("(x: 1) <| 2 |> (x: 3)"), ParseError);
}

TEST_F(TrivialExpressionTest, differentPipeOperatorsParensLeft) {
auto v = eval("((x: 1) <| 2) |> (x: 3)");
ASSERT_THAT(v, IsIntEq(3));
}

TEST_F(TrivialExpressionTest, differentPipeOperatorsParensRight) {
auto v = eval("(x: 1) <| (2 |> (x: 3))");
ASSERT_THAT(v, IsIntEq(1));
}

TEST_F(TrivialExpressionTest, forwardPipeLowestPrecedence) {
auto v = eval("false -> true |> (x: !x)");
ASSERT_THAT(v, IsFalse());
}

TEST_F(TrivialExpressionTest, backwardPipeLowestPrecedence) {
auto v = eval("(x: !x) <| false -> true");
ASSERT_THAT(v, IsFalse());
}

TEST_F(TrivialExpressionTest, forwardPipeStrongerThanElse) {
auto v = eval("if true then 1 else 2 |> 3");
ASSERT_THAT(v, IsIntEq(1));
}

TEST_F(TrivialExpressionTest, backwardPipeStrongerThanElse) {
auto v = eval("if true then 1 else 2 <| 3");
ASSERT_THAT(v, IsIntEq(1));
}

TEST_F(TrivialExpressionTest, bindOr) {
auto v = eval("{ or = 1; }");
ASSERT_THAT(v, IsAttrsOfSize(1));
Expand Down
Loading