capture

[cpurdy]

From the call today:

• Lambdas automatically capture effectively final variables by value
• Lambdas automatically capture non-effectively final variables by Ref if the lambda reads from but does not write to the variable
• Lambdas automatically capture non-effectively final variables by Var (note: Var : Ref) if the lambda writes to the variable

We want to alter these rules as follows:

• By default, lambdas will only capture variables by value
• Lambdas that explicitly refer to a reference to a variable (via the & operator) will capture by either Ref or Var as appropriate (to be defined)
• There must be a means (declaration site or use site, TBD) to indicate that a variable can be / must be captured by either Ref or Var
• Attempts to modify a variable not captured by Var would be a compile-time error
• We may choose to eliminate explicit capture by Ref vs. Var and just use capture by Var instead, since Var : Ref (with the possible exception being when capturing something that does not have a Var but does have a Ref?)

Declaration site syntax could be an annotation (as discussed):

@AllowCaptureByVar Int i = 0;

(Obviously the annotation name given here is for illustration purposes only; if we go this route, the name needs to be much more succinct.)

We didn't have any specific syntax suggestions for use-site syntax that I can recall.

Next step is to mock up specific syntax examples and discuss.

[cliffclick]

Devil's advocate:
"Lambdas automatically capture effectively final variables by value" - Since effectively final, its ambiguous how they are captured. You can't write a program to tell the implementations apart, so no need to specify.

[cliffclick]

There's a default (capture by value).
There's an unusual syntax &foo.
Seems reasonable to me to assume that you are using the unusual syntax to change the default.
I propose using &foo changes the default to capture-by-var/ptr, for all uses of foo in the lambda.

[ggleyzer]

I think it's exactly what Cam's proposal says (bullet # 2)

[cpurdy]

I think what Cliff is suggesting is that would be the only mechanism, because Cliff is comfortable allowing a Ref/Var to assume all of the surface area of its referent (which is not something that makes any sense to me). (Note: I may have misunderstood Cliff's suggestion, and it's also equally possible that Cliff hasn't yet understood the type system ... I'm just trying to get the ideas into the open where we can discuss them.)

[markandrewfalco]

@cpurdy can you describe the difference between captured by var and by ref.

[cpurdy]

Sure. There's an op produced for a Ref capture that says "give me a Ref instance for that variable x", while there's an op produced for a Var capture that says "give me a Var instance for that variable x". Since a Var is read/write, it seemed reasonable at the time to reduce the number of write "leaks" when writes were not necessary.

[cliffclick]

Some possibilities, I tried to fill out the quadrant. Surely misses some cases, and didn't compute it write. Feel free to correct or add more cases.

Just reads, capture by value

for( Int i: 0..4 )
  funs += () -> print(i,i);
for( fun : funs )
  funs();

Always Ef-Final - "0,0 1,1 2,2 3,3"
Default, changes w/& - "0,0 1,1 2,2 3,3"
Each usage changes w/& - "0,0 1,1 2,2 3,3"
Default, changes w/write - "0,0 1,1 2,2 3,3"
Each usage changes w/write - "0,0 1,1 2,2 3,3"

Just reads, capture by Var/Ptr

for( @CaptureByVal Int i: 0..4 )
  funs += () -> print(i,i);
for( fun : funs )
  funs();

Always Ef-Final - "4,4 4,4 4,4 4,4"
Default, changes w/& - "4,4 4,4 4,4 4,4"
Each usage changes w/& - "4,4 4,4 4,4 4,4"
Default, changes w/write - "4,4 4,4 4,4 4,4"
Each usage changes w/write - "4,4 4,4 4,4 4,4"

Read/Write, some choices follow

for( Int i: 0..4 )
  funs += () -> print(i,++i);
for( fun : funs )
  funs();

Always Ef-Final - Compile error, "x++" not final
Default, changes w/& - Compile error, "x++" not final
Each usage changes w/& - Compile error, "x++" not final
Default, changes w/write - "5,6 6,7 7,8 8,9"
Each usage changes w/write - "0,6 1,7 2,8 3,9"

Read/Write with &, some choices follow

for( Int i: 0..4 )
  funs += () -> print(&i.get,++i);
for( fun : funs )
  funs();

Always Ef-Final - Compile error, "x++" not final
Default, changes w/& - "5,6 6,7 7,8 8,9"
Each usage changes w/& - Compile error, "x++" not final
Default, changes w/write - "5,6 6,7 7,8 8,9"
Each usage changes w/write - "5,6 6,7 7,8 8,9"

No read/write, just &

for( Int i: 0..4 )
  funs += () -> print(&i.get,&i.put(&i.get()+1));
for( fun : funs )
  funs();

Always Ef-Final - "5,6 6,7 7,8 8,9"
Default, changes w/& - "5,6 6,7 7,8 8,9"
Each usage changes w/& - "5,6 6,7 7,8 8,9"
Default, changes w/write - "5,6 6,7 7,8 8,9"
Each usage changes w/write - "5,6 6,7 7,8 8,9"

[cpurdy]

Gene and I are discussing a path forward on this topic. We both agree that introducing a declaration site annotation makes sense. It would indicate to the compiler that the annotated variable could be captured as a Ref -- or even Var -- if and as necessary. Without an annotation, all captures would be strictly by-value captures, and thus if you were to attempt to modify a captured value that is not declaration-site annotated to indicate by-reference, then that attempted modification would be a compile-time error.

Edit: Additionally, it will be a compile time error if a variable is marked as @Volatile and it is not captured.

So at this point, we just need a name. Gene has fallen in love with @CaptureByVar, which is the name (@AllowCaptureByVar) that I used as a purposefully-horrible straw man above, just to guarantee that we would not use that name. 🤣 Gene grudgingly would accept an equally long or longer name like @RequireDynamicCapture or @CaptureReference, which makes me want to 🤮

I personally like @Volatile. The word itself means "Tending to vary often or widely, as in price." The idea being we want to describe the variable, at its declaration site, as something that (from the point of view of a capture) will continue to vary. It's a variable that varies when it's captured. I understand that this re-uses a term from Javaland -- certainly not the first time we've done this -- in a manner that does not retain the exact use of the term in Javaland.

Any other ideas/suggestions?

[ggleyzer]

@Unsafe
@Sink

[cpurdy]

@CopyProtected 🤣
@MutationsVisible
@SingleSourceOfTruth
@DisableCaptureByValue
@TheseAnnotationsJustKeepGettingLonger

Edit: Gene and I played with:
@Share
@Shared

I kind of like:
@Live - edit: Gene and I felt this one was a bit confusing
@Dynamic

But I still prefer:
@Volatile

[cpurdy]

Also worth pointing to to the "Go lang" issue that originally got us talking about this change: https://kkentzo.github.io/2021/01/21/golang-loop-variable-gotcha/

[ggleyzer]

Implemented by this commit