Unique vectors

[TimWhiting]

Addresses #544

@anfelor can you take a look at this?

I added a few public apis and a few private helpers.
Public:

• set
• update
• update for local variable vectors

The most important function is the copy function, which ensures that the vector is unique prior to applying any mutation operations. For many cases we would expect it to be unique.

[TimWhiting]

We might want to add a few more apis (cutting an array short and dropping the the rest of the array), or extending an array.

[anfelor]

Thanks @TimWhiting, this looks a lot friendlier than my draft! I made a few minor comments regarding naming of variables and documentation.

I worry a bit that the unsafe surface area is quite large still. I think we can probably pay for two is-unique checks per iteration of map (given that the second can be well-predicted). Then we can make this API more clean by incorporating the copy calls into unsafe-assign and unsafe-idx-own.

[anfelor]

We might want to add a few more apis (cutting an array short and dropping the the rest of the array), or extending an array.

Yes, that would be fun! I think we could also support vector.filter, which would require us to write box_null into the tail of the vector and keep a separate length field. In OCaml, this is unsupported since box_null might not be an inhabitant of the type of the elements though, and I wonder if there is a problem that I don't see currently.

[TimWhiting]

We might want to add a few more apis (cutting an array short and dropping the the rest of the array), or extending an array.

Yes, that would be fun! I think we could also support vector.filter, which would require us to write box_null into the tail of the vector and keep a separate length field. In OCaml, this is unsupported since box_null might not be an inhabitant of the type of the elements though, and I wonder if there is a problem that I don't see currently.

I think these sorts of APIs are maybe best left to a resizeable vector type, which could hold on to the length like you mention and have a resize policy for add / extending the vector as well. What do you think?

[anfelor]

lgtm

[TimWhiting]

I've found a bug with this. Specifically, it behaves poorly with exceptions. Note that this doesn't affect all of the other vector functions, just the mutation of vector elements in local variables (i.e. assign/update). Added a few tests that show the problem.

I think we can still do efficient update of mutable variables, but it will require some runtime changes.
Proposed runtime adjustment: have another special refcount (like STICKY), for REUSE_NO_DROP. For updates to variables, the ref that contains the datastructure needs to be kept alive until the ref is deallocated, but the datastructure itself should have a unique refcount and be reused if it can. However, it shouldn't deallocate itself, if for example an exception is thrown, since the old value could still be obtained via the ref, even if the original value was consumed in the update prior to or after the exception was thrown.

Turning this to draft for now, until we can discuss this, though I think we should probably move ahead with the other changes which are independent from the local variable change.