Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changelog for 1.13.3 #14990

Closed
wants to merge 107 commits into from
Closed

Changelog for 1.13.3 #14990

wants to merge 107 commits into from

Conversation

straight-shoota
Copy link
Member

straight-shoota and others added 30 commits July 22, 2024 16:59
This is a necessary preparatory step to update the docker image to Crystal 1.13.1 (#14810). That image is based on Alpine 3.20 which has neither OpenSSL 3.0 nor 1.1.1 in its repository.
Makes sure that a comment followed by an annotation and a macro call is properly recognized as a doc comment for the expanded macro code.
It turns out the fix in #14730 made all `ReferenceStorage` objects non-atomic; `Crystal::ReferenceStorageType#reference_type` returns a reference type, whose `#has_inner_pointers?` always returns true since the reference itself is a pointer.
This PR fixes that again by adding a special case for `ReferenceStorage`.
Refactors `Crystal::Compiler`:

1. extracts `#sequential_codegen`, `#parallel_codegen` and `#fork_codegen` methods;
2. merges `#codegen_many_units` into `#codegen` directly;
3. stops collecting reused units: `#fork_codegen` now updates `CompilationUnit#reused_compilation_unit?` state as reported by the forked processes, and `#print_codegen_stats` now counts & filters the reused units.

Prerequisite for #14748 that will introduce `#mt_codegen`.
When an overlapped operation gets cancelled, we still need to wait for completion of the operation (with status `ERROR_OPERATION_ABORTED`) before it can be freed.
Previously we stored a reference to cancelled operations in a linked list and removed them when complete. This allows continuing executing the fiber directly after the cancellation is triggered, but it's also quite a bit of overhead.
Also it makes it impossible to allocate operations on the stack.

Cancellation is triggered when an operation times out.

The change in this patch is that after a timeout the fiber is suspended again, expecting completion via the event loop. Then the operation can be freed.

* Removes the `CANCELLED` state. It's no longer necessary, we only need to distinguish whether a fiber was woken up due to timeout or completion. A follow-up will further simplify the state handling.
* Replace special timeout event and fiber scheduling logic with generic `sleep` and `suspend`
* Drops `@@cancelled` linked list.
* Drops the workaround from #14724 (comment)
Add `GC.stop_world` and `GC.start_world` methods to be able to stop and restart the world at will from within Crystal.

- gc/boehm: delegates to `GC_stop_world_external` and `GC_start_world_external`;
- gc/none: implements its own mechanism (tested on UNIX & Windows).

My use case is a [perf-tools](https://github.com/crystal-lang/perf-tools) feature for [RFC 2](crystal-lang/rfcs#2) that must stop the world to print out runtime information of each ExecutionContext with their schedulers and fibers. See crystal-lang/perf-tools#18
Allows `slice[-3, 3]` to return the last 3 elements of the slice, for example, similar to how Array#[-start, count] behaves, with the difference that Slice returns exactly *count* elements, while Array returns up to *count* elements.

Introduces a couple changes:

- negative start now returns from the end of the slice instead of returning nil/raising IndexError
- negative count now raises ArgumentError instead of returning nil/raising IndexError

I believe the current behavior is buggy (unexpected result, underspecified documentation), but this can be considered a breaking change.
If an exception raises in the code that prepares a forked process for `exec`, the error message of this exception is written through a pipe to the original process, which then raises an exception for the calling code (`Process.run`).
The error only includes a message, no stack trace. So the only stack trace you get is that of the handler which reads the message from the pipe and raises in the original process. 
But that's not very relevant. We want to know the location of the original exception.
This patch changes from sending just the exception message, to printing the entire backtrace (`inspect_with_backtrace`).
When calling `#sort!` without a block, if two elements have the same binary representations whenever they compare equal using `#<=>`, then they are indistinguishable from each other and swapping is a no-op. This allows the use of unstable sorting which runs slightly faster and requires no temporary allocations, as opposed to stable sorting which allocates memory linear in the collection size.

Primitive floats do not support it, as the signed zeros compare equal but have opposite sign bits. For simplicity, unions also aren't touched; either they don't have a meaningful, non-null `#<=>` defined across the variant types, or they break the criterion (e.g. `1_i32` and `1_i8` have different type IDs).

`#sort` always delegates to `#sort!`. This does not affect `#sort_by!` since the projected element type alone doesn't guarantee the original elements themselves can be swapped in the same way.
Updates `distribution-scripts` dependency to crystal-lang/distribution-scripts@da59efb

This includes the following changes:

* crystal-lang/distribution-scripts#326
This allows `Pointer.malloc` and `Reference#allocate` to be implemented without compiler primitives eventually, see #13589 and #13481. This might be helpful to diagnostic tools related to the GC too.
HertzDevil and others added 23 commits September 3, 2024 19:59
This private method is now directly responsible for iterating over all `LibC::Addrinfo` objects, so there is no need to store this information in the `Socket::Addrinfo` struct itself.
It does not appear the use of `read_timeout` here was intended.

This applies to connection-oriented sockets only. Connectionless sockets like `UDPSocket` call `Crystal::System::Socket#system_connect_connectionless` instead which ignores the timeout parameter.
The purpose of raising an exception here is to prevent the caller from doing something unsafe. _Changing_ the value is unsafe, but setting `buffer_size` to the same value is a safe operation.
This is in line with other places in the standard library that favor `Time::Span` over number types, such as `IO` timeouts (#14368) and `Benchmark.ips` (#14805).
More examples of valid group IDs can be obtained using `whoami.exe /groups`.
Moves the platform-specific code into a separate module, so that implementations other than `LibC.getaddrinfo` can be added without cluttering the same file (e.g. Win32's `GetAddrInfoExW` from #13619).
The `super` implementation `Enumerable#size` has the same type restriction already.
This fixes the `LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library` message that shows up on Windows CI while running compiler specs.
Macros inject code into other scopes. Paths are resolved in the expanded scope and there can be namespace conflicts.
This fixes non-global paths in macro bodies that expand into uncontrolled scopes where namespaces could clash.

This is a fixup for #14282 (released in 1.12.0).
…#14971)

`BigFloat`s represent their base-`256 ** sizeof(LibGMP::MpLimb)` exponent with a `LibGMP::MpExp` field, but `LibGMP.mpf_get_d_2exp` only returns the base-2 exponent as a `LibC::Long`, so values outside `(2.0.to_big_f ** -0x80000001)...(2.0.to_big_f ** 0x7FFFFFFF)` lead to an exponent overflow on Windows or 32-bit platforms:

```crystal
require "big"

Math.frexp(2.0.to_big_f ** 0xFFFFFFF5)  # => {1.55164027193164307015e+1292913986, -10}
Math.frexp(2.0.to_big_f ** -0xFFFFFFF4) # => {1.61119819150333097422e-1292913987, 13}
Math.frexp(2.0.to_big_f ** 0x7FFFFFFF)  # raises OverflowError
```

This patch fixes it by computing the exponent ourselves.
…4972)

The abstraction is a mere abstraction over an atomic integer and the
object itself are only ever used internally of other objects, with the
exception of Channel where the code explicitely accesses the ivar
directly (thus not making copies).

We can avoid a HEAP allocation everywhere we use them (i.e. in lots of places).
Prevents the GC from trying to cleanup resources that had already been closed in signal/process pipes.
Signal handling manipulate pipes (file descriptors) on UNIX, and messing with the evloop after fork can affect the parent process evloop in some cases.
Refactors spawning child processes on UNIX that relies on fork/exec to not involve the event loop after fork and before exec. We still continue to rely on the eventloop in the parent process, of course.

* Extract Crystal::System::FileDescriptor.system_pipe (UNIX)
* Fix: avoid evloop after fork to report failures in Process.spawn (UNIX)
* Fix: don't involve evloop in System::Process.reopen_io (UNIX)

This is called after fork before exec to reopen the stdio. We can
leverage some abstractions (set blocking, unset cloexec) but musn't call
to methods that involve the evloop and would mess with the parent
evloop.
@straight-shoota straight-shoota added this to the 1.13.3 milestone Sep 10, 2024
@straight-shoota straight-shoota changed the base branch from master to release/1.13 September 10, 2024 06:40
@straight-shoota straight-shoota deleted the changelog/1.13.3 branch September 10, 2024 06:44
@straight-shoota straight-shoota removed this from the 1.13.3 milestone Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.