4.x: Concurrency limits module, and support in Helidon WebServer

[tomas-langer]

Description

Resolves #8897
Resolves #9229

Documentation

This PR introduces

• a new module common/concurrency/limits that provides API and SPI for concurrency limit implementations, and a couple of default implementations (AIMD, Fixed).
• a new module webserver/concurrency-limits that provides feature with a filter to impose limits within a filter in routing
• update of webserver/webserver to use a Limit instead of a Semaphore in connection handlers (backward compatible)

Configuration reference will be in the generated documentation.

Configure limits on WebServer

This will configure limit for a server listener (configurable per listener), enforced on the connection level (i.e. outside of routing and filters).

The default behavior is the same (unlimited).
If server.max-concurrent-requests is configured (value is not -1), it will be used and concurrency-limit configuration for the listener will be ignored.

Configuration:

server:
  port: 8080
  concurrency-limit:
    aimd: # `limit type`
      # AIMD limit configuration

Configure limits for routing

This will configure limit for as a server feature, enforced in an HTTP filter.
Configuration:

server:
  features:
    limits: # the feature is called `limits`
      enabled: true
      concurrency-limit: # `limit` configuration of the `limits` server feature
        fixed: # `limit type`
          permits: 1
          queue-length: 10

[arouel]

LGTM, great rework and integration

[vasanth-bhat]

Have few clarifications

1. Going through Readme , there is mention about 'Semaphore' concurrency limit type in addition to aimd & BulkHead. Is this same as. "basic" (BasicLimit)? Also how is this different from specifying. "server.max-concurrent-requests"?

2. For the BulkHead, it would be good to have metrics support , around queueing , for example % queue full. This is important in production to monitor the queueing.

3. For AMID , it would be good to have a metrics around the dynamic concurrency limit in use. A Histogram may be useful here.

[tomas-langer]

After discussing the needs in Helidon, we must do this using the permit approach, so this can be reused in webclient.
I will refactor the PR.

[arouel]

I wanted to bring up that metrics would be super important for production as @vasanth-bhat brought it up as well. Within my draft I figured quickly that the limiting works as expected but metrics would give me the insights I need to build convidence. Maybe this is something for a follow-up Pull Request, idk.

[tomas-langer]

Metrics must be a follow up, as it would require much more work. I cannot just add a dependency on helidon-metrics-api, as it would introduce it everywhere (maybe that is the solution, but it requires a bit more thought).
I will finish this PR as the introduction of concurrency limits. We can look into the metrics as a follow up (this would make sense for fault tolerance as well, not just concurrency limits).

[tomas-langer]

Pushed a new version, updated original description to reflect new implementation.
I have implemented the fixed limit as a more general feature - it supports queing now, so we no longer need fault tolerance for this.
How to add this to metrics needs to be a follow up.

[tomas-langer]

Created a follow up issue: #9304

[vasanth-bhat]

Went through the new changes.

I think it would help to clarify between the below 2 , with example to understand the difference between the two. Do both them provide same functionality but via different approaches ?

1. Fixed Limit with queueing support at the connection level (or a server listener )
2. Fixed limit with queueing as server feature enforced via filter.