Error when encoding value breaks connection permanently

[jvf]

I noticed that inserting with a type error in the inserted values breaks a connection permanently (output shortened):

$ iex -S mix
==> utils
Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]
Interactive Elixir (1.15.7) - press Ctrl+C to exit (type h() ENTER for help)

iex(1)> Application.spec(:xandra)[:vsn]
~c"0.19.0"

# we start a connection with max_concurrent_requests_per_connection: 1 to see the problem faster
iex(2)> {:ok, conn} = Xandra.start_link(authentication: {Xandra.Authenticator.Password, [username: "cassandra", password: "cassandra"]}, max_concurrent_requests_per_connection: 1)

# prepare keyspace and table
iex(3)> Xandra.execute(conn, "CREATE KEYSPACE test WITH REPLICATION = {'class': 'SimpleStrategy', 'replication_factor': 1}")
iex(4)> Xandra.execute(conn, "CREATE TABLE test.participants (name text PRIMARY KEY, number int)")

# normal insertions in fast sequential order are no problem:
iex(5)> Enum.each(1..100, fn x -> Xandra.execute(conn, "INSERT INTO test.participants (name, number) VALUES (:name, :number)", %{"name" => {"text", "john#{x}"}, "number" => {"int", x}}) end)
:ok

# now we do one erroneous insertion (the `number` value is incorrectly given as string):
iex(6)> Xandra.execute(conn, "INSERT INTO test.participants (name, number) VALUES (:name, :number)", %{"name" => {"text", "john"}, "number" => {"int", "123"}})
** (FunctionClauseError) no function clause matching in Xandra.Protocol.V5.encode_value/2
[...]

# after one error the connection is broken permanently, even for correct insertions
iex(6)> Xandra.execute(conn, "INSERT INTO test.participants (name, number) VALUES (:name, :number)", %{"name" => {"text", "jane"}, "number" => {"int", 124}})
{:error,
 %Xandra.ConnectionError{
   action: "check out connection",
   reason: :too_many_concurrent_requests
 }}

This looks like a bug to me. The same works with different max_concurrent_requests_per_connection settings but you have to make more erroneous calls. It looks like each call that produces an encoding error "consumes" one of the concurrent requests, presumably by using up one a stream id without releasing it?

I also tried the same experiment with Xandra.execute(conn, "SELECT * FROM non_existing_table") but this does not break the connection.

[whatyouhide]

Ok interesting, yeah definitely a bug, thanks for reporting @jvf. Any chance you or @harunzengin could give this a try? I was on vacation and I won't have time next week (catch up week!) so it'd be a while til I could get to this.

[jvf]

@harunzengin agreed to look into this.

[siduniyal21]

Any updates here @jvf ? , I also ran into the same issue

[jvf]

I looked into this. I think the problem is this: When we execute a query we first allocate a stream id. Then the encoding fails due to the wrong type. But when an encoding error occurs during value serialization, the stream ID is never properly released. This stream id is permanently unusable, resulting in permeant too_many_concurrent_requests errors.

I tried a bit to fix releasing the stream ids correctly in case of error, but I have not yet found the correct way to do it.

@whatyouhide If you have any hints I am happy to try to implement it.

[jvf]

I think I found a way to do it, I will provide a PR soon.