Skip to content

Commit

Permalink
FEATURE: encode param as pg array (#49)
Browse files Browse the repository at this point in the history
* encode as pg array

* auto_encode_arrays

* readme
  • Loading branch information
ermolaev authored Aug 20, 2024
1 parent 7e0d428 commit 7ee85bf
Show file tree
Hide file tree
Showing 4 changed files with 51 additions and 10 deletions.
39 changes: 33 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,15 @@ p conn.query_array("select 1 as a, '2' as b union select 3, 'e'").to_h
# {1 => '2', 3 => 'e'}
```

## Auto Encode Arrays (only PostgreSQL)
```ruby
pg_conn = PG.connect(db_name: 'my_db')
conn = MiniSql::Connection.get(pg_conn, auto_encode_arrays: true)

# select * from table where id = ANY('{1,2,3}')
conn.query("select * from table where id = ANY(?)", [1, 2, 3])
```

## The query builder

You can use the simple query builder interface to compose queries.
Expand Down Expand Up @@ -241,15 +250,33 @@ Streaming support is only implemented in the postgres backend at the moment, PRs
See [benchmark mini_sql](https://github.com/discourse/mini_sql/tree/master/bench/prepared_perf.rb)
[benchmark mini_sql vs rails](https://github.com/discourse/mini_sql/tree/master/bench/bilder_perf.rb).

By default prepared cache size is 500 queries. Use prepared queries only for frequent queries.

```ruby
conn.prepared.query("select * from table where id = ?", id: 10)
```

### Prepared Statement Bloat in PostgreSQL
By default prepared cache size is __500__ queries per connection. Use prepared queries only for frequent queries.

The following code will create 100 prepared statements in PostgreSQL for a single database connection:

ids = rand(100) < 90 ? [1] : [1, 2]
builder = conn.build("select * from table /*where*/")
builder.where("id IN (?)", ids)
builder.prepared(ids.size == 1).query # most frequent query
```ruby
100.times do |i|
ids = (1..i).to_a
conn.prepared.query("SELECT * FROM table WHERE id IN (?)", ids)
end
```

This can lead to high memory usage and performance issues due to the overhead of maintaining numerous prepared statements.

To improve performance, you can enable `auto_encode_arrays: true`. With this option, only one prepared statement is created, regardless of the size of ids:

```ruby
pg_conn = PG.connect(db_name: 'my_db')
conn = MiniSql::Connection.get(pg_conn, auto_encode_arrays: true)
100.times do |i|
ids = (1..i).to_a
conn.prepared.query("select * from table where id = ANY (?)", ids)
end
```

## Active Record Postgres
Expand Down
7 changes: 4 additions & 3 deletions lib/mini_sql/inline_param_encoder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@

module MiniSql
class InlineParamEncoder
attr_reader :conn
attr_reader :conn, :array_encoder

def initialize(conn)
def initialize(conn, array_encoder = nil)
@conn = conn
@array_encoder = array_encoder
end

def encode(sql, *params)
Expand Down Expand Up @@ -62,7 +63,7 @@ def quote_val(value)
when false then "false"
when nil then "NULL"
when [] then "NULL"
when Array then value.map { |v| quote_val(v) }.join(', ')
when Array then array_encoder ? "'#{array_encoder.encode(value)}'" : value.map { |v| quote_val(v) }.join(', ')
else raise TypeError, "can't quote #{value.class.name}"
end
end
Expand Down
3 changes: 2 additions & 1 deletion lib/mini_sql/postgres/connection.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ def self.type_map(conn)
def initialize(raw_connection, args = nil)
@raw_connection = raw_connection
@deserializer_cache = (args && args[:deserializer_cache]) || self.class.default_deserializer_cache
@param_encoder = (args && args[:param_encoder]) || InlineParamEncoder.new(self)
array_encoder = PG::TextEncoder::Array.new if args && args[:auto_encode_arrays]
@param_encoder = (args && args[:param_encoder]) || InlineParamEncoder.new(self, array_encoder)
@type_map = args && args[:type_map]
end

Expand Down
12 changes: 12 additions & 0 deletions test/mini_sql/postgres/connection_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -154,4 +154,16 @@ def test_unamed_query
assert_equal(row.two, 2)
assert_equal(row.column2, 3)
end

def test_encode_array
connection = pg_connection(auto_encode_arrays: true)

ints = [1, 2, 3]
strings = %w[a b c]
row = connection.query("select ?::int[] ints, ?::text[] strings", ints, strings).first

assert_equal(row.ints, ints)
assert_equal(row.strings, strings)
end

end

0 comments on commit 7ee85bf

Please sign in to comment.