Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic query feature #1

Merged
merged 3 commits into from
Nov 11, 2024
Merged

Add basic query feature #1

merged 3 commits into from
Nov 11, 2024

Conversation

philss
Copy link

@philss philss commented Nov 8, 2024

This still don't support params - I should tackle that in another PR.

Philip Sampaio added 3 commits November 8, 2024 19:30
It accepts all formats and it integrates with Explorer if the
`:explorer` format is used.
@philss
Copy link
Author

philss commented Nov 8, 2024

This PR also don't tackle the CI workflow. I should create that infra in another PR as well.

@philss philss merged commit f528489 into main Nov 11, 2024
@philss philss deleted the ps-add-query-feature branch November 11, 2024 13:30
Copy link
Contributor

@wojtekmach wojtekmach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! As I mentioned in the comment in hindsight this is probably not the best design but I think that's ok and we can improve this wholesale in the future (alongside req_athena) while we have bigger fish to fry in the meantime.

@@ -1,12 +1,167 @@
defmodule ReqCh do
@moduledoc """
A Req plugin for ClickHouse.

By default, `ReqCh` will use TSV as the default output format.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: I'd would call it ReqCH, we shorten GitHub to GH so we should shorten ClickHouse to CH.

Comment on lines +52 to +54
iex> req = Req.new() |> ReqCh.attach()
iex> Req.post!(req, clickhouse: "SELECT number from system.numbers LIMIT 3").body
"0\\n1\\n2\\n"
Copy link
Contributor

@wojtekmach wojtekmach Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah so similar to ReqAthena discussion, this really feels like better achieved with:

iex> ReqCH.query!("SELECT ...")
iex> req = ReqCH.new(hostname: "clickhouse.cloud", retry: :transient)
iex> ReqCH.query!(req, "SELECT ...", retry: false)

Below is intentionally bizarre and unfair to this plugin because it is true for any plugin, but like, isn't it weird we can make this call?

req = Req.new() |> ReqCh.attach()
Req.get!(req, url: "https://httpbin.org", scheme: "http")

Just to be clear, this is totally on me FWIW.

url_parts = [
get_option(request, :scheme, "http"),
"://",
maybe_credentials(request),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of sending the credentials in the URL (which could show up in logs and stuff) I wonder if we can use the auth: {:basic, "user:pass"} feature.

Comment on lines +157 to +160
Req.Request.halt(request, %{
response
| body: Explorer.DataFrame.load_parquet!(response.body)
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could also be:

Req.Request.halt(request, update_in(response.body, &Explorer.DataFrame.load_parquet!/1))

philss added a commit that referenced this pull request Nov 12, 2024
philss added a commit that referenced this pull request Nov 13, 2024
* Apply some suggestions from Wojtek

#1 (review)

* ReqCh -> ReqCH

* Add missing ReqCH.attach/1

* Update lib/req_ch.ex

Co-authored-by: Wojtek Mach <[email protected]>

---------

Co-authored-by: Wojtek Mach <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants