Optionally relax sqlglot / ibis pin via env var for use in SQLMesh

[z3z1ma]

python-bigquery-dataframes/setup.py

Line 63 in 0413964

"sqlglot >=20.8.0,<=20.11",

There is very strong synergy between bigframes and https://github.com/TobikoData/sqlmesh
However because of the sqlglot pin and the transitive sqlglot pin in the ibis pin, this library is incompatible with sqlmesh. (sqlmesh is built and maintained by creators of sqlglot and therefore usually has latest pin, and a strict one at that)

Since you are using setup.py, we should be able to use an env var or something similar to have an opt-in way to remove the version constraints on those two deps such that they can be defined by the consumer. We would accept that it opens one up to undefined behavior but by contrast it frees you from needing to constantly update your pins and allows more experimentation and adoption in the community.

You can see an example sqlmesh model using bigframes here https://github.com/TobikoData/sqlmesh/pull/2667/files

[tswast]

Since you are using setup.py, we should be able to use an env var or something similar to have an opt-in way to remove the version constraints on those two deps such that they can be defined by the consumer.

Do you have an example of another package using an environment variable for this purpose?

I'm personally a bit wary of of relaxing these pins without having fully tested against such versions, but an opt-in mechanism for folks who know what they are doing seems acceptable. Happy to accept a pull request if you want to mail something. :-)

P.S. We're actively working on an ibis (and sqlglot) upgrade in #827. There's quite a bit that breaks in ibis 9.x, so our wariness is definitely warranted. It's quite likely that we'll end up "vendoring" much of the ibis package so that this kind of thing (relaxing ibis versions, or even removing ibis as a pip dependency altogether) will become much more feasible.

[rubenvereecken]

We'd love this too.

[chelsea-lin]

the SQLGlot version has been updated to sqlglot >=23.6.3,<25.2 as part of the #827 merge, though it was still pined with a maximum version to align with ibis framework.