Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated proto files to support go #460

Closed
wants to merge 1,664 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1664 commits
Select commit Hold shift + click to select a range
f39fe37
fix gradle proto compilation
May 31, 2023
95e3c63
update protocol
May 31, 2023
7eeb54e
add language request rpc
May 31, 2023
398780f
add skeleton for named values
May 31, 2023
658024d
add call to get list of query languages
May 31, 2023
ceb24b5
language list now handled by statement manager
May 31, 2023
c2447f4
add skeleton for poly value (de)serialisation
May 31, 2023
78a1ccb
add named parameter replacement
May 31, 2023
e762b40
add testcase for value replacement
Jun 1, 2023
046cd0d
start using polyvalues
Jun 1, 2023
9e6df73
add skeleton value map deserialization
Jun 1, 2023
8318548
add proto interface statement batch
Jun 1, 2023
6806ad1
add batch creation to statement manager
Jun 1, 2023
f5fe522
add values to proto interface statement
Jun 1, 2023
b196ed2
add skeleton parameterized statement
Jun 1, 2023
e1f1d05
add NamedValueProcessor
Jun 1, 2023
4cf8c3b
code cleanup
Jun 1, 2023
846647e
add abstract proto interface statement
Jun 2, 2023
1995095
add unparameterized statement
Jun 2, 2023
9b1bf73
removed unused import
Jun 2, 2023
fed5de1
add statement types to proto
Jun 2, 2023
73f68bd
code cleanup
Jun 2, 2023
5468136
add frame, row and value
Jun 2, 2023
78055db
remove unused value from logging
Jun 2, 2023
dfbfb2a
remove unused import
Jun 2, 2023
6a58485
improve unparameterized query execution
Jun 2, 2023
57bd784
proto definition cleanup
Jun 2, 2023
07248fe
removing rebase artifacts
datomo Jun 4, 2023
1604aca
proto refactor
Jun 5, 2023
3b4642d
now using string map proto for properties
Jun 5, 2023
daeac76
add frame building
Jun 5, 2023
b441d55
add serialisation for most poly values
Jun 5, 2023
bc6df2e
add serialisation for PolyValue
Jun 5, 2023
1beaeda
implement frame building using lambda
Jun 5, 2023
17b1127
add statement closing
Jun 6, 2023
b227370
improved proto poly type conversion
Jun 6, 2023
75ad0f9
added missing handling of polyvalues in jdbc converter and dbms meta
datomo Jun 6, 2023
f45d3d7
fix access to polyFloat and polyValue
Jun 7, 2023
a94673b
add status requests for running queries to protocol
Jun 7, 2023
15d6290
add periodic status update
Jun 7, 2023
7edfde3
implement periodic status updates to service
Jun 7, 2023
7854579
remove unused imports
Jun 7, 2023
3361149
allow user to disable status updates
Jun 7, 2023
fa76a42
redesigned status update system
Jun 7, 2023
63d2dda
remove unused field from statement status
Jun 7, 2023
39ca86d
add column meta data
Jun 8, 2023
f4289e5
code formatting
Jun 8, 2023
a46c6ff
add last flag to frame
Jun 13, 2023
d42be6e
add type to column meta
Jun 13, 2023
07e3c28
add fetch request
Jun 13, 2023
16320c7
add type to values
Jun 13, 2023
4b2479a
add BigDecimal Serialization
Jun 13, 2023
374f871
remove unused string map from execute
Jun 14, 2023
306b75f
remove unused map unwrap
Jun 14, 2023
4dbad6e
now using string map directly
Jun 14, 2023
ccf612c
add own meta extraction
Jun 15, 2023
35368d9
refactor result protos
Jun 15, 2023
b4c1dbe
refactor result protos
Jun 15, 2023
899eb59
refactor relational proto utils
Jun 15, 2023
fba1b3d
add automatic frame types for fetch
Jun 15, 2023
4c6137e
cosmetics
Jun 15, 2023
29ee2a7
add long and big decimal serialization
Jun 19, 2023
81c341c
cosmetics
Jun 19, 2023
70bdf91
add new row extraction
Jun 19, 2023
ae07342
Revert "Merge remote-tracking branch 'origin/proto-interface' into pr…
Jun 19, 2023
b35de7e
fix client meta null pointer exception
Jun 20, 2023
67897df
add autocommit to statement
Jun 20, 2023
0473db3
rename SupportedLanguages to LanguageResponse
Jun 20, 2023
27c4f06
add commit call to protocol
Jun 20, 2023
751660f
handle autocommit property
Jun 20, 2023
3fbaa3c
fix user not recognized error
Jun 26, 2023
364437b
add map for default properties and values
Jun 26, 2023
9347eeb
add autocommit property
Jun 26, 2023
a79a5fb
cosmetics
Jun 26, 2023
ab51642
add rollback of current statement
Jun 26, 2023
b96cdd7
add key for fetch size
Jun 26, 2023
c65681c
add frame fetch with size
Jun 26, 2023
33bca81
implement fetch size
Jun 26, 2023
e8bd6ca
add fetch size to protocol
Jun 26, 2023
e9105ac
remove unused default value and field
Jun 26, 2023
2b44f60
rename to PropertyDefaults
Jun 26, 2023
3911ce2
add update count to all frame types
Jun 28, 2023
bd411e7
implement unparameterized statement batches
Jun 28, 2023
2aa105d
make execute public in unparameterized statement batch
Jun 28, 2023
29c6f3b
rename scalar to scalars
Jun 28, 2023
6004fac
add namedIndexes and valueMap transformation
Jun 28, 2023
2271be5
implement parameterized statements
Jun 28, 2023
2f9c49c
add list serialization
Jun 28, 2023
9655459
refactor relational utils
Jun 28, 2023
4f8ae72
add parameter meta retrieval
Jun 28, 2023
350fb65
refactor
Jun 28, 2023
f506cef
add parameterized and prepared statements
Jun 28, 2023
86084dd
implement deserialization
Jun 28, 2023
3dbadd5
add parameterized and prepared statements
Jun 28, 2023
13eb25d
cosmetics
Jun 28, 2023
3e90edc
add prepared and parameterized statements
Jun 28, 2023
a960214
add connection check
Jul 3, 2023
c2fe79a
refactor client properties
Jul 3, 2023
0d49326
add dbms version request
Jul 3, 2023
b39e77e
implement version info request
Jul 3, 2023
21f7f9d
fixed rebase conflicts
datomo Jul 5, 2023
5aa8ea1
fix inappropriate call to getRowCount
Jul 6, 2023
1d2c9b9
add dbms version retrieval
Jul 6, 2023
8331479
fix openBatches null
Jul 6, 2023
b1e5aba
add meta message skeletons
Jul 6, 2023
a3f5800
add meta rpc calls
Jul 6, 2023
322ac74
add table message
Jul 6, 2023
988cfde
add tables to table response
Jul 6, 2023
ca6a496
implement table meta
Jul 6, 2023
8d2d29c
minor abjustement to result fetching and switched order of commit in …
datomo Jul 6, 2023
5919b56
add filtering by table type
Jul 7, 2023
8a725b6
add table types to meta
Jul 7, 2023
f44f085
rename schema to namespace
Jul 7, 2023
4c097fa
add get namespaces
Jul 7, 2023
c8f2ce5
make method static
Jul 7, 2023
02b3e2a
refactor pattern creation to dedicated method
Jul 7, 2023
7ca94d9
add column meta
Jul 7, 2023
ecdbbc0
add column meta
Jul 7, 2023
df97446
add column meta
Jul 8, 2023
970b834
add primary keys to meta
Jul 8, 2023
5d1cc4a
add catalogs to meta
Jul 8, 2023
2264b99
add imported keys to meta
Jul 8, 2023
6aa8a21
add exported keys to meta
Jul 8, 2023
7da47d6
add types to meta
Jul 8, 2023
d73108b
add indexes to meta
Jul 8, 2023
d925043
refactor reorder methods
Jul 9, 2023
4e9fcff
refactor reorder methods
Jul 9, 2023
8724065
refactor named value processor
Jul 9, 2023
7883b4b
discern indexed and named parameters
Jul 9, 2023
3c531a2
discern indexed and named parameters
Jul 9, 2023
435a964
add indexed statement batch to protocol
Jul 9, 2023
f6f0d70
implement indexed value batches
Jul 9, 2023
4b98521
implement indexed value batches
Jul 9, 2023
6652ba4
implement indexed value batches
Jul 9, 2023
dec22de
tidy up type enum
Jul 9, 2023
232ea01
implement poly interval serialization
Jul 9, 2023
d522e93
add udt serialization
Jul 9, 2023
bd07845
add poly list serialization
Jul 9, 2023
9d4b0ff
add map and document serialization
Jul 9, 2023
3d342e2
add graph serialization
Jul 9, 2023
26aedc1
add graph serialization
Jul 9, 2023
644d1a6
fix nullable of bug
Jul 10, 2023
a35d347
fix repeated frame bug
Jul 10, 2023
2ddefe1
meta bugfix
Jul 12, 2023
3ae3ecf
fix client properties
Jul 12, 2023
d390537
fix client properties
Jul 12, 2023
c0d112c
refactor statement properties
Jul 12, 2023
361ff9c
further property refactor
Jul 12, 2023
98b73a6
add more meta
Jul 13, 2023
40bcf82
refactor statement manager
Jul 13, 2023
e309096
add todo
Jul 13, 2023
603a290
fix connection closing
Jul 13, 2023
99538be
indexed parameters bug
Jul 14, 2023
2e21bf9
fix exception forwarding
Jul 14, 2023
cd496f7
add cancel flag to transaction
Jul 15, 2023
887c637
remove cancel flag to transaction
Jul 15, 2023
ad553ef
add get procedures meta
Jul 15, 2023
7510427
add get procedure columns meta
Jul 15, 2023
b8e92d3
add new meta system
Jul 16, 2023
108cc15
add client info properties
Jul 18, 2023
d2fac15
cosmetics
Jul 18, 2023
029bc15
add error for unknown user
Jul 18, 2023
e4ab823
rename proto interface plugin class
Jul 18, 2023
4dda7ce
fix exception handling bug
Jul 18, 2023
90a8708
fix meta bug
Jul 18, 2023
4cb799d
parameterized statement bugfix
Jul 18, 2023
2de5ea0
add info to unregistered client exception
Jul 18, 2023
16ebbe0
cosmetics
Jul 18, 2023
9667572
bugs for david
Jul 19, 2023
47ab54d
general bugfixing
Jul 19, 2023
c81850d
ProcessorTest method
Jul 19, 2023
ff0d562
add bypass for old default password
Jul 19, 2023
4904a22
add auto kick of inactive connections
Jul 19, 2023
b7fa945
added missing imports
Jul 20, 2023
c28a196
remove unused junit 5 dependency
Jul 20, 2023
242a2c1
bump mockito version
Jul 20, 2023
a69021c
delete processor test class
Jul 20, 2023
ede268c
refactor statements not quite working yet
Jul 20, 2023
14686f5
refactor fetching (needs further work to make continuous)
Jul 21, 2023
920affd
fix commit deadlock
Jul 21, 2023
3d5da18
remove redundant commit after ddl
Jul 21, 2023
be59d87
refactor plugin server service structure
Jul 21, 2023
f5611f6
implement heartbeat
Jul 23, 2023
1d226dd
implement heartbeat options to plugin settings
Jul 23, 2023
5e22e59
remove unused parameter form fetch request
Jul 23, 2023
b6a5c5c
cleanup unsed message fields
Jul 23, 2023
2103494
add document fetching and serialization
Jul 23, 2023
22b05af
implement fetch method based on language + codestyle
Jul 23, 2023
c3b842f
prepare de(serialization) of row ids
Jul 24, 2023
80b4320
add protobuf messages for advanced error details
Jul 24, 2023
dc13385
add constructor from SQLException for ProtoInterfaceService Exception
Jul 25, 2023
a7c4e85
sql exception used as cause of proto interface service exception
Jul 25, 2023
17116fd
add exception handling using errorDetails
Jul 25, 2023
b2d598b
add unit tests for url args
Jul 27, 2023
cea81aa
fix hertbeat interval bug
Aug 1, 2023
002d17f
fix typo in unit test
Aug 1, 2023
b257ae1
add connection tests
Aug 1, 2023
4cd6027
fix timestamp serialization error
Aug 1, 2023
7ed7c99
add tests for statement
Aug 1, 2023
5729283
add statement integration tests
Aug 1, 2023
006c0b1
addTimezoneTest
Aug 2, 2023
b30f3d6
add meta unit tests
Aug 2, 2023
76ac30e
add meta unit tests
Aug 2, 2023
966f701
add result set test
Aug 2, 2023
515ad8f
add result set unit tests
Aug 3, 2023
ecf1346
fix tests for result set
Aug 3, 2023
ed925de
unit tests
Aug 3, 2023
88bc4fd
generify statement processing
Aug 5, 2023
2c3f5a9
implement namespace handling
Aug 5, 2023
7ee681e
begin document implementation
Aug 5, 2023
b3925ab
remove statement properties from PIStatement
Aug 6, 2023
6ca1712
remove builder pattern from indexed statement
Aug 6, 2023
5c2fd18
replace getters and setters with lombok ones
Aug 6, 2023
7b247bd
remove builder pattern from client
Aug 6, 2023
6935d4e
clean up statement protocol files
Aug 7, 2023
f4d5802
clean up transaction and connection protocol files
Aug 7, 2023
f19133e
move frames and fetching to statements
Aug 7, 2023
4e33178
split meta.proto into different files
Aug 7, 2023
841ef45
refactor protocol files
Aug 7, 2023
6bee9ec
clean up proto imports
Aug 7, 2023
fba105a
refactor statements
Aug 7, 2023
e63043e
refactor for coherent field names
Aug 7, 2023
ddf98c9
refactor statement properties
Aug 7, 2023
1547195
refactor namespace meta messages into different files
Aug 11, 2023
9bbb6d7
minor bugfixing
Aug 12, 2023
d51a4e6
remove unused imports
Aug 12, 2023
4779962
add serializer unit tests
Aug 13, 2023
284a138
bugfix
Aug 17, 2023
0d2ee15
add continuous fetching using new functions to relational result retr…
Aug 17, 2023
d6cd4d9
add continuous fetching using new functions to document result retriever
Aug 17, 2023
3299ab2
use new getter for column visibility (used for jdbc pseudo column flag)
Aug 17, 2023
8576496
bugfix
TobiasHafner Aug 23, 2023
94b2f47
rename statement processing components
TobiasHafner Aug 23, 2023
f3f3e71
add execute without fetch size
TobiasHafner Aug 23, 2023
134a3a6
fixed new Result representations
datomo Aug 25, 2023
220849a
merge
TobiasHafner Aug 26, 2023
d450a2e
add import
TobiasHafner Aug 26, 2023
47ab9b6
implement result iterator handling to statements
TobiasHafner Aug 26, 2023
d5befc0
new fetch size handling
TobiasHafner Sep 1, 2023
b25dac7
code cleanup
TobiasHafner Sep 1, 2023
b5214e0
code cleanup
TobiasHafner Sep 1, 2023
e888988
bugfix
Sep 14, 2023
bbd278a
added missing conditional reorder for physical mapping to document
datomo Sep 15, 2023
bf7fc16
document executor fix
Sep 18, 2023
80dc9e5
remove unused fields
Sep 19, 2023
a9b8b55
cleanup
Sep 20, 2023
9c5b746
Fix inconsistent field names
gartens Sep 22, 2023
95b4621
Fix prepared statements with named parameters
gartens Sep 25, 2023
c3ab6fb
Only look up value once
gartens Sep 25, 2023
25e1eee
updated proto files to support go
Oct 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ nbdist/
# Avoid ignoring Gradle wrapper jar file (.jar files are usually ignored)
!/gradle/wrapper/gradle-wrapper.jar

/geode-adapter/vf.gf.server.status.cmd
/geode-adapterTemplate/vf.gf.server.status.cmd
/core/custom-namespace-model.json
/testTestCsv/**
!/libs/avatica-1.16.0-POLYPHENYDB-shaded.jar
Expand Down
1 change: 0 additions & 1 deletion build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ buildscript {
}
}


plugins {
id "org.jetbrains.kotlin.jvm" version "1.5.31"
id "org.jetbrains.gradle.plugin.idea-ext" version "1.1.1"
Expand Down
4 changes: 3 additions & 1 deletion core/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,10 @@ dependencies {
api group: 'io.activej', name: 'activej-serializer', version: activej_serializer_version // Apache 2.0

// --- Test Compile ---
// https://mvnrepository.com/artifact/org.mockito/mockito-core
testImplementation group: 'org.mockito', name: 'mockito-core', version: '5.4.0'
testImplementation group: "junit", name: "junit", version: junit_version
testImplementation group: "org.hamcrest", name: "hamcrest-core", version: hamcrest_core_version // BSD 3-clause
testImplementation group: "org.hamcrest", name: "hamcrest-core", version: hamcrest_core_version


}
Expand Down
30 changes: 11 additions & 19 deletions core/src/main/java/org/polypheny/db/PolyImplementation.java
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,6 @@ public class PolyImplementation<T> {

@Accessors(fluent = true)
private final boolean isDDL;
private Iterator<T> iterator;
private boolean isOpen;
private StatementEvent statementEvent;
private int batch;


/**
Expand Down Expand Up @@ -177,13 +173,6 @@ public Bindable<T> getBindable() {
}


public boolean hasMoreRows() {
if ( iterator == null ) {
throw new GenericRuntimeException( "Implementation was not opened" );
}
return iterator.hasNext();
}

public List<ColumnMetaData> getColumns() {
if ( columns != null ) {
return columns;
Expand Down Expand Up @@ -240,10 +229,6 @@ public ResultIterator<T> execute( Statement statement, int batch, boolean isAnal


private Iterator<T> createIterator( Bindable<T> bindable, Statement statement, boolean isAnalyzed ) {
if ( iterator != null ) {
return this.iterator;
}

if ( isAnalyzed ) {
statement.getOverviewDuration().start( "Execution" );
}
Expand All @@ -253,9 +238,7 @@ private Iterator<T> createIterator( Bindable<T> bindable, Statement statement, b
statement.getOverviewDuration().stop( "Execution" );
}

this.iterator = enumerable.iterator();

return this.iterator;
return enumerable.iterator();
}


Expand Down Expand Up @@ -388,6 +371,10 @@ private ResultIterator( Iterator<T> iterator, Statement statement, int batch, bo


public List<List<T>> getRows() {
return getRows(batch);
}

public List<List<T>> getRows(int rowCount) {

StopWatch stopWatch = null;
try {
Expand All @@ -397,7 +384,7 @@ public List<List<T>> getRows() {
}
List<List<T>> res = new ArrayList<>();
int i = 0;
while ( i++ < batch && iterator.hasNext() ) {
while ( i++ < rowCount && iterator.hasNext() ) {
res.add( List.of( (T[]) iterator.next() ) );
}

Expand Down Expand Up @@ -478,6 +465,11 @@ public void close() throws Exception {
}
}


public boolean hasMoreRows() {
return iterator.hasNext();
}

}


Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* Copyright 2019-2023 The Polypheny Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.polypheny.db.algebra.enumerable.document;

import org.polypheny.db.algebra.AlgNode;
import org.polypheny.db.algebra.core.document.DocumentFilter;
import org.polypheny.db.algebra.enumerable.EnumerableAlg;
import org.polypheny.db.algebra.enumerable.EnumerableAlgImplementor;
import org.polypheny.db.plan.AlgOptCluster;
import org.polypheny.db.plan.AlgTraitSet;
import org.polypheny.db.rex.RexNode;
import org.polypheny.db.schema.trait.ModelTrait;

public class EnumerableDocumentFilter extends DocumentFilter implements EnumerableAlg {

/**
* Creates a {@link DocumentFilter}.
* {@link ModelTrait#DOCUMENT} native node of a filter.
*
* @param cluster
* @param traits
* @param input
* @param condition
*/
protected EnumerableDocumentFilter( AlgOptCluster cluster, AlgTraitSet traits, AlgNode input, RexNode condition ) {
super( cluster, traits, input, condition );
}


@Override
protected AlgNode copy( AlgTraitSet traitSet, AlgNode input, RexNode condition ) {
return null;
}


@Override
public Result implement( EnumerableAlgImplementor implementor, Prefer pref ) {
return null;
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/*
* Copyright 2019-2023 The Polypheny Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.polypheny.db.algebra.enumerable.document;

import org.polypheny.db.algebra.AlgNode;
import org.polypheny.db.algebra.convert.ConverterRule;
import org.polypheny.db.algebra.enumerable.EnumerableConvention;
import org.polypheny.db.algebra.logical.document.LogicalDocumentFilter;

public class EnumerableDocumentFilterRule extends ConverterRule {

public EnumerableDocumentFilterRule() {
super( LogicalDocumentFilter.class, EnumerableConvention.NONE, EnumerableConvention.INSTANCE, "EnumerableDocumentFilterRule" );
}


@Override
public AlgNode convert( AlgNode alg ) {
LogicalDocumentFilter filter = (LogicalDocumentFilter) alg;

return new EnumerableDocumentFilter(
filter.getCluster(),
filter.getTraitSet().replace( EnumerableConvention.INSTANCE ),
filter.getInput(),
filter.condition );
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,9 @@ private static AlgNode handleDocumentEntity( AlgOptRuleCall call, Scan<?> scan,

private AlgNode handleRelationalEntity( AlgOptRuleCall call, Scan<?> scan, AllocationEntity alloc ) {
AlgNode alg = AdapterManager.getInstance().getAdapter( alloc.adapterId ).getRelScan( alloc.id, call.builder() );
alg = attachReorder( alg, scan, call.builder() );
if ( scan.getModel() == scan.entity.namespaceType ) {
alg = attachReorder( alg, scan, call.builder() );
}

if ( scan.getModel() != scan.entity.namespaceType ) {
// cross-model queries need a transformer first, we let another rule handle that
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
import org.polypheny.db.algebra.logical.relational.LogicalSort;
import org.polypheny.db.algebra.logical.relational.LogicalUnion;
import org.polypheny.db.catalog.entity.logical.LogicalTable;
import org.polypheny.db.catalog.entity.physical.PhysicalTable;
import org.polypheny.db.plan.AlgOptCluster;
import org.polypheny.db.plan.AlgOptRule;
import org.polypheny.db.plan.AlgOptRuleCall;
Expand Down
Loading
Loading