kettle -> doris 通过java集成kettle导入数据到doris报错

[fengcharles]

使用kettle本身自带的导入功能没有问题，数据可以正常导入。

如果使用java调用kettle 会经常出现下面这种错误。 请问下这个具体错误原因是什么？

org.apache.doris.common.DdlException: errCode = 2, detailMessage = already stopped, skip waiting for close. cancelled/!eos: : 1/0
at org.apache.doris.common.ErrorReport.reportDdlException(ErrorReport.java:67) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.handleInsertStmt(StmtExecutor.java:847) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:326) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-09-29 06:00:35,040 WARN (doris-mysql-nio-pool-159|16786) [StmtExecutor.execute():380] errors when abort txn
org.apache.doris.transaction.TransactionNotFoundException: errCode = 2, detailMessage = transaction not found
at org.apache.doris.transaction.DatabaseTransactionMgr.abortTransaction(DatabaseTransactionMgr.java:949) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:210) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:205) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:376) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-09-29 10:19:29,576 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:19:29,601 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:19:29,619 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:20:09,964 WARN (doris-mysql-nio-pool-162|17050) [NMysqlChannel.readAll():71] Read channel exception, ignore. Exception: java.io.IOException: Connection reset by peer
2021-09-29 10:20:09,964 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.processOnce():525] Null packet received from network. remote: 172.16.80.80:60416
2021-09-29 10:20:09,964 WARN (doris-mysql-nio-pool-162|17050) [ReadListener.lambda$handleEvent$0():58] Exception happened in one session(org.apache.doris.mysql.nio.NConnectContext@1f7af005).
java.io.IOException: Error happened when receiving packet.
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:526) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]

[fengcharles]

2021-09-29 06:00:01,263 WARN (doris-mysql-nio-pool-159|16786) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 06:00:01,264 WARN (doris-mysql-nio-pool-159|16786) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 06:00:01,265 WARN (doris-mysql-nio-pool-159|16786) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 06:00:35,038 WARN (thrift-server-pool-3|2189) [Coordinator.updateFragmentExecStatus():1369] one instance report fail, query_id=9ea4034edca042ca-a5a14742cee0f3fd instance_id=9ea4034edca042ca-a5a14742cee0f3fe
2021-09-29 06:00:35,038 WARN (thrift-server-pool-3|2189) [Coordinator.updateStatus():670] one instance report fail throw updateStatus(), need cancel. job id: -1, query id: 9ea4034edca042ca-a5a14742cee0f3fd, instance id: 9ea4034edca042ca-a5a14742cee0f3fe
2021-09-29 06:00:35,038 WARN (doris-mysql-nio-pool-159|16786) [StmtExecutor.handleInsertStmt():846] insert failed: already stopped, skip waiting for close. cancelled/!eos: : 1/0
2021-09-29 06:00:35,038 WARN (doris-mysql-nio-pool-159|16786) [StmtExecutor.handleInsertStmt():894] handle insert stmt fail: insert_9ea4034edca042ca-a5a14742cee0f3fd
org.apache.doris.common.DdlException: errCode = 2, detailMessage = already stopped, skip waiting for close. cancelled/!eos: : 1/0
at org.apache.doris.common.ErrorReport.reportDdlException(ErrorReport.java:67) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.handleInsertStmt(StmtExecutor.java:847) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:326) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-09-29 06:00:35,040 WARN (doris-mysql-nio-pool-159|16786) [StmtExecutor.execute():380] errors when abort txn
org.apache.doris.transaction.TransactionNotFoundException: errCode = 2, detailMessage = transaction not found
at org.apache.doris.transaction.DatabaseTransactionMgr.abortTransaction(DatabaseTransactionMgr.java:949) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:210) ~[palo-fe.jar:3.4.0]
at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:205) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:376) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:243) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:199) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:337) ~[palo-fe.jar:3.4.0]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:535) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-09-29 10:19:29,576 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:19:29,601 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:19:29,619 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:20:09,964 WARN (doris-mysql-nio-pool-162|17050) [NMysqlChannel.readAll():71] Read channel exception, ignore. Exception: java.io.IOException: Connection reset by peer
2021-09-29 10:20:09,964 WARN (doris-mysql-nio-pool-162|17050) [ConnectProcessor.processOnce():525] Null packet received from network. remote: 172.16.80.80:60416
2021-09-29 10:20:09,964 WARN (doris-mysql-nio-pool-162|17050) [ReadListener.lambda$handleEvent$0():58] Exception happened in one session(org.apache.doris.mysql.nio.NConnectContext@1f7af005).
java.io.IOException: Error happened when receiving packet.
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:526) ~[palo-fe.jar:3.4.0]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2021-09-29 10:32:01,089 WARN (doris-mysql-nio-pool-165|17065) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:32:01,104 WARN (doris-mysql-nio-pool-165|17065) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)
2021-09-29 10:32:01,118 WARN (doris-mysql-nio-pool-165|17065) [ConnectProcessor.dispatch():348] Unsupported command(COM_STMT_PREPARE)

[EmmyMiao87]

Unsupported command(COM_STMT_PREPARE)
这个是关键的错误信息，doris 并不支持这个命令。
推荐你用jdbc~

[fengcharles]

好的，我查看下 是不是JDBC有问题， 理论上我用java 调用应该就是JDBC 执行。

另外还有个很关键的问题是 每次 导入到 doris 里面 499个后，就开始报错了。不能超过500个，是否有相关的配置参数影响到了？