Dropping All Vertexes #3769
Answered
by
hadoopmarc
ZachMurray
asked this question in
Q&A
Dropping All Vertexes
#3769
-
|
Why is I want JanusGraph to let go of all its Vertices on a Cassandra [Docker container]. 2023-05-10 09:23:19,141 [INFO] [c.d.o.d.i.c.ContactPoints.main] :: Contact point localhost:9042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-10 09:23:19,226 [INFO] [c.d.o.d.i.c.DefaultMavenCoordinates.main] :: DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.15.0
2023-05-10 09:23:19,726 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] :: Using native clock for microsecond precision
2023-05-10 09:23:19,987 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] :: [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=37423099)=null; please provide the correct local DC, or check your contact points
2023-05-10 09:23:20,231 [INFO] [o.j.g.i.UniqueInstanceIdRetriever.main] :: Generated unique-instance-id=c0a8563c404-rmt-lap-win201
2023-05-10 09:23:20,252 [INFO] [c.d.o.d.i.c.ContactPoints.main] :: Contact point localhost:9042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-10 09:23:20,297 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] :: Using native clock for microsecond precision
2023-05-10 09:23:20,346 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] :: [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=1ec16fcd)=null; please provide the correct local DC, or check your contact points
2023-05-10 09:23:20,362 [INFO] [o.j.d.c.ExecutorServiceBuilder.main] :: Initiated fixed thread pool of size 40
2023-05-10 09:23:20,469 [INFO] [o.j.g.d.StandardJanusGraph.main] :: Gremlin script evaluation is disabled
2023-05-10 09:23:20,494 [INFO] [o.j.d.l.k.KCVSLog.main] :: Loaded unidentified ReadMarker start time 2023-05-10T14:23:20.494021Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@f1d0004
2023-05-10 09:23:20,549 [WARN] [o.j.g.t.StandardJanusGraphTx.main] :: Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:23:21,835 [INFO] [Main.main] :: g.V().count().next(): 70005
2023-05-10 09:23:21,839 [WARN] [o.j.g.t.StandardJanusGraphTx.main] :: Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:26:13,733 [WARN] [o.j.g.t.StandardJanusGraphTx.main] :: Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:26:15,427 [INFO] [Main.main] :: g.V().count().next(): 0
2023-05-10 09:33:40,131 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Lock write succeeded but took too long: duration PT0.197258S exceeded limit PT0.1S
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000008080, c=0x0x02] with timestamp 2023-05-09T20:16:07.308532Z
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000008180, c=0x0x02] with timestamp 2023-05-09T20:16:08.146591Z
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000008280, c=0x0x02] with timestamp 2023-05-09T20:16:08.988543Z
...
2023-05-10 09:38:03,876 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-10T14:28:09.380600Z
2023-05-10 09:38:03,876 [INFO] [o.j.d.u.BackendOperation.main] :: Temporary exception during backend operation [CacheMutation]. Attempting backoff retry.
org.janusgraph.diskstorage.locking.consistentkey.ExpiredLockException: Expired lock on KeyColumn [k=0x0x7800000000018080, c=0x0x02]: lock timestamp 2023-05-10T14:28:09.380600Z Micros is older than storage.lock.expiry-time=PT5M
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:472) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:131) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:348) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:178) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:157) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:77) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:99) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:96) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:66) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:52) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:96) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:147) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:210) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:147) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:917) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1558) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:322) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) [gremlin-core-3.6.2.jar:3.6.2]
at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.commit(JanusGraphBlueprintsGraph.java:300) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
at Test6.main(Test6.java:28) [test-classes/:?]
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:09:05.345276Z
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:14:34.917340Z
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:21:07.864503Z
2023-05-10 09:38:03,933 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] :: Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-10T14:28:09.380600Z
...
2023-05-10 09:38:45,394 [INFO] [o.j.d.u.BackendOperation.main] :: Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. ...
Process finished with exit code -1import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.janusgraph.core.JanusGraph;
import org.janusgraph.core.JanusGraphFactory;
import org.janusgraph.core.JanusGraphVertex;
public class Test {
private static final Logger logger = LogManager.getLogger(Main.class);
public static void main(String[] args) {
JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "cql").set("storage.hostname", "localhost:9042").open();
GraphTraversalSource g = janusGraph.traversal();
logger.info("g.V().count().next():\t" + g.V().count().next());
g.V().drop().iterate();
logger.info("g.V().count().next():\t" + g.V().count().next());
janusGraph.tx().commit();
logger.info("g.V().count().next():\t" + g.V().count().next());
JanusGraphVertex janusGraphVertex = janusGraph.addVertex();
janusGraph.close();
}
}I've tried just doing a straight 2023-05-10 08:05:30,437 [WARN] [o.j.g.t.StandardJanusGraphTx.main] :: Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 08:05:31,861 [INFO] [Main.main] :: g.V().count().next(): 70003
2023-05-10 08:05:31,866 [WARN] [o.j.g.t.StandardJanusGraphTx.main] :: Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 08:05:33,045 [INFO] [Main.main] :: g.V().count().next(): 70003If I try to open a new transaction, the Java execution will throw-up an error saying there's already one open. Exception in thread "main" java.lang.IllegalStateException: Stop the current transaction before opening another
at org.apache.tinkerpop.gremlin.structure.Transaction$Exceptions.transactionAlreadyOpen(Transaction.java:143)
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.open(AbstractTransaction.java:93)
at Test5_2.main(Test5_2.java:15)How do you |
Beta Was this translation helpful? Give feedback.
Answered by
hadoopmarc
May 11, 2023
Replies: 2 comments 2 replies
-
|
If you try to drop all vertices in one transaction, you load all vertices into memory which will not necessarily fit. Usually, transactions with 10,000 mutations work fine. Fortunately, for dropping an entire graph there is a simpler solution: https://docs.janusgraph.org/common-questions/#dropping-a-database |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Even limiting of 1 just stalls out JanusGraph + Cassandra. This happens for either just ...
2023-05-11 08:38:35,063 [INFO] [Main.main] :: g.V().count().next(): 70013
2023-05-11 08:38:35,067 [WARN] [o.j.g.t.StandardJanusGraphTx.main] :: Query requires iterating over all vertices [[]]. For better performance, use indexes logger.info("g.V().count().next():\t" + g.V().count().next());
g.V().drop().iterate().limit(1);
//g.V().drop().limit(1).iterate();
//g.V().drop().limit(1);
logger.info("g.V().count().next():\t" + g.V().count().next()); |
Beta Was this translation helpful? Give feedback.
-
|
Try to do it the other was around: Also, 70,000 vertices should fit in memory, maybe give your JVM some more memory, defaults possibly low in the JanusGraph with Gremlin Server docker container. |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Yup, found this as I was rewording the question!
https://stackoverflow.com/questions/76229209/why-does-limit-of-1-not-record-drop-1-vertex |
Beta Was this translation helpful? Give feedback.
Answer selected by
ZachMurray
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Try to do it the other was around:
g.V().limit(1).drop().iterate()Also, 70,000 vertices should fit in memory, maybe give your JVM some more memory, defaults possibly low in the JanusGraph with Gremlin Server docker container.