Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cardano-db-sync stops/kills cardano-node #1803

Open
catad83 opened this issue Aug 2, 2024 · 2 comments
Open

cardano-db-sync stops/kills cardano-node #1803

catad83 opened this issue Aug 2, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@catad83
Copy link

catad83 commented Aug 2, 2024

OS
Your OS:
Ubuntu 24.04

Versions
cardano-node 9.1.0 - linux-x86_64 - ghc-8.10
git rev 176f99e51155cb3eaa0711db1c3c969d67438958

cardano-db-sync 13.3.0.0 - linux-x86_64 - ghc-8.10
git revision d21895f

psql (PostgreSQL) 16.3 (Ubuntu 16.3-0ubuntu0.24.04.1)

Build/Install Method
cardano-db-sync - build from source

Run method
PGPASSFILE=config/pgpass-mainnet cardano-db-sync
--config $HOME/cardano-src/cardano-node/db-sync-config.json
--socket-path $HOME/cardano-src/cardano-node/db/node.socket
--state-dir ledger-state
--schema-dir schema

Additional context

Problem Report
Trying to get a node running with db-sync. Running/starting cardano-db-sync crashes cardano-node. Can somebody point me into a direction on why it's this happening? And maybe how to fix it. Thank you all.

Cardano-Node output:

[cardanon:cardano.node.DiffusionInitializationTracer:Critical:5] [2024-08-02 11:08:06.44 UTC] DiffusionErrored (ExceptionInLinkedThread "ThreadId 174" (ApiMisuse ClosedDBError CallStack (from HasCallStack):
prettyCallStack, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/API.hs:350:42 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.API
throwApiMisuse, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/Impl/State.hs:156:28 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.Impl.State
getOpenState, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/Impl.hs:431:49 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.Impl
getTipImpl, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/Impl.hs:287:34 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.Impl
getTip_, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/API.hs:115:7 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.API
getTip, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/API.hs:568:38 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.API
getTipSlot, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ChainDB/Impl/Background.hs:154:48 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ChainDB.Impl.Background
copyToImmutableDB, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ChainDB/Impl/Background.hs:254:22 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ChainDB.Impl.Background))
[cardanon:cardano.node.ChainDB:Error:5] [2024-08-02 11:08:06.44 UTC] Immutable DB was already closed. Double closing.
[cardanon:cardano.node.ChainDB:Info:5] [2024-08-02 11:08:06.44 UTC] Closed Volatile DB.
[cardanon:cardano.node.ChainDB:Info:5] [2024-08-02 11:08:06.44 UTC] Closed db with immutable tip at f06990fbfc8b1fe3454fa374002f244d5473014df1ea5ad25222e70a15df6313 at slot 130986321 and tip 0a89b4644d9db9b0827af02b46815c8b1674b897f370c45c8f41754f574a5c6a at slot 131030595

Shutting down..
cardano-node: ExceptionInLinkedThread "ThreadId 174" (ApiMisuse ClosedDBError CallStack (from HasCallStack):
prettyCallStack, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/API.hs:350:42 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.API
throwApiMisuse, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/Impl/State.hs:156:28 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.Impl.State
getOpenState, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/Impl.hs:431:49 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.Impl
getTipImpl, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/Impl.hs:287:34 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.Impl
getTip_, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/API.hs:115:7 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.API
getTip, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ImmutableDB/API.hs:568:38 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ImmutableDB.API
getTipSlot, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ChainDB/Impl/Background.hs:154:48 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ChainDB.Impl.Background
copyToImmutableDB, called at src/ouroboros-consensus/Ouroboros/Consensus/Storage/ChainDB/Impl/Background.hs:254:22 in ouroboros-consensus-0.20.0.0-c8c9ce6b8e9df299037ea9243bfbe92c8be67d472a67362d1d65d774304e4bc5:Ouroboros.Consensus.Storage.ChainDB.Impl.Background)

cardano-db-sync:

[db-sync-node:Error:255] [2024-08-02 11:08:06.22 UTC] ChainSyncWithBlocksPtcl: AsyncCancelled
[db-sync-node.Subscription:Error:251] [2024-08-02 11:08:06.22 UTC] Identity Application Exception: LocalAddress "/home//cardano-src/cardano-node/db/node.socket" MuxError MuxBearerClosed "<socket: 22> closed when reading data, waiting on next header True"
[db-sync-node.ErrorPolicy:Warning:80] [2024-08-02 11:08:06.22 UTC] IP LocalAddress "/home/
/cardano-src/cardano-node/db/node.socket" ErrorPolicySuspendPeer (Just (ApplicationExceptionTrace (MuxError MuxBearerClosed "<socket: 22> closed when reading data, waiting on next header True"))) 20s 20s
[db-sync-node.Subscription:Notice:84] [2024-08-02 11:08:07.22 UTC] Identity Restarting Subscription after 2.014054817s desired valency 1 current valency 0
[db-sync-node.Subscription:Notice:84] [2024-08-02 11:08:15.21 UTC] Identity Starting Subscription Worker, valency 1
[db-sync-node.Subscription:Notice:264] [2024-08-02 11:08:15.21 UTC] Identity Connection Attempt Start, destination LocalAddress "/home//cardano-src/cardano-node/db/node.socket"
[db-sync-node.Subscription:Error:264] [2024-08-02 11:08:15.21 UTC] Identity Connection Attempt Exception, destination LocalAddress "/home/
/cardano-src/cardano-node/db/node.socket" exception: Network.Socket.connect: <socket: 22>: does not exist (Connection refused)
[db-sync-node.ErrorPolicy:Warning:80] [2024-08-02 11:08:15.21 UTC] IP LocalAddress "/home/***/cardano-src/cardano-node/db/node.socket" ErrorPolicySuspendPeer (Just (ConnectionExceptionTrace Network.Socket.connect: <socket: 22>: does not exist (Connection refused))) 20s 20s
[db-sync-node.Subscription:Warning:84] [2024-08-02 11:08:15.24 UTC] Identity Failed to start all required subscriptions
[db-sync-node:Error:78] [2024-08-02 11:08:15.38 UTC] runDBThread: AsyncCancelled

@catad83 catad83 added the bug Something isn't working label Aug 2, 2024
@kderme
Copy link
Contributor

kderme commented Aug 2, 2024

I think I've seen this issue myself a couple times. I'm assuming it's a node bug, since db-sync shouldn't be able to kill cardano-node

@catad83
Copy link
Author

catad83 commented Aug 2, 2024

I find it odd also. I'm trying to build node 9.0.0 since it states it's compatible with db-sync 13.3.0.0, if it works, it's a bug in node I suppose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants