0

We have setup three node Cockroach-DB cluster(version: 22.2.14) on GKE kubernetes cluster on GCP using Helm. Helm chart version is 6.0.3. Suddenly, we’re getting below error on Cockroach-DB pod’s. Also, Cockroach-DB cluster is not form correctly. All Cockroach DB pods are restarted gracefully and getting below error.

E240708 09:53:55.048576 277 kv/kvserver/queue.go:1127 ⋮ [n1,replicaGC,s1,r755/1:‹/Table/61{0-1}›] 1 ‹operation “replicaGC queue process replica 755” timed out after 1m0.001s (given timeout 1m0s)›: aborted in DistSender: context deadline exceeded initiating graceful shutdown of server too early to drain; used hard shutdown instead

Can anyone help me to resolve the issue?

Moreover, How we can enable debug mode in Cockroach DB cluster?

Thank you in advance…

We have checked log but did not help it to resolve the issue.

What we can do to resolve the issue?

Improve this question
2
  • Hey, can you give the helm link and the doc that you have followed? or maybe a full pod log (In json) would be helpful. Please share them
    – Nani
    CommentedJul 9, 2024 at 17:57
  • Hi, to gather more information about your cluster to debug, you can try the cockroach debug zip (link), but note that this is an expensive operation and impacts cluster performance. Also, just for your information, from v23.1.8 and onward, you can use cockroach sql --debug-sql-cli to start the SQL client with debug mode enabled, see also here.
    – IsaIkari
    CommentedNov 11, 2024 at 15:26
Load 5 more related questions Show fewer related questions

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.