FAQ-万象数据库-database space exceeded

问题描述/异常栈
万象数据库出现服务异常告警,agent-common 出现database space exceeded 关键字
[251102.115831][MainThread] E  Error when update agent config with etcd. <class 'grpc._channel._InactiveRpcError'>/<_InactiveRpcError of RPC that terminated with:
        status = StatusCode.RESOURCE_EXHAUSTED
        details = "etcdserver: mvcc: database space exceeded"
        debug_error_string = "{"created":"@1762055911.949140652","description":"Error received from peer ipv4:111.124.204.139:2379","file":"src/core/lib/surface/call.cc","file_line":1066,"grpc_message":"etcdserver: mvcc: database space exceeded","grpc_status":8}"
发现版本
万象数据库-3.3.7.3
解决方案
临时方案:压缩并清理etcd存储数据 长期方案:升级ETCD服务,开启自动压缩清理
# 临时方案操作步骤
## 登录etcd容器
docker exec -it mysql-paas-etcd /bin/bash
## 执行压缩
rev=`etcdctl --endpoints=$ETCD_LISTEN_CLIENT_URLS endpoint status --write-out="json" | egrep -o '"revision":[0-9]*' | egrep -o '[0-9].*'`

result=`etcdctl --endpoints=$ETCD_LISTEN_CLIENT_URLS compact "${rev}"`

## 此命令可能出现超时,等待10分钟左右重试一般可执行成功
result=`etcdctl --endpoints=$ETCD_LISTEN_CLIENT_URLS defrag`

size=`etcdctl --endpoints=$ETCD_LISTEN_CLIENT_URLS endpoint status|awk -F ',' '{print $4}'`
echo "压缩后的etcd数据大小如下"
echo $size


# 长期方案参考灵犀文档:https://docs.popo.netease.com/lingxi/382da0713ab84b579fad3bf188ec85c5
问题原因
etcd未开启自动压缩清理功能,导致数据持续增长直到超出存储空间配额。

作者:Simon