背景

某些客户场景ck原先的22.3.6.5版本已经满足不了,所以需要升级到24.3.3.1版本。

新版CK镜像

新ck镜像采用内部研发的ck v24.3.3.1版本,不再使用ck官方镜像。其yaml配置与之前的不同在于:

环境变量:不用配置环境变量

文件映射: users.xml和config.xml的配置文件不用映射,已内置。(如果有特殊配置需求,见下文)

日志和数据目录映射的内部路径已变化

  ck:
    image: localhost:5000/yddocker/youdata-clickhouse:24.3.3.1
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.labels.role == master
    networks:
      - youdata
    ports:
      - 8123:8123
      - 9005:9000
      - 9363:9363
    volumes:
      - /youdata/logs/ck:/root/ck/log
      - /youdata/ck/data:/root/ck/data
    healthcheck:
      test: "curl -f http://localhost:8123 || exit 1"
      interval: 60s
      timeout: 10s
      retries: 3
      start_period: 30s
    logging:
      options:
        max-size: 10m
        max-file: 10

升级说明

目前ck单机镜像不在有数默认安装包内,因此常规升级不会动到CK。

目前测试看来,ck升级后整体数据是兼容的。但是安全起见,建议将 /youdata/ck 再cp备份一下。

查看日志ls

容器外部为:/youdata/logs/ck下的clickhouse-server.log 和 clickhouse-server.err.log

容器内部路径为:/root/ck/log

登入客户端

./bin/clickhouse-client --password youdata

当进入容器时,如果直接敲clickhouse回车,也能进入客户端,但是这个不是真正对应的客户端!这个是ck local模式!请务必采用上述命令进入。这个是ck v24才发现的特性,之前的v22 v23行为不一样。

特定化配置 如果需要变更users.xml和config.xml,创建/youdata/ck/config.xml和/youdata/ck/users.xml,加入磁盘映射:

- /youdata/ck/config.xml:/root/ck/config.xml
- /youdata/ck/users.xml:/root/ck/users.xml

config.xml

<clickhouse>
    <logger>
        <level>information</level>
        <log>/root/ck/log/clickhouse-server.log</log>
        <errorlog>/root/ck/log/clickhouse-server.err.log</errorlog>
        <size>1000M</size>
        <count>10</count>
    </logger>

    <http_port>8123</http_port>
    <tcp_port>9000</tcp_port>
    <listen_host>0.0.0.0</listen_host>
    <timezone>Asia/Shanghai</timezone>

    <max_connections>4096</max_connections>
    <max_concurrent_queries>500</max_concurrent_queries>
    <max_server_memory_usage>21474836480</max_server_memory_usage>
    <max_thread_pool_size>10000</max_thread_pool_size>
    <max_server_memory_usage_to_ram_ratio>0.9</max_server_memory_usage_to_ram_ratio>
    <total_memory_profiler_step>4194304</total_memory_profiler_step>
    <total_memory_tracker_sample_probability>0</total_memory_tracker_sample_probability>
    <uncompressed_cache_size>8589934592</uncompressed_cache_size>
    <mark_cache_size>5368709120</mark_cache_size>

    <path>/root/ck/data</path>
    <tmp_path>/root/ck/data/tmp/</tmp_path>
    <user_files_path>/root/ck/data/user_files/</user_files_path>
    <format_schema_path>/root/ck/data/path/format_schemas/</format_schema_path>
    <user_directories>
        <users_xml>
            <path>/root/ck/users.xml</path>
        </users_xml>
        <local_directory>
            <path>/root/ck/data/path/access/</path>
        </local_directory>
    </user_directories>

    <prometheus>
        <endpoint>/metrics</endpoint>
        <port>9363</port>
        <metrics>true</metrics>
        <events>true</events>
        <asynchronous_metrics>true</asynchronous_metrics>
        <status_info>true</status_info>
    </prometheus>

    <query_log>
        <partition_by>toYYYYMMDD(event_date)</partition_by>
        <order_by>event_time, initial_query_id</order_by>
        <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
    </query_log>

    <metric_log>
        <ttl>event_date + INTERVAL 90 DAY DELETE</ttl>
        <collect_interval_milliseconds>2000</collect_interval_milliseconds>
    </metric_log>

    <asynchronous_metric_log>
        <ttl>event_date + INTERVAL 90 DAY DELETE</ttl>
    </asynchronous_metric_log>
</clickhouse>

users.xml

<clickhouse>
    <profiles>
        <default>
            <max_memory_usage>4294967296</max_memory_usage>
            <max_threads>6</max_threads>
            <max_bytes_before_external_group_by>2147483648</max_bytes_before_external_group_by>
            <max_bytes_before_external_sort>3435973836</max_bytes_before_external_sort>
            <load_balancing>random</load_balancing>
            <join_use_nulls>1</join_use_nulls>
            <max_execution_time>120</max_execution_time>
            <optimize_move_to_prewhere>0</optimize_move_to_prewhere>
        </default>

        <readonly>
            <readonly>1</readonly>
        </readonly>
    </profiles>

    <users>
        <default>
            <password>youdata</password>
            <networks>
                <ip>::/0</ip>
            </networks>
            <profile>default</profile>
            <quota>default</quota>
        </default>
    </users>

    <quotas>
        <default>
            <interval>
                <duration>3600</duration>
                <queries>0</queries>
                <errors>0</errors>
                <result_rows>0</result_rows>
                <read_rows>0</read_rows>
                <execution_time>0</execution_time>
            </interval>
        </default>
    </quotas>
</clickhouse>