如何在Cassandra中正确配置CDC
卡桑德拉版本:3.11 我已经在cassandra.yaml中启用了cdc:如何在Cassandra中正确配置CDC,cassandra,cdc,Cassandra,Cdc,卡桑德拉版本:3.11 我已经在cassandra.yaml中启用了cdc: cdc_enabled: true cdc_raw_directory: /var/lib/cassandra/data/cdc_raw 并启用了表: cqlsh> describe cycling.cyclist_name; CREATE TABLE cycling.cyclist_name ( id uuid PRIMARY KEY, firstname text, lastnam
cdc_enabled: true
cdc_raw_directory: /var/lib/cassandra/data/cdc_raw
并启用了表:
cqlsh> describe cycling.cyclist_name;
CREATE TABLE cycling.cyclist_name (
id uuid PRIMARY KEY,
firstname text,
lastname text
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND cdc = true <<<<<<<<<
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
我执行了一个小Python脚本,在新表中插入了一些数据:
>>> for lp in range(50000):
... session.execute("INSERT INTO cycling.cyclist_name (lastname, firstname, id) VALUES (%s, %s, %s)", ["RATTO_BULK", "Rissella", uuid.uuid4()])
...
但即使在这50000个插入脚本之后,cdc_原始目录仍然是空的。有人能解释一下CDC是如何在卡桑德拉工作的吗
root@docker-desktop:/var/lib/cassandra# ls -la /var/lib/cassandra/data/cdc_raw/
total 8
drwxr-xr-x 2 cassandra cassandra 4096 Jan 28 09:48 .
drwxr-xr-x 9 cassandra cassandra 4096 Jan 28 10:04 ..
在Cassandra 3.11中,当memtable中的数据由于某种原因被刷新到磁盘时,commitlog段被复制到
cdc_raw
目录中-达到memtable限制、commit log限制或通过nodetool flush
。50k写入可能不足以触发这种情况
在Cassandra 4.0(尚未发布)中,情况略有改善,您可以更快地读取数据。如果您对详细信息感兴趣,您可以查看在以下位置所做的演示-有2到3个关于CDC的演示。在Cassandra 3.11中,当memtable中的数据由于某种原因刷新到磁盘时,commitlog段会被复制到
CDC_raw
目录中-达到memtable限制、提交日志限制或通过nodetool flush
。50k写入可能不足以触发这种情况
在Cassandra 4.0(尚未发布)中,情况略有改善,您可以更快地读取数据。如果您对细节感兴趣,您可以查看在上的演示文稿-有2到3个关于CDC的演示文稿
root@docker-desktop:/var/lib/cassandra# ls -la /var/lib/cassandra/data/cdc_raw/
total 8
drwxr-xr-x 2 cassandra cassandra 4096 Jan 28 09:48 .
drwxr-xr-x 9 cassandra cassandra 4096 Jan 28 10:04 ..