Caching 在cassandra中,启用行缓存不会提高性能
我在Cassandra中创建了表,并启用了行缓存,但启用它并没有提高查询性能,查询延迟根本没有改变 卡桑德拉版本:3.11.1 配置: 行缓存大小(单位:mb):10240 行缓存保存周期:3600Caching 在cassandra中,启用行缓存不会提高性能,caching,cassandra,row,Caching,Cassandra,Row,我在Cassandra中创建了表,并启用了行缓存,但启用它并没有提高查询性能,查询延迟根本没有改变 卡桑德拉版本:3.11.1 配置: 行缓存大小(单位:mb):10240 行缓存保存周期:3600 table schema: CREATE TABLE worldscope.test_10 ( year int, rank int, cycname text, name text, value_0 float, value_1 float, value_2 float, value_3 floa
table schema:
CREATE TABLE worldscope.test_10 (
year int,
rank int,
cycname text,
name text,
value_0 float,
value_1 float,
value_2 float,
value_3 float,
value_4 float,
value_5 float,
value_6 float,
value_7 float,
value_8 float,
value_9 float,
PRIMARY KEY (year, rank)
) WITH CLUSTERING ORDER BY (rank ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 100000}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 0
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
查询:从测试_10中选择值_1,其中年份=2016
当我运行nodetool时,它显示行缓存被命中,如下所示:
Key Cache : entries 144, size 103.85 KiB, capacity 100 MiB, 567 hits, 723 requests, 0.784 recent hit rate, 14400 save period in seconds
Row Cache : entries 31, size 2.07 MiB, capacity 10 GiB, 279 hits, 310 requests, 0.900 recent hit rate, 3600 save period in seconds
但无论是否启用行缓存,查询延迟都是相同的,在某些测试用例中,启用行缓存后,延迟甚至会变长
当我运行trace时,详细信息如下:
启用行缓存时:
Tracing session: 0667b5d0-1c33-11e8-8e0d-c18210ebaedc
activity | timestamp | source | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 87 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 154 | 10.81.220.6
Read-repair DC_LOCAL [Native-Transport-Requests-1] | 2018-02-27 21:56:51.117000 | 10.81.220.6 | 284 | 10.81.220.6
Row cache hit [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 | 2366 | 10.81.220.6
Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:56:51.119000 | 10.81.220.6 | 2460 | 10.81.220.6
Request complete | 2018-02-27 21:56:51.119563 | 10.81.220.6 | 2563 | 10.81.220.6
未启用行缓存时:
跟踪会话:bc0cd420-1c32-11e8-8e0d-c18210ebaedc
activity | timestamp | source | source_elapsed | client
---------------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_0 from test_10 where year = 20160102 limit 10; [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 124 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 230 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 570 | 10.81.220.6
Acquiring sstable references [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 610 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-27 21:54:46.370000 | 10.81.220.6 | 639 | 10.81.220.6
Key cache hit for sstable 1 [ReadStage-3] | 2018-02-27 21:54:46.370001 | 10.81.220.6 | 690 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 | 847 | 10.81.220.6
Read 10 live and 0 tombstone cells [ReadStage-3] | 2018-02-27 21:54:46.371000 | 10.81.220.6 | 878 | 10.81.220.6
Request complete | 2018-02-27 21:54:46.370998 | 10.81.220.6 | 998 | 10.81.220.6
您可以看到,未启用缓存(998)明显比启用缓存(2563)时快
原因是什么 您是否使用
跟踪运行了一些查询,并且可以发布结果?(见附件)
当缓存命中将阻止磁盘读取时,Cassandra将受益于行缓存。您的缓存的命中率为0.9,这已经相当高了。但由于行缓存的大小约为2M字节,所以我猜您之前的读取是从cassandras memtables提供的,而不是从磁盘读取,因此加速效果非常小
cqlsh:demo> SELECT value_1 FROM test_10 WHERE year = 2018;
value_1
---------
0.1
0.1
0.1
(3 rows)
Tracing session: 8ea1bd70-1ad9-11e8-b37d-63be3c2a5a29
activity | timestamp | source | source_elapsed
-------------------------------------------------------------------------------------------------+----------------------------+--------------+----------------
Execute CQL3 query | 2018-02-26 09:43:53.799000 | 85.10.240.52 | 0
Parsing SELECT value_1 FROM test_10 WHERE year = 2018; [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 262
Preparing statement [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 385
reading data from /85.10.240.60 [SharedPool-Worker-1] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 739
Sending READ message to /85.10.240.60 [MessagingService-Outgoing-/85.10.240.60] | 2018-02-26 09:43:53.800000 | 85.10.240.52 | 1005
READ message received from /85.10.240.52 [MessagingService-Incoming-/85.10.240.52] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 19
Executing single-partition query on test_10 [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 184
Acquiring sstable references [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 278
Merging memtable tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 323
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 373
Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 401
Read 3 live and 0 tombstone cells [SharedPool-Worker-1] | 2018-02-26 09:43:53.802001 | 85.10.240.60 | 608
Enqueuing response to /85.10.240.52 [SharedPool-Worker-1] | 2018-02-26 09:43:53.803000 | 85.10.240.60 | 709
REQUEST_RESPONSE message received from /85.10.240.60 [MessagingService-Incoming-/85.10.240.60] | 2018-02-26 09:43:53.803000 | 85.10.240.52 | 3972
Sending REQUEST_RESPONSE message to /85.10.240.52 [MessagingService-Outgoing-/85.10.240.52] | 2018-02-26 09:43:53.803000 | 85.10.240.60 | 948
Processing response from /85.10.240.60 [SharedPool-Worker-2] | 2018-02-26 09:43:53.804000 | 85.10.240.52 | 4060
Request complete | 2018-02-26 09:43:53.803553 | 85.10.240.52 | 4553
cqlsh:demo>
有关部分是:
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 373
Merging data from memtables and 0 sstables [SharedPool-Worker-1] | 2018-02-26 09:43:53.802000 | 85.10.240.60 | 401
我的小演示案例中不涉及sstable读写 行缓存关闭时,sstable似乎被读取。
当“行缓存”处于启用状态时,按如下方式跟踪:
Key Cache : entries 144, size 103.85 KiB, capacity 100 MiB, 567 hits, 723 requests, 0.784 recent hit rate, 14400 save period in seconds
Row Cache : entries 31, size 2.07 MiB, capacity 10 GiB, 279 hits, 310 requests, 0.900 recent hit rate, 3600 save period in seconds
从测试_10中选择值_1,其中年份=20160102
Tracing session: 71705c70-1afc-11e8-892f-edfe3d6c5915
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 318 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:37.208000 | 10.81.220.6 | 762 | 10.81.220.6
Row cache hit [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 | 3433 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:37.211000 | 10.81.220.6 | 3955 | 10.81.220.6
Request complete | 2018-02-26 08:53:37.223503 | 10.81.220.6 | 15503 | 10.81.220.6
Tracing session: 722ad2d0-1afc-11e8-892f-edfe3d6c5915
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 381 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:53:38.429000 | 10.81.220.6 | 592 | 10.81.220.6
Row cache hit [ReadStage-2] | 2018-02-26 08:53:38.437000 | 10.81.220.6 | 8411 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:53:38.438000 | 10.81.220.6 | 8720 | 10.81.220.6
Request complete | 2018-02-26 08:53:38.449057 | 10.81.220.6 | 20057 | 10.81.220.6
缓存关闭时:
Tracing session: efaad070-1afc-11e8-bafb-efaf5c6d9339
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:57:08.984000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:08.988000 | 10.81.220.6 | 4448 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:08.989000 | 10.81.220.6 | 4730 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 | 9314 | 10.81.220.6
Acquiring sstable references [ReadStage-3] | 2018-02-26 08:57:08.993000 | 10.81.220.6 | 9493 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 | 9727 | 10.81.220.6
Partition index with 2 entries found for sstable 1 [ReadStage-3] | 2018-02-26 08:57:08.994000 | 10.81.220.6 | 10200 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 | 13002 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-3] | 2018-02-26 08:57:08.997000 | 10.81.220.6 | 13265 | 10.81.220.6
Request complete | 2018-02-26 08:57:08.998950 | 10.81.220.6 | 14950 | 10.81.220.6
Tracing session: f0ad9a70-1afc-11e8-bafb-efaf5c6d9339
activity | timestamp | source | source_elapsed | client
------------------------------------------------------------------------------------------+----------------------------+-------------+----------------+-------------
Execute CQL3 query | 2018-02-26 08:57:10.679000 | 10.81.220.6 | 0 | 10.81.220.6
Parsing select value_1 from test_10 where year = 20160102; [Native-Transport-Requests-1] | 2018-02-26 08:57:10.679000 | 10.81.220.6 | 355 | 10.81.220.6
Preparing statement [Native-Transport-Requests-1] | 2018-02-26 08:57:10.680000 | 10.81.220.6 | 752 | 10.81.220.6
Executing single-partition query on test_10 [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3110 | 10.81.220.6
Acquiring sstable references [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3339 | 10.81.220.6
Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones [ReadStage-2] | 2018-02-26 08:57:10.682000 | 10.81.220.6 | 3477 | 10.81.220.6
Key cache hit for sstable 1 [ReadStage-2] | 2018-02-26 08:57:10.683000 | 10.81.220.6 | 3598 | 10.81.220.6
Merged data from memtables and 1 sstables [ReadStage-2] | 2018-02-26 08:57:10.684000 | 10.81.220.6 | 5438 | 10.81.220.6
Read 100 live and 0 tombstone cells [ReadStage-2] | 2018-02-26 08:57:10.685000 | 10.81.220.6 | 5593 | 10.81.220.6
Request complete | 2018-02-26 08:57:10.685204 | 10.81.220.6 | 6204 | 10.81.220.6