为什么Cassandra不从CSV文件加载所有值?
我刚刚开始使用Cassandra(单节点设置,2.0.9版本)。我试图从CSV文件将数据加载到为什么Cassandra不从CSV文件加载所有值?,cassandra,cassandra-2.0,cqlsh,Cassandra,Cassandra 2.0,Cqlsh,我刚刚开始使用Cassandra(单节点设置,2.0.9版本)。我试图从CSV文件将数据加载到COLUMNFAMILY,但注意到它实际上只加载了239595行中的2行。我不明白为什么。如果有任何提示,我将不胜感激 cqlsh控制台输出: cqlsh:keyspace_test1> DESCRIBE COLUMNFAMILY keyspace_test1.invoices; CREATE TABLE invoices ( date text, customer_id text,
COLUMNFAMILY
,但注意到它实际上只加载了239595行中的2行。我不明白为什么。如果有任何提示,我将不胜感激
cqlsh控制台输出:
cqlsh:keyspace_test1> DESCRIBE COLUMNFAMILY keyspace_test1.invoices;
CREATE TABLE invoices (
date text,
customer_id text,
product_id text,
quantity int,
sales float,
PRIMARY KEY ((date))
) WITH
bloom_filter_fp_chance=0.010000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.100000 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
从CSV加载数据
cqlsh:keyspace_test1> COPY invoices (date, product_id, customer_id, quantity, sales) FROM '/home/martakarass/Desktop/invoices.csv';
239595 rows imported in 1 minute and 52.766 seconds.
请注意,SELECT
仅显示2行
cqlsh:keyspace_test1> SELECT * FROM invoices limit 10;
date | customer_id | product_id | quantity | sales
----------+-------------+---------------+----------+--------
2/1/2015 | Client_100 | Product_15702 | 6 | 123.42
1/9/2015 | Client_998 | Product_43550 | 3000 | 15.368
(2 rows)
与count一起检查是否未加载所有行
cqlsh:keyspace_test1> SELECT count(*) FROM invoices;
count
-------
2
(1 rows)
cqlsh:keyspace_test1>
(更新)表格详细信息:
cqlsh:keyspace_test1> DESCRIBE COLUMNFAMILY keyspace_test1.invoices;
CREATE TABLE invoices (
date text,
customer_id text,
product_id text,
quantity int,
sales float,
PRIMARY KEY ((date))
) WITH
bloom_filter_fp_chance=0.010000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.100000 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
我将大胆猜测,日期在invoices.csv文件中不是唯一的。创建类似表时:
CREATE TABLE stackoverflow.invoices (
date timestamp PRIMARY KEY,
amount bigint,
id bigint
)
我使用的CSV文件有4行,如下所示:
date|id|amount
2015-03-30 00:00:00-0500|1|4500
2015-03-31 00:00:00-0500|2|5500
2015-03-31 00:00:00-0500|3|6600
2015-03-31 00:00:00-0500|4|7500
接下来,我使用从复制导入它们:
aploetz@cqlsh:stackoverflow> COPY invoices (date, id, amount) FROM
'/home/aploetz/invoices.csv' WITH DELIMITER='|' AND HEADER=true;
4 rows imported in 0.035 seconds.
我应该有4排,对吗?错
aploetz@cqlsh:stackoverflow> SELECT * FROm invoices;
date | amount | id
--------------------------+--------+----
2015-03-30 00:00:00-0500 | 4500 | 1
2015-03-31 00:00:00-0500 | 7500 | 4
(2 rows)
卡桑德拉主键是唯一的。因此,如果要从一个文件中导入239595行,但实际上只有两个唯一的日期,那么只有两行了。这是一个bug,没有解决
您可以通过以下方式解决问题:
首先进入根目录,您将获得。cassandra文件夹进入此文件夹,您将获得
cqlshrc
把这两行加起来
[csv]
field_size_limit=1000000000
然后关闭您的文件。重新启动cassandra,您的问题就会得到解决。还可以共享表的详细信息,描述COLUMNFAMILY keyspace\u test1.1;我用descripe COLUMNFAMILY keyspace\u test1.0的输出更新了我的注释代码>(在从CSV文件加载了这两行之后)。您可以发布top invoices.CSV的输出吗?
?是的,您是对的!!感谢您分享您的经验!:)没问题@Marciszka,很高兴我能帮上忙。