如何恢复Cassandra快照?
我正在为Cassandra数据库构建一个备份和恢复过程,以便在需要时随时准备就绪,并了解详细信息,以便构建适合生产的产品。我遵循税务局的指示: 首先,我将在一个dev box上植入数据库,然后尝试使备份/恢复工作。以下是备份脚本:如何恢复Cassandra快照?,cassandra,datastax,Cassandra,Datastax,我正在为Cassandra数据库构建一个备份和恢复过程,以便在需要时随时准备就绪,并了解详细信息,以便构建适合生产的产品。我遵循税务局的指示: 首先,我将在一个dev box上植入数据库,然后尝试使备份/恢复工作。以下是备份脚本: #!/bin/bash cd /opt/apache-cassandra-2.0.9 ./bin/nodetool clearsnapshot -t after_seeding makeyourcase ./bin/nodetool snapshot -t aft
#!/bin/bash
cd /opt/apache-cassandra-2.0.9
./bin/nodetool clearsnapshot -t after_seeding makeyourcase
./bin/nodetool snapshot -t after_seeding makeyourcase
cd /var/lib/
tar czf after_seeding.tgz cassandra/data/makeyourcase/*/snapshots/after_seeding
是的,焦油也许不是最有效的方法,但我现在只是想让它起作用。我查过焦油,所有的文件都在那里
备份数据库后,我关闭Cassandra和我的应用程序,然后rm-rf/var/lib/Cassandra/
模拟完全丢失
现在恢复数据库。从中恢复“方法2”比方法1更兼容我的架构创建组件
因此,方法2/步骤1,“重新创建模式”:重启Cassandra,然后重启我的应用程序。该应用程序用于在必要时在启动时重新创建架构。一旦启动,就会有一个工作的Cassandra节点,其中包含应用程序的模式,但没有数据
方法2/步骤2“恢复快照”:他们给出了三种选择,第一种是使用sstableloader,记录在。加载程序所需的文件夹结构与快照工具创建的文件夹结构完全不同,因此所有内容都必须移动到位。在遇到这些麻烦之前,我先在一张桌子上试试:
>./bin/sstableloader makeyourcase/users
Error: Could not find or load main class org.apache.cassandra.tools.BulkLoader
嗯,那是行不通的。BulkLoader位于./lib/apache-cassandra-2.0.9.jar中,但似乎没有对其进行开箱即用的设置。与其调试该工具,不如转到第二个备选方案,将快照目录复制到makeyourcase/users/snapshots/目录中。这应该很容易,因为我们正在将快照目录扔回它的来源,所以tar xzf after_seeding.tgz
应该做到以下几点:
cd /var/lib/
tar xzf after_seeding.tgz
chmod -R u+rwx cassandra/data/makeyourcase
这会将快照目录放回各自的“快照”目录下,刷新会恢复数据:
cd /opt/apache-cassandra-2.0.9
./bin/nodetool refresh -- makeyourcase users
这是毫无怨言的。请注意,您必须为每个表运行此操作,因此必须首先生成表列表。但是,在我们这样做之前,请注意卡桑德拉日志中有一些有趣的东西:
INFO 14:32:26,319 Loading new SSTables for makeyourcase/users...
INFO 14:32:26,326 No new SSTables were found for makeyourcase/users
我们把快照放回去了,但是卡桑德拉没有找到。我还尝试将快照目录移动到现有SSTables目录下,并将旧的SSTable文件复制到现有目录中,但日志中出现了相同的错误。卡桑德拉没有记录它希望在哪里找到它们,只是它找不到它们。文档说要将它们放入名为data/keyspace/table_name-UUID的目录中,但没有这样的目录。有一个名为data/makeyourcase/users/snapshots/1408820504987 users/,但将snapshot dir或单个文件放在那里不起作用
第三种选择,“节点重启方法”看起来不适合多节点生产环境,所以我没有尝试过
编辑:
为了让下一个人完全明白这一点,下面是应用公认答案的初步、有效的备份和恢复脚本
myc_backup.sh:
#!/bin/bash
cd ~/bootstrap/apache-cassandra-2.0.9
./bin/nodetool clearsnapshot -t after_seeding makeyourcase
./bin/nodetool snapshot -t after_seeding makeyourcase
cd /var/lib/
tar czf after_seeding.tgz cassandra/data/makeyourcase/*/snapshots/after_seeding
myc_restore.sh:
#!/bin/bash
cd /var/lib/
tar xzf after_seeding.tgz
chmod -R u+rwx cassandra/data/makeyourcase
cd ~/bootstrap/apache-cassandra-2.0.9
TABLE_LIST=`./bin/nodetool cfstats makeyourcase | grep "Table: " | sed -e 's+^.*: ++'`
for TABLE in $TABLE_LIST; do
echo "Restore table ${TABLE}"
cd /var/lib/cassandra/data/makeyourcase/${TABLE}
if [ -d "snapshots/after_seeding" ]; then
cp snapshots/after_seeding/* .
cd ~/bootstrap/apache-cassandra-2.0.9
./bin/nodetool refresh -- makeyourcase ${TABLE}
cd /var/lib/cassandra/data/makeyourcase/${TABLE}
rm -rf snapshots/after_seeding
echo " Table ${TABLE} restored."
else
echo " >>> Nothing to restore."
fi
done
添加了更多详细信息:
您可以使用以下方法为特定密钥空间运行快照:
$ nodetool snapshot <mykeyspace> -t <SnapshotDirectoryName>
准备好恢复后,将快照数据复制回keyspace/table目录(每个表一个):
$pwd
/var/lib/cassandra/data/mykeyspace/mytable
$sudo cp快照//*。
你提到:
这会将快照目录放回各自的“快照”目录下,刷新>会还原数据:
cd /opt/apache-cassandra-2.0.9
./bin/nodetool refresh -- makeyourcase users
我认为问题在于您正在将快照数据恢复到快照目录中。它应该在表目录中。其他一切似乎都是对的,让我知道
文档说把它们放到一个名为
data/keyspace/table_name-UUID,但没有这样的目录
您没有这个UUID目录,因为您使用的是cassandra 2.0,而这个UUID是从cassandra 2.2开始的。步骤1:我使用下面的命令创建了一个表
CREATE TABLE Cricket (
PlayerID uuid,
LastName varchar,
FirstName varchar,
City varchar,
State varchar,
PRIMARY KEY (PlayerID));
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Pendulkar', 'Sachin', 'Mumbai','Maharastra');
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Vholi', 'Virat', 'Delhi','New Delhi');
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Sharma', 'Rohit', 'Berhampur','Odisha');
cassandra@node1:~$ sstableloader -d 10.213.61.21 -username cassandra --password cassandra /home/cassandra/data/students_details/cricket-d3576f60960f11ea947b39646348bb4f/snapshots
Established connection to initial hosts
Opening sstables and calculating sections to stream
Summary statistics:
Connections per host : 1
Total files transferred : 0
Total bytes transferred : 0.000KiB
Total duration : 2920 ms
Average transfer rate : 0.000KiB/s
Peak transfer rate : 0.000KiB/s
步骤2:使用下面的命令插入3条记录
CREATE TABLE Cricket (
PlayerID uuid,
LastName varchar,
FirstName varchar,
City varchar,
State varchar,
PRIMARY KEY (PlayerID));
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Pendulkar', 'Sachin', 'Mumbai','Maharastra');
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Vholi', 'Virat', 'Delhi','New Delhi');
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Sharma', 'Rohit', 'Berhampur','Odisha');
cassandra@node1:~$ sstableloader -d 10.213.61.21 -username cassandra --password cassandra /home/cassandra/data/students_details/cricket-d3576f60960f11ea947b39646348bb4f/snapshots
Established connection to initial hosts
Opening sstables and calculating sections to stream
Summary statistics:
Connections per host : 1
Total files transferred : 0
Total bytes transferred : 0.000KiB
Total duration : 2920 ms
Average transfer rate : 0.000KiB/s
Peak transfer rate : 0.000KiB/s
第三步:我不小心删除了板球桌
drop table Cricket;
步骤4:需要使用自动快照备份恢复该表
注意:auto_snapshot(默认值:true)启用或禁用在键空间截断或删除表之前是否对数据进行快照。为防止数据丢失,强烈建议使用默认设置
步骤5:查找快照位置和文件
cassandra@node1:~/data/students_details$ cd cricket-88128dc0960d11ea947b39646348bb4f
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f$ ls -lrth
total 0
drwxrwxr-x 2 cassandra cassandra 6 May 14 18:05 backups
drwxrwxr-x 3 cassandra cassandra 43 May 14 18:06 snapshots
步骤6:您将在快照位置获得一个.cql文件,其中包含DDL表
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f/snapshots/dropped-1589479603749-cricket$ ls -lrth
total 44K
-rw-rw-r-- 1 cassandra cassandra 92 May 14 18:06 md-1-big-Summary.db
-rw-rw-r-- 1 cassandra cassandra 61 May 14 18:06 md-1-big-Index.db
-rw-rw-r-- 1 cassandra cassandra 16 May 14 18:06 md-1-big-Filter.db
-rw-rw-r-- 1 cassandra cassandra 179 May 14 18:06 md-1-big-Data.db
-rw-rw-r-- 1 cassandra cassandra 92 May 14 18:06 md-1-big-TOC.txt
-rw-rw-r-- 1 cassandra cassandra 4.7K May 14 18:06 md-1-big-Statistics.db
-rw-rw-r-- 1 cassandra cassandra 9 May 14 18:06 md-1-big-Digest.crc32
-rw-rw-r-- 1 cassandra cassandra 43 May 14 18:06 md-1-big-CompressionInfo.db
-rw-rw-r-- 1 cassandra cassandra 891 May 14 18:06 schema.cql
-rw-rw-r-- 1 cassandra cassandra 31 May 14 18:06 manifest.json
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f/snapshots/dropped-1589479603749-cricket$
more schema.cql
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f/snapshots/dropped-1589479603749-cricket$ more schema.cql
CREATE TABLE IF NOT EXISTS students_details.cricket (
playerid uuid PRIMARY KEY,
city text,
firstname text,
lastname text,
state text)
WITH ID = 88128dc0-960d-11ea-947b-39646348bb4f
AND bloom_filter_fp_chance = 0.01
AND dclocal_read_repair_chance = 0.1
AND crc_check_chance = 1.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND min_index_interval = 128
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE'
AND comment = ''
AND caching = { 'keys': 'ALL', 'rows_per_partition': 'NONE' }
AND compaction = { 'max_threshold': '32', 'min_threshold': '4', 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' }
AND compression = { 'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor' }
AND cdc = false
AND extensions = { };
步骤7:登录数据库并使用该DDL创建表
apiadmin@cqlsh:coopersdev> use students_details;
apiadmin@cqlsh:students_details> CREATE TABLE IF NOT EXISTS students_details.cricket (
... playerid uuid PRIMARY KEY,
... city text,
... firstname text,
... lastname text,
... state text)
... WITH ID = 88128dc0-960d-11ea-947b-39646348bb4f
... AND bloom_filter_fp_chance = 0.01
... AND dclocal_read_repair_chance = 0.1
... AND crc_check_chance = 1.0
... AND default_time_to_live = 0
... AND gc_grace_seconds = 864000
... AND min_index_interval = 128
... AND max_index_interval = 2048
... AND memtable_flush_period_in_ms = 0
... AND read_repair_chance = 0.0
... AND speculative_retry = '99PERCENTILE'
... AND comment = ''
... AND caching = { 'keys': 'ALL', 'rows_per_partition': 'NONE' }
... AND compaction = { 'max_threshold': '32', 'min_threshold': '4', 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' }
... AND compression = { 'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor' }
... AND cdc = false
... AND extensions = { };
apiadmin@cqlsh:students_details>
步骤8:将快照文件夹中的所有文件复制到现有的cricket table文件夹
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f/snapshots/dropped-1589479603749-cricket$ pwd
/home/cassandra/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f/snapshots/dropped-1589479603749-cricket
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f/snapshots/dropped-1589479603749-cricket$ cp * /home/cassandra/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f/snapshots/dropped-1589479603749-cricket$ cd /home/cassandra/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f$ ls -lrth
total 44K
drwxrwxr-x 2 cassandra cassandra 6 May 14 18:05 backups
drwxrwxr-x 3 cassandra cassandra 43 May 14 18:06 snapshots
-rw-rw-r-- 1 cassandra cassandra 891 May 14 18:11 schema.cql
-rw-rw-r-- 1 cassandra cassandra 92 May 14 18:11 md-1-big-TOC.txt
-rw-rw-r-- 1 cassandra cassandra 92 May 14 18:11 md-1-big-Summary.db
-rw-rw-r-- 1 cassandra cassandra 4.7K May 14 18:11 md-1-big-Statistics.db
-rw-rw-r-- 1 cassandra cassandra 61 May 14 18:11 md-1-big-Index.db
-rw-rw-r-- 1 cassandra cassandra 16 May 14 18:11 md-1-big-Filter.db
-rw-rw-r-- 1 cassandra cassandra 9 May 14 18:11 md-1-big-Digest.crc32
-rw-rw-r-- 1 cassandra cassandra 179 May 14 18:11 md-1-big-Data.db
-rw-rw-r-- 1 cassandra cassandra 43 May 14 18:11 md-1-big-CompressionInfo.db
-rw-rw-r-- 1 cassandra cassandra 31 May 14 18:11 manifest.json
cassandra@node1:~/data/students_details/cricket-88128dc0960d11ea947b39646348bb4f$
步骤9:使用下面的命令,使用sstableloader开始还原表数据
CREATE TABLE Cricket (
PlayerID uuid,
LastName varchar,
FirstName varchar,
City varchar,
State varchar,
PRIMARY KEY (PlayerID));
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Pendulkar', 'Sachin', 'Mumbai','Maharastra');
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Vholi', 'Virat', 'Delhi','New Delhi');
INSERT INTO Cricket (PlayerID, LastName, FirstName, City, State)
VALUES (now(), 'Sharma', 'Rohit', 'Berhampur','Odisha');
cassandra@node1:~$ sstableloader -d 10.213.61.21 -username cassandra --password cassandra /home/cassandra/data/students_details/cricket-d3576f60960f11ea947b39646348bb4f/snapshots
Established connection to initial hosts
Opening sstables and calculating sections to stream
Summary statistics:
Connections per host : 1
Total files transferred : 0
Total bytes transferred : 0.000KiB
Total duration : 2920 ms
Average transfer rate : 0.000KiB/s
Peak transfer rate : 0.000KiB/s
步骤10:表已成功还原。请验证
playerid | city | firstname | lastname | state
--------------------------------------+-----------+-----------+-----------+------------
d7b12c90-960f-11ea-947b-39646348bb4f | Berhampur | Rohit | Sharma | Odisha
d7594890-960f-11ea-947b-39646348bb4f | Delhi | Virat | Vholi | New Delhi
d7588540-960f-11ea-947b-39646348bb4f | Mumbai | Sachin | Pendulkar | Maharastra
你是说*.db和其他文件应该直接放在/var/lib/cassandra/data/中,而不是放在/var/lib/cassandra/data/makeyourcase/下?而不是在/var/lib/cassandra/data/after_seaking?编辑对Clarify的响应时,您会说“我认为问题在于您正在将快照数据恢复到快照目录中。它应该在表目录中。”这是我尝试的目录之一。Cassandra仍然没有找到它们。“您可以使用:…”运行特定键空间的快照。“快照部分工作正常,如图所示。”删除数据时,请确保不删除快照文件夹,否则将无法还原它(除非您将其移到另一个位置/计算机)。”,卸载时,将重新创建快照目录。