Ssis 如何清理SSIDB？
当我设置这个时，我忽略了保留期。我的数据库变得相当大，所以我想减小它的大小。如果我只是更改保留期，它是365，则会导致运行我的包的SSI出现问题。我甚至以较小的增量对其进行了更改，但删除语句将创建锁，从而阻止新作业运行Ssis 如何清理SSIDB？,ssis,sql-server-2012,Ssis,Sql Server 2012,当我设置这个时，我忽略了保留期。我的数据库变得相当大，所以我想减小它的大小。如果我只是更改保留期，它是365，则会导致运行我的包的SSI出现问题。我甚至以较小的增量对其进行了更改，但删除语句将创建锁，从而阻止新作业运行 有没有办法解决这个问题？我曾考虑过创建一个新的SSIDB。菲尔·布拉默（Phil Brammer）遇到了这个问题，以及与SSIS目录的维护和提供相关的许多其他问题，他在自己的文章中介绍了这些问题 根本问题 根本问题是MS试图在设计SSI时考虑到RI，但它们很懒惰，允许级联删除而不
有没有办法解决这个问题？我曾考虑过创建一个新的SSIDB。菲尔·布拉默（Phil Brammer）遇到了这个问题，以及与SSIS目录的维护和提供相关的许多其他问题，他在自己的文章中介绍了这些问题 根本问题 根本问题是MS试图在设计SSI时考虑到RI，但它们很懒惰，允许级联删除而不是显式处理它们 新的SSIS 2012目录数据库SSIDB采用了一些基本的索引，并将引用完整性设置为在大多数表之间进行级联删除 输入SQL代理作业“SSIS服务器维护作业”。默认情况下，此作业设置为每天午夜运行，并使用两个目录参数发挥作用：“定期清理日志”和“保留期天数”。设置这两个参数后，维护作业将清除所注保留期以外的所有数据 此维护作业从internal.operations中一次删除10条记录，然后级联到下游的许多表中。在我们的案例中，我们每天要删除大约3000条操作记录，每次删除10条！这将从internal.operation_消息转换为160万行。那只是一张下游的桌子！整个过程完全地、彻底地将SSISDB数据库从任何SELECT/INSERT数据中锁定 决议 在MS改变工作方式之前，支持的选项是 将维护作业计划移动到更适合您的环境的时间 我知道，在我当前的客户机上，我们只在凌晨加载数据，因此SSIDB在工作时间内保持安静 如果在一段安静的时间内运行维护作业不是一个选项，那么您需要设计自己的delete语句，以尽量减少级联删除 在我目前的客户那里，在过去的10个月里，我们每晚运行大约200个软件包，而且已经有365天的历史了。从数量级上看，我们最大的表是
Schema Table RowCount
internal event_message_context 1,869,028
internal operation_messages 1,500,811
internal event_messages 1,500,803
SSIS DB可能会有不同的表名，但最后会有扩展。它可以是internal.event\u message\u context，而不是internal.event\u context\u scaleout。它可以是internal.operations\u messages\u scaleout，而不是internal.operations\u messages。只需相应地修改代码中的表名，它就会正常运行 如果您在Azure Data Factory中使用SSIS IR，则肯定会出现这种情况。您会发现普通表仍然存在，但为空，*\u scaleout版本包含所有数据 工具书类
我创建了一个类似的存储过程来进行下面的归档。如果有任何错误，请告诉我。我不提供任何保证，但对我来说效果很好。这段代码绝对不是经过润色的，但我想与大家分享一下，即使用临时表意味着一次只能运行一次，也许一个会话范围的表会更好 我遇到了一个问题，MS自己的clean过程会将LDF文件吹出并长时间锁定表，一旦导致服务器空间不足。我决定写我自己的，先删除较大的表，然后删除操作表。下面的过程在日志空间中使用的空间永远不会超过1GB，并且不会在很长时间内锁定表，这是SSIS作业必须全天运行时遇到的难题 首先，它记录到一个表中
CREATE TABLE [dbo].[ETL_SSIS_Operations_Archived](
[id] [int] IDENTITY(1,1) NOT NULL,
[operation_id_str] [varchar](900) NOT NULL,
[event_messages_context] [int] NULL,
[event_messages] [int] NULL,
[operation_messages] [int] NULL,
[num_operators] [int] NULL,
[chunksize] [int] NULL,
[DateStarted] [datetime] NOT NULL,
[DateFinished] [datetime] NULL,
[executionSecs] [int] NULL,
[DelOperationsDateStarted] [datetime] NULL,
[DelOperationsDateFinished] [datetime] NULL,
[DelOperationsExecutionSecs] [int] NULL
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[tmp_etl_operations_id](
[operation_id] [int] NULL,
[dateCreated] [datetime] NULL default getdate()
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[tmp_etl_operations_id] ADD DEFAULT (getdate()) FOR [dateCreated]
GO
CREATE PROCEDURE [dbo].[sp_Archive_SSIDB_Catalogue]
AS
BEGIN
DECLARE @MyCursor as CURSOR;
DECLARE @l_operation_id int;
declare @l_rows_del int = 1
declare @l_operation_id_str varchar(8000) = ''
declare @l_id int
declare @l_event_message_context int = 0
declare @l_event_messages int = 0
declare @l_operation_messages int = 0
declare @l_loop_num int = 1
declare @C_BULK_NUM int = 100
declare @C_CHUNK_SIZE int = 100000
declare @l_last_rec char(1)
SET @MyCursor = CURSOR FOR
with params as
(
-- i round up the midnight that day, just so i know once it is done for the day it is done
-- and if the real maintenance job was to run after this (just for the sake of it to double ensure nothing has been missed), but not actually need to do
-- anything as its already done in here, no new operations would have snuck in due to the sliding system time
SELECT cast(dateadd(day,1,GETDATE() - CONVERT(int,property_value)) as date) ArchiveDate
FROM ssisdb.[catalog].[catalog_properties]
WHERE property_name = 'RETENTION_WINDOW'
)
select operation_id,iif(r=c,'Y','N') lastrec
from
(
select operation_id,row_number() over (partition by null order by operation_id) r,count(*) over (partition by null) c
FROM ssisdb.[internal].[operations]
WHERE ( [end_time] <= (select ArchiveDate from params)
-- A special case when END_TIME is null, we will delete the records based on the created time
OR ([end_time] IS NULL AND [status] = 1 AND [created_time] <= (select ArchiveDate from params) ))
) x
order by operation_id
OPEN @MyCursor;
FETCH NEXT FROM @MyCursor INTO @l_operation_id,@l_last_rec
WHILE @@FETCH_STATUS = 0
BEGIN
set @l_operation_id_str = @l_operation_id_str+','+cast(@l_operation_id as varchar(100))
if @l_loop_num = 1
begin
delete from tmp_etl_operations_id
set @l_operation_id_str = cast(@l_operation_id as varchar(100))
end
insert into tmp_etl_operations_id (operation_id) select @l_operation_id
if @l_loop_num = @C_BULK_NUM or @l_last_rec='Y'
begin
set @l_loop_num = 1
set @l_event_message_context = 0
set @l_event_messages = 0
set @l_operation_messages = 0
insert into ETL_SSIS_Operations_Archived ([operation_id_str], num_operators,chunksize, event_messages_context, event_messages, operation_messages, datestarted)
select @l_operation_id_str, @C_BULK_NUM,@C_CHUNK_SIZE,@l_event_message_context,@l_event_messages,@l_operation_messages,getdate()
--where 0 = (select count(*) from ETL_SSIS_Operations_Archived where operation_id=@l_operation_id_str)
set @l_id = Scope_Identity()
set @l_rows_del = @C_CHUNK_SIZE
while (@l_rows_del >= @C_CHUNK_SIZE)
begin
delete top (@C_CHUNK_SIZE)
from ssisdb.internal.event_message_context
where operation_id in (select operation_id from etl..tmp_etl_operations_id)
set @l_rows_del = @@ROWCOUNT
set @l_event_message_context = @l_event_message_context+@l_rows_del
update ETL_SSIS_Operations_Archived
set event_messages_context = event_messages_context+@l_rows_del
where id = @l_id--operation_id = @l_operation_id_str
end
set @l_rows_del = @C_CHUNK_SIZE
while (@l_rows_del >= @C_CHUNK_SIZE)
begin
delete top (@C_CHUNK_SIZE)
from ssisdb.internal.event_messages
where operation_id in (select operation_id from tmp_etl_operations_id)
set @l_rows_del = @@ROWCOUNT
set @l_event_messages = @l_event_messages+@l_rows_del
update ETL_SSIS_Operations_Archived
set event_messages = event_messages+@l_rows_del
where id = @l_id--operation_id = @l_operation_id_strwhere operation_id = @l_operation_id_str
end
set @l_rows_del = @C_CHUNK_SIZE
while (@l_rows_del >= @C_CHUNK_SIZE)
begin
delete top (@C_CHUNK_SIZE)
from ssisdb.internal.operation_messages
where operation_id in (select operation_id from tmp_etl_operations_id)
set @l_rows_del = @@ROWCOUNT
set @l_operation_messages = @l_operation_messages+@l_rows_del
update ETL_SSIS_Operations_Archived
set operation_messages = operation_messages+@l_rows_del
where id = @l_id--operation_id = @l_operation_id_strwhere operation_id = @l_operation_id_str --
end
update ETL_SSIS_Operations_Archived
set DateFinished = getdate()
,executionSecs = Datediff(s, DateStarted, getdate())
,DelOperationsDateStarted = getdate()
where id = @l_id--operation_id = @l_operation_id_strwhere operation_id = @l_operation_id_str --
-- lets delete the operations now
delete --top (@C_CHUNK_SIZE)
from ssisdb.internal.operations
where operation_id in (select operation_id from tmp_etl_operations_id)
update ETL_SSIS_Operations_Archived
set DelOperationsDateFinished = getdate()
,DelOperationsExecutionSecs = Datediff(s, DelOperationsDateStarted, getdate())
where id = @l_id--operation_id = @l_operation_id_strwhere operation_id = @l_operation_id_str --
end
else
begin
set @l_loop_num = @l_loop_num+1
end
FETCH NEXT FROM @MyCursor INTO @l_operation_id,@l_last_rec
END
CLOSE @MyCursor;
DEALLOCATE @MyCursor;
END
已从本文中找到有用的清理脚本：
还考虑将数据库切换到简单恢复模式，但为日志文件留出足够的空间。
< P>运行这个SPROC：EXEC [internal].[cleanup_server_retention_window]
这是非常有帮助的。我能把数据库吹走，从头开始吗？我失去了历史，但还有其他的失败吗？只要你有你的.ispac
handy+环境内容handy或不介意重新创建，drop and recreate将可能更新工作，包括@MartinSmith Done和thank you。顺便说一句，由于我必须清理一个2012年的SSISDB，它将床上的东西清理干净，所以SSISDB很可能有不同的表名，并且在现在结束时会有扩展。它可以是internal.event\u message\u context，而不是internal.event\u context\u scaleout。它可以是internal.operations\u messages\u scaleout，而不是internal.operations\u messages。只需相应地修改代码中的表名，它就会正常运行。
USE SSISDB
GO
DELETE FROM [internal].[executions]
GO
DELETE FROM [internal].[executable_statistics]
GO
DELETE FROM [internal].[execution_component_phases]
GO
DELETE FROM [internal].[execution_data_statistics]
GO
DELETE FROM [internal].[execution_data_taps]
GO
DELETE FROM [internal].[execution_parameter_values]
GO
DELETE FROM [internal].[execution_property_override_values]
GO
DELETE FROM [internal].[extended_operation_info]
GO
DELETE FROM [internal].[operation_messages]
GO
DELETE FROM [internal].[event_messages]
GO
DELETE FROM [internal].[event_message_context]
GO
DELETE FROM [internal].[operation_os_sys_info]
GO
DELETE FROM [internal].[operation_permissions]
GO
DELETE FROM [internal].[validations]
GO
EXEC [internal].[cleanup_server_retention_window]