Sql server 使用SSIS从多级XML到单表的数据迁移
目标 我正在尝试将数据从具有嵌套元素的多层XML文件迁移到单个表中 系统参数Sql server 使用SSIS从多级XML到单表的数据迁移,sql-server,xml,ssis,database-migration,Sql Server,Xml,Ssis,Database Migration,目标 我正在尝试将数据从具有嵌套元素的多层XML文件迁移到单个表中 系统参数 MS SQL Server管理工作室 Microsoft Visual Studio SSIS XML文件 正如您所看到的，它不仅仅是一个简单的布局。整件东西都用一个“人”的标签包着，大约有1000人。每个“Person”标签包含以下信息元素。XML如下所示： 人 名字 姓氏 传记 专业知识 形象 链接 书 书 年 细节 书 年 细节 。。。（其中可能有很多） 文章 文章 年 细节 文章 年
- MS SQL Server管理工作室
- Microsoft Visual Studio SSIS
- 名字
- 姓氏
- 传记
- 专业知识
- 形象
- 链接
- 书
- 书
- 年
- 细节
- 书
- 年
- 细节
- 。。。（其中可能有很多）
- 书
- 文章
- 文章
- 年
- 细节
- 文章
- 年
- 细节
- 。。。（其中可能有很多）
- 文章
- 文件
- 艺术品
- 网站
- 约翰
- 斯坦贝克
- 书
- 伊甸以东
- 1952年
- 一本好书
- 人与鼠的关系
- 1937年
- 关于老鼠的书
- 愤怒的葡萄
- 1939年
- 关于愤怒的葡萄的书
- 伊甸以东
- 文章
- 文章
- XML文件中的某些条目最多包含60000个字符。这意味着我应该使用什么数据类型
<?xml version="1.0" encoding="UTF-8" ?>
<People>
<Person>
<FirstName>Eliza</FirstName>
<LastName>Ablovatski</LastName>
<Biography>
<![CDATA[<p>Eliza Ablovatski joined the Kenyon history department in 2003, after graduate work in East Central European history at Columbia University and research and fellowships in Munich and Berlin, Germany and Budapest, Hungary. She teaches classes on Europe from 1500 to the present, focusing on the nineteenth and twentieth centuries, Germany, Russia, the Habsburg Monarchy, film, nationalism and identity, gender, race, and the interwar period.</p>
<p>Her dissertation and first book, <em>Revolution and Political Violence in Central Europe: The Deluge of 1919</em> (forthcoming from Cambridge University Press), focus on the revolutionary upheavals in Munich and Budapest following the First World War, and their relationship to political violence and antisemitism. She is currently researching the occupation of Austria (1945-1955) at the end of the Second World War, and the nuclear idea in postwar Europe. She has also researched and written extensively on the history of Jews in the former Habsburg regional capital of Czernowitz (now Ukraine).</p>]]>
</Biography>
<Expertise>
<![CDATA[<p>Modern Europe, especially Germany and Central/East Central Europe in the nineteenth and twentieth centuries; European Jewish and women's history, East European and German film and literature, socialism, war, and revolution.</p>]]>
</Expertise>
<Image>http://www.kenyon.edu/images/directory/ablovatski.jpg</Image>
<Link>http://www.kenyon.edu/directories/campus-directory/biography/eliza-ablovatski/</Link>
<Books>
<Book>
<Year></Year>
<Details>
<![CDATA[<p><em>Zwischen Pruth und Jordan. Lebenserinnerungen Czernowitzer Juden</em><em> , </em>with Gaby Coldewey and others Köln: Böhlau Verlag, 2003</p>]]>
</Details>
</Book>
<Book>
<Year></Year>
<Details>
<![CDATA[<p><em>Czernowitz ist gewen an alt jiddische Stdt: Überlebende berichten,</em> With Gaby Coldewey and others. First Edition: Czernowitz,Ukraine: distributed by the Heinrich-Böll-Stiftung, 1998 Second Edition: Berlin, 1999 (Third edition: Potsdam, forthcoming 2009)</p>]]>
</Details>
</Book>
</Books>
<Articles>
<Article>
<Year></Year>
<Details>
<![CDATA[<p>"The Central European Revolutions of 1919 and the Myth of Judeo-Bolshevism," <em>European Review of History, Vol. 17/ Issue 3: Cosmopolitanism, Nationalism and the Jews of East Central Europe (2010), 473-489.</em></p>]]>
</Details>
</Article>
<Article>
<Year></Year>
<Details>
<![CDATA[<p>"Between Red Army and White Guard: Women in Budapest, 1918-1919," in <em>Gender and War in Twentieth-Century Eastern Europe,</em> edited by Maria Bucur and Nancy Wingfield Bloomington: Indiana University Press 2006</p>]]>
</Details>
</Article>
<Article>
<Year></Year>
<Details>
<![CDATA[<p>"The Girl with the Titus-head: Women in Revolution in Munich and Budapest, 1919" <em>Nationalities Papers </em>28/3 (September 2000), 541-550</p>]]>
</Details>
</Article>
</Articles>
<Papers>
</Papers>
<Artwork>
</Artwork>
<Websites>
</Websites>
</Person>
...This goes on to include many <Person> elements. (About 1000)
</People>
伊丽莎
阿布洛瓦茨基
Eliza Ablovatski于2003年加入肯扬历史系，此前曾在哥伦比亚大学（Columbia University）攻读中东欧历史，并在慕尼黑、柏林、德国和匈牙利布达佩斯获得研究和奖学金。她教授1500年至今的欧洲课程，重点是十九世纪和二十世纪、德国、俄罗斯、哈布斯堡君主制、电影、民族主义和身份、性别、种族和两次大战之间的时期
她的论文和第一本书《中欧的革命和政治暴力：1919年的洪水》（即将由剑桥大学出版社出版），重点关注第一次世界大战后慕尼黑和布达佩斯的革命剧变，以及它们与政治暴力和反犹主义的关系。她目前正在研究二战结束时对奥地利的占领（1945-1955），以及战后欧洲的核理念。她还对前哈布斯堡地区首府切尔诺维茨（现乌克兰）的犹太人历史进行了广泛的研究和写作。]>
现代欧洲，特别是十九世纪和二十世纪的德国和中欧/东欧；欧洲犹太人和妇女历史、东欧和德国电影和文学、社会主义、战争和革命。]>
http://www.kenyon.edu/images/directory/ablovatski.jpg
http://www.kenyon.edu/directories/campus-directory/biography/eliza-ablovatski/
茨维辛·普鲁思和约旦。Lebenserinerungen Czernowizer Juden，与加比·科尔德韦和其他Kö；ln:Bö；赫劳·维拉格，2003年]>
捷尔诺维茨是一位伟大的政治家；berlebende berichten和Gaby Coldewey等人。第一版：切尔诺维茨，乌克兰：由Heinrich-Bö发行；ll Stiftung，1998年第二版：柏林，1999年（第三版：波茨坦，即将于2009年出版）]>
“1919年中欧革命和犹太布尔什维克主义的神话”，《欧洲历史评论》，第17卷/第3期：世界主义、民族主义和中东欧犹太人（2010年），473-489。]>
“红军和白衣卫之间：布达佩斯的妇女，1918-1919年”，《二十世纪东欧的性别与战争》，玛丽亚·布克尔和南希·温菲尔德·布卢明顿编辑：印第安纳大学出版社2006年]>
“提图斯头像的女孩：慕尼黑和布达佩斯革命中的妇女，1919年”民族论文28/3（东南欧）
1 Eliza Ablovatski <p>Eliza Ablovatski joined ...
2 One More Biography: Some interesting facts...
1 1 0 <p><em>Zwischen Pruth und ...
1 2 0 <p><em>Czernowitz ist gewen ...
2 3 2001 Book1
2 4 2002 Book2
1 1 0 <p>"The Central European ...
1 2 0 <p>"Between Red Army and White ...
1 3 0 <p>"The Girl with the Titus-head: ...
2 4 2001 Article1
With MyPersonCTE AS
(
SELECT ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS PersonID
,p.value('FirstName[1]','varchar(max)') AS FirstName
,p.value('LastName[1]','varchar(max)') AS LastName
,p.value('Biography[1]','varchar(max)') AS Biography
,p.value('Expertise[1]','varchar(max)') AS Expertise
,p.value('Image[1]','varchar(max)') AS Image
,p.value('Link[1]','varchar(max)') AS Link
,p.query('Books') AS BookNode
,p.query('Articles') AS ArticleNode
--same for Papers, Artwork...
FROM @x.nodes('/People/Person') AS A(p)
)
,MyBooksCTE AS
(
SELECT MyPersonCTE.*
,ROW_NUMBER() OVER(PARTITION BY PersonID ORDER BY (SELECT NULL)) AS BookID
,x.value('Year[1]','int') AS BookYear
,x.value('Details[1]','varchar(max)') AS BookDetails
FROM MyPersonCTE
CROSS APPLY MyPersonCTE.BookNode.nodes('/Books/Book') A(x)
)
,MyArticlesCTE AS
(
SELECT MyPersonCTE.*
,ROW_NUMBER() OVER(PARTITION BY PersonID ORDER BY (SELECT NULL)) AS ArticleID
,x.value('Year[1]','int') AS ArticleYear
,x.value('Details[1]','varchar(max)') AS ArticleDetails
FROM MyPersonCTE
CROSS APPLY MyPersonCTE.ArticleNode.nodes('/Articles/Article') A(x)
)
--same for Papers, Artwork...
SELECT p.*
,b.BookID
,b.BookYear
,b.BookDetails
,a.ArticleID
,a.ArticleYear
,a.ArticleDetails
INTO #tempAllData
FROM MyPersonCTE AS p
LEFT JOIN MyBooksCTE AS b ON p.PersonID=b.PersonID
LEFT JOIN MyArticlesCTE AS a ON p.PersonID=a.PersonID ;
--#tempAllData is now filled with all data, copied in all combination: much to much
--but DISTINCT is your friend
--in this case you'd use the PersonID as FK in all related tables
SELECT DISTINCT PersonID,FirstName,LastName,Biography,Expertise --other fields
INTO #tempPerson
FROM #tempAllData;
SELECT DISTINCT PersonID,BookID,BookYear,BookDetails
INTO #tempBooks
FROM #tempAllData;
SELECT DISTINCT PersonID,ArticleID,ArticleYear,ArticleDetails
INTO #tempArticles
FROM #tempAllData;
DECLARE @columnNames VARCHAR(MAX)=
STUFF((SELECT DISTINCT ',Book_'+CAST(BookID AS VARCHAR(10)) FROM #tempBooks FOR XML PATH('')),1,1,'')
+(SELECT DISTINCT ',Article_'+CAST(ArticleID AS VARCHAR(10)) FROM #tempArticles FOR XML PATH(''));
DECLARE @cmd VARCHAR(MAX)=
'SELECT p.*
FROM
(
SELECT p.*
,''Book_''+CAST(BookID AS VARCHAR(10)) AS ColumnName
,ISNULL(CAST(BookYear AS VARCHAR(4)),'''') + '' '' + BookDetails AS Data
FROM #tempPerson AS p
INNER JOIN #tempBooks AS b ON p.PersonID=b.PersonID
UNION ALL
SELECT p.*
,''Article_''+CAST(ArticleID AS VARCHAR(10)) AS ColumnName
,ISNULL(CAST(ArticleYear AS VARCHAR(4)),'''') + '' '' + ArticleDetails AS Data
FROM #tempPerson AS p
INNER JOIN #tempArticles AS a ON p.PersonID=a.PersonID
) AS tbl
PIVOT
(
MAX(Data) FOR ColumnName IN(' + @columnNames + ')
) AS p;'
EXEC(@cmd);
DROP TABLE #tempArticles
DROP TABLE #tempBooks
DROP TABLE #tempPerson
DROP TABLE #tempAllData;