Click here to Skip to main content
15,898,926 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
Hi, Please see my code below it works fine with small xml files but it does not work with large xml files (of size 2496kb or more)
appreciate your help
-------------------

SQL
USE [SP3]
GO
/****** Object:  StoredProcedure [dbo].[usp_user_longimportInfo_FileName_trial1]    Script Date: 05/10/2010 16:09:59 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
---- Good One ---- for content 
ALTER procedure [dbo].[usp_user_longimportInfo_FileName_trial1] 
(@XMLFileName as nvarchar(100))
as
begin
set @XMLFileName='c:\trial\shortperson.xml'
create table #tempPerson ([UserID] [int] IDENTITY(1,1) NOT NULL,
	[Email] [nvarchar](50)  NULL,
	[Pass] [nvarchar](20)  NULL,
	[FirstName] [nvarchar](20) NULL,
	[LastName] [nvarchar](40) NULL,
	[DisplayName] [nvarchar](50) NULL,
	[Profile] [nvarchar](max) NULL,
	[DisplayEmail] [nvarchar](50) NULL,
	[CellPhone] [nvarchar](20) NULL,
	[UpdatedBy] [int] NULL,
	[UpdateDate] [datetime]  NULL,
	[Deleted] [bit]  NULL)

DECLARE @TempCurrentTime datetime
DECLARE @USERID int
Declare @UpdateDate datetime
Declare @Email nvarchar(50)
DECLARE @Pass NVARCHAR(20)
DECLARE @FirstName NVARCHAR(20)
DECLARE @LastName nvarchar(40)
Declare @DisplayName nvarchar(50)
Declare @Organization int
Declare @UserAccessRole int
Declare @OrgUserID nchar(12)
Declare @OrgPassword nvarchar(50)
Declare @OrgUserName nvarchar(50)
declare @mySQL nvarchar(max)
declare @PersonXML xml
declare @ParamDefinition nvarchar(500)
 
SELECT @TempCurrentTime = GETDATE()
 
		select @UpdateDate=@TempCurrentTime
		-- retrieve the file content as xml
					set @mysql=N'select  @PersonXML=
					 CONVERT(xml, BulkColumn, 2) FROM 
						OPENROWSET(Bulk '''+ @XMLFileName+''', SINGLE_BLOB ) [rowsetresults]'
					    
				  
					 Set @ParamDefinition = '@XMLFileName nvarchar(max),@PersonXML XML out'
					  Execute sp_Executesql	@mySql,
					   @ParamDefinition,
					   @XMLFileName,
					   @PersonXML out
 
insert into #tempPerson (Email,Pass,FirstName,LastName,DisplayName,[Profile],DisplayEmail,CellPhone,UpdatedBy,UpdateDate,deleted)

			(SELECT TempXML.Node.value('(email)[1]','nvarchar(50)') as Email,
			' ',
			  TempXML.Node.value('(name/n/given)[1]', 'nVARCHAR(50)') as FirstName,
			 TempXML.Node.value('(name/n/family)[1]', 'VARCHAR(50)') as LastName,
			  TempXML.Node.value('(name/fn)[1]','nvarchar(50)') as DisplayName,
			  ' ',
			  TempXML.Node.value('(email)[1]','nvarchar(50)') as DisplayEmail,
			  ' ',1,GETDATE(),0
			  FROM @PersonXML.nodes('/enterprise/person') TempXML (Node)) 
select * from #tempPerson
insert into dbo.longPersonError
select  USERID, Email 
from(
MERGE SP.UserTrial 
    USING (SELECT Email,Pass,FirstName,LastName,DisplayName,[Profile],DisplayEmail,CellPhone,UpdatedBy,UpdateDate,deleted from #tempPerson) AS source 
    ON (SP.UserTrial.Email = source.Email)
    WHEN Not MATCHED  THEN
    
    		INSERT (Email,Pass,FirstName,LastName,DisplayName,[Profile],DisplayEmail,CellPhone,UpdatedBy,UpdateDate,deleted)
			VALUES (source.Email,source.Pass,source.FirstName,source.LastName,source.DisplayName,source.[Profile],source.DisplayEmail,source.CellPhone,source.UpdatedBy,source.UpdateDate,source.deleted) 
	when matched then
	       UPDATE SET SP.UserTrial.Email = SP.UserTrial.Email
	       OUTPUT $ACTION, deleted.userid,deleted.Email ) AS CHANGES (Action,userid,email)
WHERE Action = 'UPDATE';
  
end


Thanks in advance
Posted
Updated 10-May-10 13:25pm
v2
Comments
DaveAuld 10-May-10 19:27pm    
Are you sure its a size thing and not a command timeout etc?
Sandeep Mewara 11-May-10 1:31am    
I too agree with Dave, looks like a timeout issue when file is greater than certain size.

Instead of SQL Server breaking the XML, you can transform XML using XSL in your code file and generate Insert Queries/SP call as text output, this output inturn can be supplied to SQL server one by one using ADO.Net..

Beauty of this approach is you can easily debug XSL and test various scenario quickly. I have done this many a times and its incredibly fast and easy to maintain

Just food for thought
 
Share this answer
 
v2
Thanks I am not using .net I am using coldfusion.
May you provide me a link to your approach
I tried to use ssis xml task xpath to extract only the required information into another xml file but I got one problem
I used ssis xml task
My input file has the following format
<enterprise>
<person>
<sourcedid>
<source>111111</source>
<id>22222</id>
</sourcedid>


<name>
<fn>xxxxxxx</fn>
<n>
<family>yyyyy</family>
<given>zzzzzz</given>
</n>
</name>
<demographics>
<gender>2</gender>
</demographics>
<email>xxxxxx@fffff.edu</email>
<adr>
<street>cccccc</street>
<locality>ccccc</locality>
<region>cccc</region>
<pcode>ccccccc</pcode>
</adr>
<academics>
<academicmajor>gggggg</academicmajor>
<customrole>hhhhhh</customrole>
<customrole>dddddd</customrole>
<customrole>cccccc</customrole>
</academics>

</person>
<person>

</person>
</enterprise>
_______________________
on using ssis xml task xpath I got the following output

The output I got is in the form of
<resultsetnode>
<fn>xxxx</fn>
<family>yyyy</family>
<email>zzzz</email>
<customrole>xxxxx</customrole>
<customrole>zzz</customrole>
<customrole>kkkkkk</customrole>
<fn>aaaa</fn>
<family>bbbb</family>
<email>cccc</email>
<customrole>dddddd</customrole>
<customrole>eeee</customrole>
<customrole>wwwww</customrole>
</resultsetnode>
I want the output to be in the form of
<resultsetnode>
<person>
<fn>xxxx</fn>
<family>yyyy</family>
<email>zzzz</email>
<customrole>xxxxx</customrole>
<customrole>zzz</customrole>
<customrole>kkkkkk</customrole>
</person>
<person>
<fn>aaaa</fn>
<family>bbbb</family>
<email>cccc</email>
<customrole>dddddd</customrole>
<customrole>eeee</customrole>
<customrole>wwwww</customrole>
</person>
</resultsetnode>
How can I achieve that
Thanks again
 
Share this answer
 
I broke it using ssis and xml task xslt it is reduced to 800 kb but still it is slow please advise
 
Share this answer
 
Comments
DaveAuld 11-May-10 17:10pm    
For info: Below the Question, and Below each answer, there is an Add Comment facility to allow discussion around the questions and the offered solutions. Please use these. What you have posted as answers are not answers but comments.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900