|
Thanks that will be enough.
|
|
|
|
|
Hi All,
Feels good to be a part of this forum of technical stalwarts and Gurus. I feel I will get a solution for my long time problem here .
The problem we are facing here is the performance issues when processing the large XML files. Using Java API transformer factory in interpretive mode takes unbelievable 6 days for processing huge files which is a point of very serious concern in a production system.
Please help me out and do let me know what are the different things required to be attached which will be helpful for the analysis.
|
|
|
|
|
praveen3g wrote: takes unbelievable 6 days for processing huge files
Most people would have figured there was something wrong before this. Take a look at the data you are processing and how you are dealing with it. The chances of anyone being able to guess what is wrong is not very high.
|
|
|
|
|
Hi,
Thanks for the reply.
Answer is no , to the people whom I have checked with was not able to diagnose where the problem is and what is the way around it. Everyone definitely understand that the volume of data in the file is huge , say 100MB xml document but no one till data have given a way to work around it efficiently. Let me know if you wish to have a glance and will look forward to your suggestions
|
|
|
|
|
This is not the sort of issue that can be resolved in a forum like this. You need to do some analysis of your code and measuring of your processing time and code paths. Try running against some small data files and see how long it takes and try to work out whether that is an acceptable length of time.
|
|
|
|
|
I have done my analysis and I guess you do not understand actually what the issue is.Do understand am not a kinder garden student to just post a question and expect a ready made answer as I have done quite a lot of research in it more than you . I will look for someone who can just help me out on what are the different parameters that can be looked for rather than just replying generic. Please do not bother to reply further. Thanks again.
|
|
|
|
|
praveen3g wrote: you do not understand actually what the issue is.
Yes, I understand perfectly: your code runs slow and you want to know why.
praveen3g wrote: I will look for someone who can just help me out on what are the different parameters that can be looked for rather than just replying generic.
Given the information you have provided how can anyone guess what those parameters are?
|
|
|
|
|
I have asked specifically what sort of information I should update here which is required for your analysis as I don't know what logs, JVM settings , XSL snippets which is taking more memory and time you may require for your analysis.If you have asked me any one of the above details or whatever you think is required, I can definitely provide them at first place.
|
|
|
|
|
praveen3g wrote: you may require for your analysis.
Like most people on this forum, I do not have the time or resources to analyse an issue like this. As I said earlier this question cannot be answered in a forum such as this. You need to do the analysis and ask a more specific and detailed question before anyone can hope to offer any suggestions.
|
|
|
|
|
I have done my part of analysis and even know where it is taking time.
So I am left with 2 options
1) Splitting the large XML file to small chunk of files based on a condition.
2) Increasing the DTM threads in Java API so that compiled version of Xalan can be used.
Let me know if you have any ideas in one of the options above. I can ask questions to the point on the option you say
|
|
|
|
|
Have you tried using a profiler? A colleague of mine wrote an xml parser/validator in .Net for large files and the first file would complete in about 20-30 minutes. After the first though, it would get incredibly slow, taking around 18-30 hours to process 50mb files. When we checked the memory allocations there were about 10 times more than necessary. The app was running out of memory and continually hitting the page file on disk to make up the difference. All of that because he didn't understand the immutable nature of .Net Strings.
EDIT - I realize you are not using .Net, but I am thinking you could try a profiler for java.
That is the best advice I can give, good luck friend. 
|
|
|
|
|
I feel for this poster....
The issue is most likely poor XML and XSLT design. Too many people refuse to take the time to understand their data and XML and in the rush to 'get something working' they create a schema that is bloated, incomplete and that requires a lot of resources to even attempt to use.
Six days for 100MB? Yes - I can say that the schema does not represent the data well and the XSL is probably even worse because it must translate this rickety XML into something that might be even more poorly designed and implemented and it is apparent that the XSL has been where all of the changes and fixes have been applied.
For starters -- do not avoid using attributes simply because elements seem 'easier' -- they are not. Things that describe a thing -- are attributes. Things that own things -- are usually elements.
I post here because -- when you try to engineer a system and write the entire application at the same time -- all the while saying -- just get it running, we'll fill in the details later -- and also you take alot of shortcuts ('it will be easier if we do ....')....all you are doing is moving the work further down the line and making it more difficult....
There is a definite amount of work that must be done -- never assume that you can avoid it and do as much up front as you can.
|
|
|
|
|
I have a program I have written that displays the contents of a xml file in a browser. I am trying to make it more user friendly for people to change the xml files without opening them in notepad or a canned xml editor. What I would like to do is be able to pull the information from the xml file into a webpage and allow the users to make changes and save it back to the xml file. I've done google searches for about a week and can't come up with a good way to do it. Any suggestions would be much appreciated.
|
|
|
|
|
I'm doing one currently where the xml is shown in the browser control (after xslt applied to it) and if the user needs to add data (we are not changing anything)I have a form come up to fill in the data. This was the most direct method for me but if that doesn't work you may have to look in the javascript direction.
|
|
|
|
|
Marc Clifton has a good article here where he parses the schema, and produces a dialog based form showing the message content. Well worth a look.
Ger
|
|
|
|
|
Is there any free xml tool to acquire xpath value of some element or attribute?
thanks!
|
|
|
|
|
Hello. Im trying to create an xml schema that can capture the following behavior:
1. <info> element should be the first inside <root>
2. <ParamA>, <ParamB>, <ParamC>, <ParamD>, and <ParamE> are optional, should appear after the <info> element and can appear in any order.
The following xml are valid:
<root>
<info></info>
<ParamA></ParamA>
<ParamB></ParamB>
<ParamC></ParamC>
<ParamD></ParamD>
<ParamE></ParamE>
</root>
OR
<root>
<info></info>
<ParamB></ParamB>
<ParamD></ParamD>
<ParamC></ParamC>
<ParamA></ParamA>
<ParamE></ParamE>
</root>
The following xml is NOT valid:
<root>
<ParamB></ParamB>
<ParamD></ParamD>
<ParamC></ParamC>
<ParamA></ParamA>
<info></info>
<ParamE></ParamE>
</root>
I have tried doing:
<xs:sequence>
<xs:element name="info" minOccurrence="0">
<xs:all>
<xs:element name="ParamA" minOccurrence="0">
<xs:element name="ParamB" minOccurrence="0">
<xs:element name="ParamC" minOccurrence="0">
<xs:element name="ParamD" minOccurrence="0">
<xs:element name="ParamE" minOccurrence="0">
</xs:all>
</xs:sequence>
This causes an invalid xml schema error because an all model group is not allowed to appear inside another model group.
I've also tried substitution:
<xs:complexType name="information">
<xs:all>
<xs:element name="info" minOccurrence="0">
</xs:all>
</xs:complexType>
<xs:complexType>
<xs:complexContent>
<xs:extension base="information">
<xs:all>
<xs:element name="ParamA" minOccurrence="0">
<xs:element name="ParamB" minOccurrence="0">
<xs:element name="ParamC" minOccurrence="0">
<xs:element name="ParamD" minOccurrence="0">
<xs:element name="ParamE" minOccurrence="0">
</xs:all>
</xs:extension>
</xs:complexContent>
</xs:complexType>
This is still getting an xml schema error on the usage of all model group.
Does anyone have a similar problem or have any idea how to create an xml schema for the desired behavior stated above? Thanks.
modified 27-Apr-12 5:56am.
|
|
|
|
|
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="root">
<xs:complexType>
<xs:sequence>
<xs:element name="info" maxOccurs="unbounded" minOccurs="1">
<xs:complexType>
<xs:all>
<xs:element ref="paramA" maxOccurs="1" minOccurs="1"/>
<xs:element ref="paramB" maxOccurs="1" minOccurs="1"/>
<xs:element ref="paramC" maxOccurs="1" minOccurs="1"/>
<xs:element ref="paramD" maxOccurs="1" minOccurs="1"/>
<xs:element ref="paramE" maxOccurs="1" minOccurs="1"/>
</xs:all>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="paramA" type="xs:string"/>
<xs:element name="paramB" type="xs:string"/>
<xs:element name="paramC" type="xs:string"/>
<xs:element name="paramD" type="xs:string"/>
<xs:element name="paramE" type="xs:string"/>
</xs:schema>
|
|
|
|
|
Hello,
Below is my XML document.
<GetCountersResult>
<GetCountsResult xmlns="">
<Count>
<ID>FP108MR</ID>
<Value>0</Value>
<Part>Z9999</Part>
</Count>
</GetCountsResult>
</GetCountersResult>
I need to remove only <GetCountsResult> node so that my xml document remains as below.
<GetCountersResult>
<Count>
<ID>FP108MR</ID>
<Value>0</Value>
<Part>Z9999</Part>
</Count>
</GetCountersResult>
I used the following code to convert Dataset to a XMLdocument.
Dim dtCounter As New DataSet("GetCountsResult")
oDatabase.Connect()
oDatabase.PopulateDataSet("Count", "Get_Current_Counter", True, dtCounter)
Dim sw As New StringWriter
dtCounter.WriteXml(sw, XmlWriteMode.IgnoreSchema)
Dim xmldoc As New XmlDocument
xmldoc.LoadXml(sw.ToString())
Please help me in getting this.
Thanks.
|
|
|
|
|
The ReplaceChild method of XmlNodeclass explained here http://msdn.microsoft.com/en-us/library/system.xml.xmlnode.replacechild.aspx[^] can be used for replacing the desired nodes as shown below:
void Main()
{
string xmlCounters =
@"<GetCountersResult>
<GetCountsResult xmlns="""">
<Count>
<ID>FP108MR</ID>
<Value>0</Value>
<Part>Z9999</Part>
</Count>
</GetCountsResult>
<GetCountsResult xmlns="""">
<Count>
<ID>FP108MR No2</ID>
<Value>0 No2</Value>
<Part>Z9999 No2</Part>
</Count>
</GetCountsResult>
</GetCountersResult>";
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(xmlCounters);
var countNodes = xmlDoc.GetElementsByTagName("Count");
for (int i=0; i < countNodes.Count; i++)
{
xmlDoc.DocumentElement.ReplaceChild(countNodes[i], countNodes[i].ParentNode);
}
string fileName = @"F:\ModifiedXml.xml";
xmlDoc.Save(fileName);
}
modified 25-Apr-12 4:34am.
|
|
|
|
|
Thank you so much. I will try this.
|
|
|
|
|
You're welcome. After trying please give your feedback and if it is helpful you may consider to vote.
Thank you for the response.
|
|
|
|
|
I have an integer attribute declared in the xsd file. In the xml file, I need to use hex for the value. I can’t find info about the syntax about hex value.
I tried something but neither
<MyData MyValue="0x52590"/> nor <MyData MyValue="#52590"/> works. Any clues are appreciated.
modified 10-Apr-12 9:38am.
|
|
|
|
|
<MyData MyValue="0x52590"/>
Independent ACN Business OwnerMake toll free long distance calls from your smart phone with ACN Mobile World. When connected via wifi, calls will not use any of your minutes or data, nor will there be any roaming charges.
Certain conditions apply. See my website for details.
Within you lies the power for good - Use it!
|
|
|
|
|
Frank W. Wu wrote: I have an integer attribute declared in the xsd file. In the xml file, I need to use hex for the value. I can’t find info about the syntax about hex value.
Because that isn't possible.
Definition of an 'int' is a optional sign followed by decimal digits. Nothing else.
The definition.
http://www.w3.org/TR/xmlschema-2/#int[^]
In terms of a solution - convert the hex value to an integer or change the xsd.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
