|
Hi All,
I have a Bulk Copying Process which runs as nightly job puts data from db1 to db2 and scheduled in Windows schedular.
During this time its putting data in 3 files like 1. LogFile, 2. Status.txt and 3. Dat files. As it continues the files sizes increase, when it exceeds the size of existing physical memory, the BCP is failing.
What we need to do is to track if the BCP is failed. If it fails we need to get that information. Either from windows schedular or by calling any com component of emailing and to send the failure information.
Thanks,
Aleem Mohammad.
Thanks & Regards,
Md. Abdul Aleem
NIIT technologies
|
|
|
|
|
It sounds like what you really need to do is rewrite your code so you're not loading entire files into memory all at once. Since this is just a bulk copy, you can get away with reading and processing one line at a time.
|
|
|
|
|
Sorry I didnt get your point can u please eloborate me.
Thanks & Regards,
Md. Abdul Aleem
NIIT technologies
|
|
|
|
|
The point is you didn't explain anything about the code that's causing the problem. You don't say anything about how it handles these files, why you suspect your using so much RAM, no nothing. So, how can anyone possibly help you.
|
|
|
|
|
If you're talking about the bcp utility which ships with SQL Server, it doesn't read the whole file to memory so you are most likely hitting some other problem. One way to enhance bcp is to use batches.
However, to your question, add -e err_file parameter to the bcp command. That will log the errors to the file specified.
|
|
|
|
|
hi, can someone help me. i am a bit confused on the compatibility of vb.net and ms excel 2007.
im working on a project that would use excel in a form..all the sample codes that i have researched, they didnt say if it'll work with 2007 excel. i really hope that you guys woud help on this.
thanx! kudos...\m/
|
|
|
|
|
|
I have two existing (complex) .NET 3.0 Winform apps A and B that need to exchange information synchronously (using request/response). Both apps are running on the same computer at present, but perhaps over a LAN later.
Since I've used .NET Remoting in the past, I figured that's the way to go...
So I setup app A to host class X for Remoting, and I setup app B to instantiate class X (on A) and then invoke its methods.
Works great, except class X (being hosted on A) can't actually access the hosting app A. For example, it can't invoke A's methods, access its variables, etc.
The functional requirement is to have apps A and B exchange information, not simply to allow B to invoke an "isolated" object that's hosted by A, but which can't really interact with app A.
I've been searching all day, and so far all I've found is a passing reference that it should be possible for app A "to get a proxy to the remote object". But I don't have a clue how to do that, or what it implies.
ANY help would be greatly appreciated!
DT
|
|
|
|
|
I was wondering...you could you a message queue for this kind of scenario...which could make the code more simpler...
Moim Hossain
R&D Project Manager
BlueCielo ECM Solutions BV
|
|
|
|
|
I've encountered this exception with the code below. Clearly the stream hasn't been disposed, and Microsoft have acknowledged this as a bug and released a hotfix, but when I run the hotfix, I get an error saying the program to be updated may be missing or the wrong version. Does anyone know anything about this?
using (FtpWebResponse response = (FtpWebResponse)ftpRequest.GetResponse())
{
StreamReader sr = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
string fileName;
do
{
fileName = sr.ReadLine();
Console.WriteLine(sr);
}
while (sr != null);
|
|
|
|
|
Hi Brady,
I am not familiar with this problem, I do have some comments/questions:
- how often does it fail and under which circumstances?
- there are several 100 Google hits for your subject line
- you did not mention at what line the exception gets thrown
- you did not provide any info/link regarding the Microsoft acknowledgement
- are you aware StreamReaders need disposed of too? you might consider a second using statement.
Hope you'll get a solution.
|
|
|
|
|
Apologies.
1. It always fails, under all circumstances. There aren't really many circumstances.
2. That's how I found the MS patch.
3. The exception gets thrown on sr.ReadLine();
4. KB918462.
5. Thanks. This is server side and a leak could be expensive.
|
|
|
|
|
OK,
based on "...The HttpWebRequest class automatically tries to reopen a connection to the Web server to complete the transfer. When the transfer is successful, the EndGetRequestStream method incorrectly returns the Stream object from the first failed connection attempt, instead of the Stream object from the successful connection attempt..." from here[^] my best guess is you are systematically getting the same error on the first attempt, possibly a timeout (that is assuming your request is correct of course). Can you try and allow a larger timeout? or is there a huge amount of data involved?
You might try to validate your web response before calling on its stream? Maybe the StatusDescription property might shed some light.
|
|
|
|
|
Luc Pattyn wrote: Maybe the StatusDescription property might shed some light.
226 Transfer completed.  StatusCode: Closing Data.
This is a listing of my personal web site, not too many files. When I run LS in a console I get:
1751 bytes received in 0.07Seconds 25.75Kbytes/sec.
|
|
|
|
|
First impression is the FTP server has a bug!
You might want and test:
1. your client code on a different FTP server;
2. other FTP clients on your current FTP server.
|
|
|
|
|
Luc Pattyn wrote: First impression is the FTP server has a bug!
My first impression is that the framework has a bug and that Microsoft have even issued a hotfix for it. I will nonetheless perform your suggested tests.
|
|
|
|
|
Yes, I believe Microsoft admitting to a bug that the wrong stream gets returned; but there should
not consistently be a need for a retry in the first place, (the internals of) GetRequestStream should succeed right away, of course a failure might happen but not all the time, so your code should work most of the time. Since it doesn't, I would suspect the server side.
|
|
|
|
|
Pick an FTP site I can rely on for this investigation.
|
|
|
|
|
Whatever public server you like, a software or hardware vendor's, Microsoft ("ftp.microsoft.com"), Intel ("ftp.intel.com"), whatever offers access without an account (i.e. anonymous);
or a site you have an account for, e.g. your own ISP if you have your own site.
All servers and all clients are slightly different, so no test is definitive, but performing both
tests (cross-testing: suspected client+known good server, and known-good client+suspected server) gives a good indication whether the problem is probably in the server or in the client.
|
|
|
|
|
Luc Pattyn wrote: whatever offers access without an account
I just tried the below code on "ftp.microsoft.com" and got the same exception, but not before some console output, i.e:
deskapps
KBHelp
MISC1
Products
ResKit
Softlib
FtpWebRequest req = (FtpWebRequest)FtpWebRequest.Create("ftp://ftp.microsoft.com");
req.Proxy = null;
req.Method = WebRequestMethods.Ftp.ListDirectory;
using (FtpWebResponse res = (FtpWebResponse)req.GetResponse())
{
StreamReader sr = new StreamReader(res.GetResponseStream(), Encoding.UTF8);
string fileName;
do
{
fileName = sr.ReadLine();
Console.WriteLine(sr.ReadLine());
}
while (fileName != null);
}
|
|
|
|
|
Hi,
I ran the code below, which is basically yours (I now count the lines, and I fixed your Console output which was reading another line):
log("start");
FtpWebRequest req = (FtpWebRequest)FtpWebRequest.Create("ftp://ftp.microsoft.com");
req.Proxy = null;
req.Method = WebRequestMethods.Ftp.ListDirectory;
using (FtpWebResponse res = (FtpWebResponse)req.GetResponse()) {
StreamReader sr = new StreamReader(res.GetResponseStream(), Encoding.UTF8);
int count=0;
for( ; ; ) {
string fileName = sr.ReadLine();
if (fileName==null) break;
log(fileName);
count++;
}
log("Received "+count+" lines");
}
and got what I was expecting:
23:46:20.277 CPTest.log-42 start
23:46:25.932 CPTest.log-42 bussys
23:46:25.945 CPTest.log-42 deskapps
23:46:25.958 CPTest.log-42 developr
23:46:25.970 CPTest.log-42 KBHelp
23:46:25.983 CPTest.log-42 MISC
23:46:25.996 CPTest.log-42 MISC1
23:46:26.009 CPTest.log-42 peropsys
23:46:26.037 CPTest.log-42 Products
23:46:26.050 CPTest.log-42 PSS
23:46:26.063 CPTest.log-42 ResKit
23:46:26.077 CPTest.log-42 Services
23:46:26.086 CPTest.log-42 Softlib
23:46:26.281 CPTest.log-42 Received 12 lines
So that tells me the Microsoft server and your client code are rather compatible, so there might be
something wrong with your Internet connection, although I can't begin to understand what it might be.
Is it special in any way? maybe extremely slow? I suggest you try with a well established FTP client so you can feel how it reacts, and maybe see some errors/warnings.
FYI: I did try Microsoft's site with FileZilla 3.1.2, and my Internet is a 3Mb/s ADSL.
|
|
|
|
|
Luc, for you, I keel dee bool! ![Rose | [Rose]](https://codeproject.global.ssl.fastly.net/script/Forums/Images/rose.gif)
It was that extra read that swung it.
|
|
|
|
|
Brady Kelly wrote: I keel dee bool
I don't know that bull, but you're welcome.
|
|
|
|
|
Come to think of it, since you where reading two lines per iteration, you only showed six,
and you might (did) try and read a line after you tested for null. That must be why the stream objected.
BTW: the code you showed originally was incorrect in a different way, and would not have compiled.
I guess you are better of showing actual code right from the start.
|
|
|
|
|
Yes, I just didn't have the original code at hand. BTW, why wouldn't it compile? The only reason I can see is that it's incomplete, but I just posted an excerpt.
This one was a nasty. The original code was getting an FTP 550 error, file not found or no access, because the directory I was trying to create already existed. It made no check of this, and almost ignored the FTP error by only catching and logging the exception. Because I was breaking when thrown, I noticed an exception that is actually often expected, but I suspected my changes, adding authorization, had introduced the problem. Because another developer had broken the function I needed to execute the code, I extracted it into a separate project to experiment, beginning my troubles.
[Edit]
Ah, I see in the original code that I was trying to write the StreamReader, not the filename, which results in System.IO.StreamReader appearing in the console, just before the same disposed object exception.
[/Edit]
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
