|
I have to download lots of web pages simultaneously, I'm able to queue all the discovered URLs and each is going to be downloaded in a new thread but I'm only able to download 20 to 25 web pages and for the remaining ones i get the Operation has Timed Out exception. I am using the HttpWebRequest/HttpWebResponse classes to communicate with the web server.
AliAmjad (MCP)
|
|
|
|
|
You cannot download any number of web pages you want at the same time. IE usually only allows 4 downloads to be going at once. Microsoft's implementation of TCP/IP on XP SP2 also only allows a limited number of connection attempts per second to occur. Any more than, I think, 10/second and the connection attempts fail.
Why?? It's there to limit the proliferation of viruses and reduce the impact of "denial of service" attacks.
Your queue should be popping off URLs to your threads at a throttled rate to stay under these limits.
|
|
|
|
|
Is there any other way to extend this limitation or we have to go with it actually I've to download a huge number of web pages and 4 simultaneous downloads will decrease the performance of this Web Crawler I want to at least download 100 web pages at once and for this I've generated 100 Worker threads using a custom Thread Pool and to prevent the denial of service attacks I've also implemented the Politeness Policy. So, What should i do should i change the operating system or is there any other way on windows...
AliAmjad(MCP)
|
|
|
|
|
AliAmjad wrote: Is there any other way to extend this limitation
Not that I'm going to suggest.
AliAmjad wrote: I've to download a huge number of web pages and 4 simultaneous downloads will decrease the performance of this Web Crawler I want to at least download 100 web pages at once and for this I've generated 100 Worker threads using a custom Thread Pool
Run it on either XP SP1, or, preferrably, on a Server edition of Windows.
AliAmjad wrote: and to prevent the denial of service attacks
This statement tells me you don't understand what a DOS, or DDOS, attack is.
|
|
|
|
|
Dave Kreskowiak wrote: This statement tells me you don't understand what a DOS, or DDOS, attack is.
Not exactly but I was thinking of it in terms of not overloading the web server. Did I say something incorrect. But thanks I'll run this web crawler under a server edition of Windows.
AliAmjad(MCP)
|
|
|
|
|
I am new to VB . NET and have been teaching myself with suprisingly significant success, however, I still have a lot to learn.
I've attempted to look this up on my own, but I am faced with a recurring question. Is there a method in "system.net.mail" that will return the VB Application User's e-mail provider information?
The reason I ask is that all the documentation I have read seems to imply that the programmer or user has to input the SMTP Server infromation manually. I was hoping to find a solution where the user's OS/.NET will be able to return this information automatically, eliminating interaction from the user.
It appears that Microsoft Excel has incorporated this functionality. It is demonstrated when a user creates an "E-mail HyperLink". You click on the HyperLink and the users E-Mail Client Pops up populated with the e-mail address of the recipient.
Any assistance is greatly appreciated.
Thank you in advance,
JM
|
|
|
|
|
Johnny Mar wrote: I've attempted to look this up on my own, but I am faced with a recurring question. Is there a method in "system.net.mail" that will return the VB Application User's e-mail provider information?
Nope. This is because that information can be stored in different places by different email apps.
Johnny Mar wrote: The reason I ask is that all the documentation I have read seems to imply that the programmer or user has to input the SMTP Server infromation manually.
True.
Johnny Mar wrote: I was hoping to find a solution where the user's OS/.NET will be able to return this information automatically, eliminating interaction from the user.
Not exactly possible considering the number of different places any email application can store settings like that.
Johnny Mar wrote: It appears that Microsoft Excel has incorporated this functionality. It is demonstrated when a user creates an "E-mail HyperLink". You click on the HyperLink and the users E-Mail Client Pops up populated with the e-mail address of the recipient.
This is NOT because Excel uses "standard" information. All Excel has to do to accomplish this is to shell launch a URL with the recipient address: "mailto:recipient@somewhere.com" You can try it yourself by typing this in the Start/Run box (without the quotes). Excel doesn't have to know anything about the registered default email client.
|
|
|
|
|
Dear All,
I am using access 2003 databas for my vb.net project (use vb.net 2003). My initial database size is 564KB (access file size). But when I run the program access file size is increasing. If I run the program 50 times the access file size will increase up to 6MB. I applied the "Compact on Close" feature also in Access application. But if i open and close the access file manually through MS Access it will decrease again to 564KB.
My database connection opens at the program start and closes at exit the program.
Pls help me to resolve this problem
Dimuthu
|
|
|
|
|
when accessing access thru vb the db isn't 'realy' there is only a connection to it so it will not compact.
depending on what you do in you'r programme this is normal behavior since access keeps track of everything that happens and never decreases its size unasked.
opening a access db as following wil automacticly compact and repair the db and close it again
"C:\Program Files\Microsoft Office\OFFICE11\MSACCESS.EXE" C:\DossierP.mdb /repair /compact
you can put it in a shortcut or in you'r code to be done on exiting the programme or so
If my help was helpfull let me know, if not let me know why.
The only way we learn is by making mistakes.
|
|
|
|
|
The increasing size of the database is normal for Jet (Access) databases. The option you picked only works if the database is opened in Access. It has no effect on other applications using the database file.
You can add Compact functionality to your app. There's an example on MSDN, here[^]. And a bunch of Google results for "vb.net compact database access[^]".
|
|
|
|
|
I have a form in which I have a Binding navigator, a binding Source, a DataSet and a TableAdapter, several text boxes to display data. when I change the data in the textboxes and move to the next or previous record, I want to display a msg to ask whether to save changes or not. for this I declared a boolean variable and changed its state to true when the user changes any data on the form. Now when i move records only (without making any changes to the data also), the boolean variable gets its value changed to true because of this change in record. so in which procedure i should i change the boolean variable's value so that I can track the changes on the screen by the user.
I think it should be when the binding is complete by the application, but don't know in which procedure of which control I should check it.
Any Idea, Please Help.
Sekhar 
|
|
|
|
|
If I get what you're asking, you can create an IsDirty field on your form and handle the ListChanged event of your BindingSource. In that handler, you can do something like:
Dim changes As DataTable = MyDataSet.TableName.GetChanges()
If changes IsNot Nothing AndAlso changes.Rows.Count > 0 Then
IsDirty = True
End If
When the user clicks the save button to commit the changes to the database, you handle the Click event of that button and check the IsDirty flag. If true, you can put up a MsgBox asking if they want to save changes or not.
|
|
|
|
|
Thank You Dave for your response.
But that doesn't seem to solve my problem. I wanted to know when the bindingsource's binding data getting completed. because when the binding source is populating the textboxes, the textbox's value gets changed and so does the value of the boolean variable. and when the user clicks on the next,previous buttons it should track whether data on the form has changed or not. And if changed it should ask the question. but when the binding source is binding data with the textboxes the value of boolean variable is changed to true. so i wanted to know when is the binding of data to all the textboxes is complete (for a particular record when we are moving from one record to another), So that after binding of data to all the textboxes is complete i can again change the value of the boolean variable to false and if the user changes the data while navigating, the boolean variable's value will change to true and then i can ask the question to the user whether to save or not.
Please give me some suggestions in which event of which control i should change the boolean variable's value to false.
Sekhar
|
|
|
|
|
SekharOne wrote: I wanted to know when the bindingsource's binding data getting completed.
There's no way to know that. The BindingCompleted event fires for EACH field in the row, not when the entire row is done all at once.
SekharOne wrote: the textbox's value gets changed and so does the value of the boolean variable.
Why are you using the Textchanged events of the TextBox's?? You don't have to.
SekharOne wrote: and when the user clicks on the next,previous buttons it should track whether data on the form has changed or not.
I showed you that. It fires when any field in the current row changes.
SekharOne wrote: And if changed it should ask the question.
This I misunderstood. The changes you make in the textboxes get made to the underlying DataSet. Unless you want to commit the changes to the database while leaving each changed row, you would normally just track if the dataSet is dirty and needs to be written back. Doing this on every change to the dataset would be a bit excessive.
But, if this is what you want, then you reset the dirty flag only after you call Update on the DataAdapter and write the changes back to the database.
|
|
|
|
|
Hello,
I have an error for using UDL file for connection strings in VB.NET.
Dim oConnection as New SqlConnection
oConnection.ConnectionString = "..\..\Connections.UDL"
It raises "Format 0" Exception.
Kindly let me know How to using udl file in VB2005.
Thanx,
Kumaresh
|
|
|
|
|
The error is telling you that the contents of the file are not valid for a UDL file.
I'm wondering why you're even bothering with it considering it's an outdated concept and everything in a UDL file can already be done in using a connection string in your app and storing connection strings in your app's config file.
|
|
|
|
|
Hi Dave,
I find the solution. That UDL file works on OLEDBConnection but I was using SQLConnection.
Thanks,
Kumaresh
|
|
|
|
|
Again, UDL files are an obsolete concept.
If you want to use OleDb on an SQL Server, go ahead. Just remember you're sacrificing performance and features specific to SQL Server if you do.
|
|
|
|
|
Hi,
Can somebody please tell me how to manage the property grid dynamically (i.e)
there are 50-100 properties in my database and not all are assigned values, i want to display only those properties which have values assigned..
can somebody suggest me some method for this.
i would be a great help.
Thanks in Advance
Anu
|
|
|
|
|
You cannot use a PropertyGrid for this. A PropertyGrid shows the public properties of an object, like an instance of a TextBox, or Label control.
You're need to use something like a DataGridView control, or some other bindable grid control to show your database data.
|
|
|
|
|
Hi,
simply write a class implementing ICustomTypeDescriptor and drop it to the SelectedObject property. This class will take care of publishing the properties it wants. With .Net 2.0 you can also use the TypeDescriptionProvider so that you don't have to modify your existing class to make it implement ICustomTypeDescriptor. You will find some resources about this subject on the PropertyGrid Resource List.
If a commercial component is an option for you, you have also Smart PropertyGrid. It has a mode where you can simply call AppendProperty-like methods and fill the grid completely dynamicall without having to build separate TypeDescriptor classes.
I hope this helps.
|
|
|
|
|
I think both of these guys are doing the same thing. My take on what they were doing was storing storing a bunch of name/value pairs in a database (they call them "properties"), then expecting the PropertyGrid to show those records. This, obviously, won't work in a straight forward manner by pointing the SelectedObject of the PG at the DataTable object. You'll get the properties of the table, not the data in it.
Are you saying you would either create a custom class deriving from CustomTypeDescriptor or modifying an existing class, implementing ICustomTypeDescriptor, to wrap these name/value pairs as a PropertyDescriptorCollection?? Wouldn't you also have to extend the schema of the datatable to include, at a minimum, type information for each "property"??
|
|
|
|
|
Hello Dave,
There is not a lot of detailed informations in the original post. So I gave a generic advice since the OP seems to ignore that the PropertyGrid can display something else than the simple sum of the public properties in a target instance. How he will get the data in his business objects and how he knows their type is beyond my knowledge of his application.
|
|
|
|
|
I'm just guessing at their applications too.
But, any time I come up with, for example, a settings class, it usually involves a serializable class full of properties, custom types, editors, and maybe a little custom serialization code, then wrapping a UI around it, customized for that class and/or a little more generic, like using a PropertyGrid.
As you probably know, adding new properties to a class like that results in "some" extra work and versioning issues. What I'm curious about is making that process more generic, where I could get away with, say, dumping a bunch of settings name/values into a database or other collection, along with the respective type information, and exposing it to a PropertyGrid through a ICustomTypeDescriptor implementation.
Hmmm...
|
|
|
|
|
If it is for end user (versus a quick and dirty object property view for developers), property grid is not a good control for it. Probably that's why it's not included in the toolbox by default.
You can add an attribute Browsable (System.ComponentModel.BrowsableAttribute)to true or false but I don't think you can change it dynamically.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
