HTML contents using screen scraping in ASP.NET RSS

Question

0.00/5 (No votes)

See more:

I am trying to access the webpage programmatically. I do have the authorized account as well as the valid username and password. However,i run into problem of getting the access by passing the valid username and password using screen scraping so that i cannot do the further steps. I don't know which part of my code goes wrong.Please help.Below is my code

myReq2.Method = "POST"
myReq2.ContentLength = postData.ToString.Length
myReq2.ContentType = "application/x-www-form-urlencoded"
myReq2.CookieContainer = cookies
myReq2.KeepAlive = True

Dim sw As System.IO.StreamWriter = New System.IO.StreamWriter(myReq2.GetRequestStream())
sw.Write(postData)
sw.Close()

Dim myResp2 As System.Net.HttpWebResponse = CType(myReq2.GetResponse(), System.Net.HttpWebResponse)

Dim sr2 As System.IO.StreamReader = New System.IO.StreamReader(myResp2.GetResponseStream())

resp2 = sr2.ReadToEnd
sr2.Close()

Private Function ExtractViewState(ByVal s As String) As String

Return System.Web.HttpUtility.UrlEncodeUnicode(System.Text.RegularExpressions.Regex.Match(s, "(?<=__VIEWSTATE"" value="")(?<val>.*?)(?="")").Groups("val").Value)

End Function

Posted 2-May-13 8:32am

vidkaat

Updated 3-May-13 10:28am

v7

Add a Solution

Comments

ZurdoDev 2-May-13 16:51pm

I think you need to refer to the google api docs.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Steve44 · Answer 1 · 2013-05-02T18:45:00

I think the content of the stream that you post is not correct, you need to ASCII encoded the string before adding it to your request.
See the modifications to your code here:

VB

Dim postString As String
postString = "use...=<>&pass...=<>" ' insert the name-value pairs to post here

Dim postData As Byte()                         ' you are going to post encoded bytes

postData = Encoding.UTF8.GetBytes(postString) ' convert the string to the UTF8 encoded bytes

Dim myReq2 As System.Net.HttpWebRequest = System.Net.WebRequest.Create(roServerSettings.GET_MSI_URL & "Login.aspx")

myReq2.Method = "POST"
myReq2.ContentLength = postData.Length         ' The length of the encoded bytes
myReq2.ContentType = "application/x-www-form-urlencoded"
myReq2.CookieContainer = cookies
myReq2.KeepAlive = True
 
Dim sw As System.IO.StreamWriter = New System.IO.StreamWriter(myReq2.GetRequestStream())
sw.Write(postData)
sw.Close()

This should get the string correctly encoded to the other side.
If you still get an error, check, if you are sending correctly formatted name-value pairs as per the API specification.