Click here to Skip to main content
15,898,036 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hey guys. I'm trying to gather picture links at web.[DELETED].com but when i run my code I get a remote server error (500). Basically my program runs through a loop for every page every and gathers all the picture links into a list. I'm not sure if it's an error on their part or mine. There's 12 picture per page plus a link to the next page to parse. Here's my code:

What I have tried:

VB
<pre>        Do
            Dim wrWebRequest As HttpWebRequest = CType(WebRequest.Create(strURL), HttpWebRequest)
            Dim wrWebResponse As HttpWebResponse = CType(wrWebRequest.GetResponse(), HttpWebResponse)
            Dim strStream As Stream = wrWebResponse.GetResponseStream()
            Dim intBuffer(8388608) As Byte
            Dim intBytesToRead As Integer = CInt(intBuffer.Length)
            Dim intBytesRead As Integer = 0
            While intBytesToRead > 0
                Dim intNumber As Integer = strStream.Read(intBuffer, intBytesRead, intBytesToRead)
                If intNumber = 0 Then
                    Exit While
                End If
                intBytesRead += intNumber
                intBytesToRead -= intNumber
            End While
            strStream.Close()
            Dim fsFileStream As New FileStream("index.html", FileMode.Create, FileAccess.Write)
            fsFileStream.Write(intBuffer, 0, intBytesRead)
            fsFileStream.Close()
            fsFileStream = New FileStream("index.html", FileMode.Open, FileAccess.Read)
            Dim srSteamReader As New StreamReader(fsFileStream)
            Dim strData As String = srSteamReader.ReadToEnd
            srSteamReader.Close()
            fsFileStream.Close()

            Dim rgxPost As Regex = New Regex("""url"": ""https://web.[DELETED].com/p/\w*\S*"">")
            Dim mchPost As Match = rgxPost.Match(strData)
            While mchPost.Success
                lstPosts.Add(mchPost.ToString)
                mchPost = mchPost.NextMatch
            End While

            Dim rgxCursor As Regex = New Regex("cursor=\w*\S*&uid=\d*")
            Dim mchCursor As Match = rgxCursor.Match(strData)
            If mchCursor.Success Then
                strURL = "https://web.[DELETED].com/kaiagerber?" & mchCursor.ToString
            Else
                strURL = ""
                Exit Do
            End If
        Loop
Posted
Updated 11-Apr-19 2:45am
v3
Comments
OriginalGriff 11-Apr-19 8:23am    
Please don't post URL's to "random" websites - it makes you look like a spammer, and that with get you kicked off the site...

You can assume that this is your one and only warning. :laugh:

500 errors being returned by the server means the server-side code failed in some way.
That can be caused by two things. The first is a bug in the server-side code. Its rare in a production web site, but it can happen.

The other type of failure is one where the URL you put together and sent to the web server couldn't be legally parsed by the server and data properly assigned to variables in the web application code.

This second failure type is the most common generator of 500 errors. You gave the server some data the app code didn't know what to do with and/or you failed to give it some data it was expecting.
 
Share this answer
 
From :RFC for HTTP (https://tools.ietf.org/html/rfc7231#section-6.6.1[^])

6.6.1. 500 Internal Server Error

The 500 (Internal Server Error) status code indicates that the server
encountered an unexpected condition that prevented it from fulfilling
the request.


It looks like the error may be related to how the server is handling the request.

If you were submitting a bad URL then most likely you'd get a 404 (resource not found) but that does depend upon the server too.

You should double-check your URL that is created though (by stepping through code) and then verify manually using a browser what happens with the generated URL.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900