Click here to Skip to main content
15,891,951 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I have been working on a proxy server and have been using sockets to request data from web servers but now i then i get a Forbidden 403 error with 'directory listing denied' and yet when i test the request headder in fiddler it works in returning the page but looking at the logs it shows.

HTTP Pipelining Client detected; excess data on client socket for session #1.

Not sure what this i telling me but shown below is sample code that produces the 403 error.

C++
string Request = "GET http://www.bmw.co.uk/bmwuk/homepage/ HTTP/1.1\r\n";
            Request += "Host: www.bmw.co.uk\r\n";
            Request += "Referer: www.bmw.co.uk\r\n\r\n";
            IPEndPoint Ep = new IPEndPoint(Dns.GetHostEntry("www.bmw.co.uk").AddressList[0], 80);
            Socket Soc= new Socket(Ep.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
            Soc.Connect(Ep);
            Soc.ReceiveBufferSize = 1024;
            Soc.Send(ASCIIEncoding.ASCII.GetBytes(Request));
            byte[] Buffer = new byte[1024];
            int BytesReply=Soc.Receive(Buffer);
            string strReply=ASCIIEncoding.ASCII.GetString(Buffer);
            Soc.Close();


Typing "www.bmw.co.uk/bmwuk/homepage/[^] in to a browser show the page and not the directory so i would guess the folder uses a default web page.

Some sites are coded to check the Referer string but as you can see i've sent it and i also tried tweaking the buffer size so i'm a bit stuck on this one if anyone has the answer.
Posted
Updated 16-Mar-10 6:16am
v2

Are you saying that this error happens intermittently, or repeatedly?

I'm not familiar with "fiddler," or its log files, please explain.

Here's one fact which might help you out:

When browsers send a GET request, they usually don't include the domain name in the URL of the request. For instance, if you're connected to www.bmw.co.uk, your GET should read

GET /bmwuk/homepage HTTP/1.1

Try that and see if it helps.

Also, leave off the trailing slash character. That might be what's causing the server to think you're requesting a directory list.
 
Share this answer
 
v2
Thank you Richard Andrew that did the job.

Fiddler is a tool that works as a proxy server so that you can inspect the request/responce sent to web servers and this shows the full address in the GET and when you edit a request and leave the full url in the request it works fine from fiddler on all server and hence my mistake.

I think fiddler tweaks the truth because it does not show the real request from the browser that often included

C++
Proxy-Connection: Keep-Alive


and i would guess it's not showing the real request sent to the server which is bad when you think about it.

using the full url in the GET works on most servers like IIS/6 but not on others so i suspected it had something to do with socket setting so thanks for your help.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900