Handling HTTPS CONNECT request

Question

1.00/5 (1 vote)

See more:

.NET4.5

The CONNECT request don't work with HttpWebRequestand they always throw exceptions like 404 not found, or underlying connection closed. I think it is because they are HTTPS and not HTTP. So when requiring to handle CONNECT requests what exactly can be done? (Sending Connect Request and collecting back the Response) I will write the response later to a TCPClient stream.

Posted 28-Feb-15 8:53am

Member 8353523

Add a Solution

Comments

Member 11484204 1-Mar-15 2:29am

in c# in the console apllication to create the app

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

**Zoltán Zörgő** · Answer 1 · 2015-02-28T09:08:00

Solution 1

CONNECT is a method reserved for tunelling (see: https://tools.ietf.org/html/rfc2616#section-9.9[^]).
On the other hand, HttpWebRequest implements proxy support[^].
What is your purpose with it?

Posted 28-Feb-15 9:08am

Zoltán Zörgő

Comments

Member 8353523 28-Feb-15 15:18pm

I have implemented a proxy to which a browser connects and surfs the web (for unblocking regional content filtering). It is working but I am only able to handle HTTP GET and POST and not able to handling HTTPS CONNECT AND PUT using HttpWebRequest. So that is where I have stuck.

Zoltán Zörgő 28-Feb-15 15:24pm

Still unclear.
You already have a proxy?
It is made by you or it is a ready made one?
If you use a browser with that proxy, it is working as expected?

I am asking this because CONNECT is a method not meant to be used directly as GET and POST. And PUT, DELETE (used for REST APIs in general) might not be supported or enabled by default on that proxy.

With other words while GET, POST, PUT, DELETE (and others) are meant to pass trough the proxy, CONNECT is meant for the proxy/tunnel itself. On the other hand not all proxies implement CONNECT.

Member 8353523 28-Feb-15 15:32pm

I made a the proxy myself through TCP communication. So when you for example run chrome under the proxy (127.0.0.1, 500) it connects to it and my proxy gets the browser requests and streams back internet by decoding the TCP stream and DeSerializing it. So when I have "GET www.Yahoo.com HTTP...", I separate the GET and the URL and pass it to the HttpWebRequest. But sometime the browser requests `CONNECT us.yahoo.com:443 ...` so these cannot be handled by the `HttpWebRequest` and they throw exceptions and website does not load.

Zoltán Zörgő 28-Feb-15 16:10pm

No, because it is intended to be porocessed by the proxy itself. Please read this RFC: https://www.ietf.org/rfc/rfc2817.txt. You need to provide the tunneling with your proxy. So you don't forward the CONNECT to the third party.

Zoltán Zörgő 28-Feb-15 16:24pm

This c# proxy implementation might be also interesting for you: http://www.mentalis.org/soft/projects/proxy/

Member 8353523 28-Feb-15 16:24pm

I tried this also but still get "Connection forcably closed" error

Socket MySocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Unspecified);
MySocket.Connect("us.yahoo.com", 443);
var i = MySocket.Connected;
if (i)
{
request.act();
MySocket.Send(System.Text.Encoding.Default.GetBytes("CONNECT us.yahoo.com:443 HTTP/1.0\r"));
Byte[] _Data = new Byte[2048];
MySocket.Receive(_Data);
return new System.IO.MemoryStream(_Data);
}

Zoltán Zörgő 28-Feb-15 16:29pm

No, you should not send CONNECT to yahoo. "You" as proxy should tunnel between the client and yahoo server.

Member 8353523 28-Feb-15 16:28pm

I just need to fix my own problem, the link you send is a completely different solution, I don't wanna copy past and I'm not interested in that. I just wanna know how deal with CONNECT Http requests that's it.

Zoltán Zörgő 28-Feb-15 16:31pm

Well, you don't need to copy paste. But you should understand what you have to do when client send CONNECT. You either read RFC carefully, or you check how others have done this. It is up to you. So far your code is not doing what it should. You can't fix your code as long as you are not aware what it should do. And that is preciselly defined in the RFC.

Member 8353523 28-Feb-15 16:32pm

"No, you should not send CONNECT to yahoo. "You" as proxy should tunnel between the client and yahoo server." This is very very TRUE Zoltan. and this the problem. I just need to know the approach in a very abstract level. Like at least what TYPE/CLASS to use.That is what I don't know and I can't find

Zoltán Zörgő 28-Feb-15 16:39pm

I still suggest you to consult the code I have linked. You need to use sockets at this level. But to have a proper knowledge about what exactly you need to take into account you have to read rfc2817.

Member 8353523 28-Feb-15 16:45pm

That does not fit in my solution, I'm dealing with the most advanced content filtering system in the world (Iranian Content Filtering, you can google about it). I already designed the solution, we have been able to bypass the filter completely, but have not been able to redirect the traffic to our solution. The proxy if for redirecting the traffic and it will be used locally. Your code is incompatible with what we are doing and I can't dispose the solution publicly. If I have to use Socket that is all I need but if Socket is not the right move please let me know.
*** PPTP, L2TP are fully blocked, UDP limited on high traffic, SSTP highly limited, SOCKS protocols fully blocked, OpenSSL working with limitation, other tunneling protocols highly limited

Zoltán Zörgő 28-Feb-15 16:49pm

Your choice.
You need sockets. At this level, you have to tailor http headers (of course part of the original one is passed), thus you can't rely on any upper level class (HttpClient even less HttpWebRequest).
But if you expect your solution to work, you have to follow the RFC. As browsers will.

Member 8353523 28-Feb-15 16:50pm

Zoltan please check updated info the above comment (after **), the only working protocol is actually TCP

Zoltán Zörgő 28-Feb-15 17:17pm

HTTP/HTTPS is working over TCP so you actually use TCP sockets. Still, the protocols you have listed are not on the same level of the TCP/IP or ISO/OSI stack, so if filtering is working on different layers, you might not get trough. Usign the yahoo example: the proxy will need to open a socket directly to the server and port after CONNECT. Simplest taken it needs to send everything after line that trough the socket - and it needs to send back what it gets. After that the communication between the server and the client should be transparent. You could also use SslStream.

On the other hand you might find this approach interesting: http://www.codeproject.com/Articles/93301/Implementing-a-Multithreaded-HTTP-HTTPS-Debugging.

Still, in this case the type of the communication can be detected as it will be standard one after leaving the proxy:
USER -> PROXY -> FILTER -> SERVER
In this case it should be more complex and not being standard at all. But to do that, you need an other party outside.
USER -> PROXY/OBFUSCATOR -> FILTER -> DEOBFUSCATOR -> SERVER