Http WebRequest with relative url

Question

0.00/5 (No votes)

See more:

Hi,
does anyone know whether there is anyway to get around following limitation? :

{"Invalid URI: The format of the URI could not be determined."}

...when trying to re-create http GET request like this:

C#

private static DataTable GetWebpage(string requestURL, string pathname)
{
var webRequest = WebRequest.Create(requestURL);
webRequest.Method = "GET";
webRequest.Proxy = new WebProxy(proxy, proxyPort);
// header parameters settings

StreamReader streamIn = 
           new StreamReader(webRequest.GetResponse().GetResponseStream());                     
string response = streamIn.ReadToEnd();
streamIn.Close();
 
//response parsing and data storage...
}

The trick is that destination server won't accept absolute address and redirects me to error page. I need Http header in following format, but WebRequest won't let me(the same problem is with System.Net.WebClient):

VB

- Http:
    Command: GET
  - URI: requestUrl(with params)
     Location: requestUrl(without params)
   + Parameters: 0x1
    ProtocolVersion: HTTP/1.1
// header parameters settings
    Host:  proxy:proxyPort
    Connection:  Keep-Alive
    HeaderEnd: CRLF

Thanks, for any suggestion.

EDIT:
Found out interesting thing, when I try to getResponse through fiddler's local http proxy, request's http header is actualy fixed (base part of url is removed).

NOTE: I'm sniffing outgoing traffic with 'Microsoft Network Monitor' utility.

Posted 30-Jan-12 2:47am

lynkx.vavri

Updated 30-Jan-12 22:21pm

v2

Add a Solution

Comments

lynkx.vavri 30-Jan-12 8:59am

Correction: webRequest should be of HttpWebRequest type:

var webRequest = (HttpWebRequest)WebRequest.Create(requestURL);

Sergey Alexandrovich Kryukov 30-Jan-12 14:17pm

You are basically right, but OP does not use any methods of HttpWebRequest, so in this simple case this is just fine.
--SA

3 solutions

Solution 1

This is because you are using System.Net.WebRequest.Create(string) where you rather need System.Net.WebRequest.Create(System.Uri).

You are dealing with invalid URI created by default from string, because URL is relative, but by default is is assumed it was absolute. Construct your Uri from string explicitly indicating System.UriKind.Relative:

C#

var webRequest =
    System.Net.WebRequest.Create(new System.Uri(
          requestUrl,
          System.UriKind.Relative //it should solve the problem
    ));

Please see:
http://msdn.microsoft.com/en-us/library/system.net.webrequest.aspx[^],
http://msdn.microsoft.com/en-us/library/0aa3d588.aspx[^],
http://msdn.microsoft.com/en-us/library/system.uri.aspx[^],
http://msdn.microsoft.com/en-us/library/ms131565.aspx[^],
http://msdn.microsoft.com/en-us/library/system.urikind.aspx[^].

—SA

Posted 30-Jan-12 8:26am

Sergey Alexandrovich Kryukov

Comments

NeptuneHACK! 30-Jan-12 16:57pm

my 5 ;D

Sergey Alexandrovich Kryukov 30-Jan-12 17:05pm

Thank you.
--SA

lynkx.vavri 31-Jan-12 3:55am

Thanks, but System.Net.WebRequest.Create method doesn't support relative URIs:

{"This operation is not supported for a relative URI."}

I also tryied to create requestUrl using 'baseUrl' and 'relativeUrl'
new Uri(baseUri, relativeUri);
, hoping to tell webRequest to use only relative part later on, but webRequest.RequestUri.IsAbsoluteUri is readonly property. Kinda don't don't understand why is it there.

Solution 2

A similar question is asked earlier (for silver light, but you still use the stated steps). These discussions may add up more :

How to Navigate to A page outside Project in Silverlight[^]

Posted 30-Jan-12 8:38am

Lakamraju Raghuram

Comments

lynkx.vavri 31-Jan-12 4:11am

My app is system service used for automated data download with database import. I don't think that System.Windows.Navigation lib is exactly what I need, but I'll take a look at it just for getting out of the box. Thanks.

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

lynkx.vavri · Accepted Answer · 2012-01-31T21:19:00

Ok, in order for generated http header not to look like proxy request I needed to tell to
connection management class that it should not connect to set up proxy:

C#

private static DataTable GetWebpage(string requestURL, string pathname)
{
var webRequest = WebRequest.Create(requestURL);
webRequest.Method = "GET";
webRequest.Proxy = new WebProxy(proxy, proxyPort);

FieldInfo field_ServicePoint_ProxyServicePoint = (typeof(ServicePoint))
.GetField("m_ProxyServicePoint", BindingFlags.NonPublic | BindingFlags.Instance);
field_ServicePoint_ProxyServicePoint.SetValue(webRequest.ServicePoint, false);

// header parameters settings

StreamReader streamIn = 
           new StreamReader(webRequest.GetResponse().GetResponseStream());                     
string response = streamIn.ReadToEnd();
streamIn.Close();
 
//response parsing and data storage...
}

I also had to enable unsafeHeaderparsing because target server violated
header format in response and I got this exception:

{"The server committed a protocol violation. Section=ResponseStatusLine"}

I found solution for programatically solving this on msdn forum[^]

Http WebRequest with relative url

3 solutions

Solution 1

Solution 3

Solution 2

Add your solution here

Preview 0

Existing Members

...or Join us