CORS response working in IE 10 only, fails for chrome and firefox

Hello,

I am making a cross origin ajax HTTP request from 'http://localhost:8080/[some string value].html' to a remote server machine.Actually this request is made to get a list of HDFS file system information which is installed in the remote server. Request URL is:
 
var formURL =  http://[remote server name]:[port number]/**webhdfs**/v1/xd/?op=LISTSTATUS. 

 and the ajax call is like:
 $.ajax({
		url: formUrl,
		dataType: 'json',
		success: function(response)
 		{
            alert("Response is: "+response);
        }
});

response is expected a json array like below

{"FileStatuses":{"FileStatus":[
{"accessTime":1449652575549,"blockSize":134217728}]}}

While I am getting this response in IE 10, but for all other browsers it's  throwing a browser level error as:

"XMLHttpRequest cannot load http://[remote server name]:[port number]/**webhdfs**/v1/?op=LISTSTATUS. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8080' is therefore not allowed access."

I searched google, its solution is to write java servlet method to set response header as "Access-Control-Allow-Origin, http://localhost:8080".

As this ajax request is not meant for any server side java method as ajax request generally does, how can I get rid this issue?