C++ get request - unusual issue

Question

0.00/5 (No votes)

See more:

Hi,

I've written some code to perform a GET request and it's working to an extent, except it won't receive large amounts of data properly. It's strange because how I output the data appears to determine how much was received.

I own the server this is performing the GET request to, and I've set it to output the numbers 0 to 9999 with a PHP script.

Working:

C++

while (resp_len == BUFFER_SIZE)
{
	resp_len = recv(sock, (char*)&buffer, BUFFER_SIZE, 0);
		
	if (resp_len > 0)
	{
		string data = string(buffer).substr(0, resp_len);
		printf(data.c_str());
	}
}

Not working:

C++

while (resp_len == BUFFER_SIZE)
{
	resp_len = recv(sock, (char*)&buffer, BUFFER_SIZE, 0);
		
	if (resp_len > 0)
	{
		string data = string(buffer).substr(0, resp_len);
		response += data;
	}
}

printf(response.c_str());

With the working example, it returns 49,062 bytes consistently. With the failing one, only 8,372, and this is also consistent.

I need the entire response in a std::string to return from this function. Could you help me figure this out please?

What I have tried:

Multiple methods to perform GET requests, none of which work apart from this, and this is temperamental depending on how the data is outputted.

Posted 1-Jul-18 8:40am

User 13204940

Updated 1-Jul-18 11:53am

v3

Add a Solution

Comments

Rick York 1-Jul-18 16:18pm

What happens if you get a response of less than BUFFER_SIZE? Shouldn't it continue while it gets any data ie., resp_len > 0?

2 solutions

Solution 1

Your not working code is too fast (the code executes faster than the time required to receive BUFFER_SIZE bytes). The first code is only working because the printf() call requires some time for execution.

The recv() function returns the number of bytes actually available, up to the specified buffer size. If you call it again too fast, the return value will be less than BUFFER_SIZE.

You would have to change your code to receive data until a complete data set has been received. This can be done by an end marker (e.g. a null byte with string data), or by sending the number of the bytes in the data set at first. The latter is typically done by sending a defined header before the payload. The definition of such headers and the payload data format is called protocol.

Posted 1-Jul-18 10:47am

Jochen Arndt

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

User 13204940 · Accepted Answer · 2018-07-01T11:53:00

Fixed - when playing around with the code, I saw that some data at the end was repeating, and I realised it was because it was trying to fill up the buffer for the last chunk of data even though the remaining data wasn't large enough, so it re-read from the start of that chunk to fill up the buffer, and duplicated some data.

This version reads the data in chunks but only tries to assign what's available, not always BUFFER_SIZE.

C++

int nDataLength;

while ((nDataLength = recv(sock, (char*)&buffer, BUFFER_SIZE, 0)) > 0)
{
	for(int x = 0; x < nDataLength; x++)
	{
		response += buffer[x];
	}
}