Does wsarecv wsabuf need to be page-aligned?

Question

0.00/5 (No votes)

See more:

Calling WSARecv with an address of WSABUF.buf+TotalReceivedBytes will have very weird results. I use this method to overcome buffer copying.

TotalReceivedBytes is just a per io variable to hold each user totalbytes

received through GetQueuedCompletionStatus then each time i add lpNumberOfBytes

to my defined variable. But when this happens and the TotalReceivedBytes is not DWORD aligned for example 0x1015 returned from GetQueuedCompletionStatus

function this causes the GetQueuedCompletionStatus to stop returning and sleeps forever and even the close socket events wont wake it up.

This had led me to memory leaking as the buffers and per IO resources never freed up. But it works well if the TotalReceivedBytes were 0x2000 but i can not control how many to
receive i used WSARecv with 0x2000 buffer size but it just returns data received 0x1015 on the first call.

So how do i overcome this problem? should i always reallocate buffer using heap functions? but this makes the app slow. What is the solution to this problem?

What I have tried:

Tried to use timeout value on

GetQueuedCompletionStatus

but it returns with 0 GetLastError reports WAIT_TIMEOUT. And

lpNumberOfBytes

returns 0 as well which skips my WSARecv call and so the data stuck there never polled out.

using dynamic buffer reallocation each time through HeapRealloc works fine but this has too much overhead when too many WSARecv called.

Posted 8-Oct-17 7:18am

NoviceCoder87

Updated 8-Oct-17 21:24pm

CPallini

v4

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Jochen Arndt · Accepted Answer · 2017-10-08T21:24:00

Solution 1

Please check the error code (again). 0x1015 is not a defined error.

Note that you must also change WSABUF.len accordingly when changing WSABUF.buf.

But I would not do so. The intention of such buffers is what the name implies: Buffering. If you expect large amounts of data blocks, use small buffers with fixed size and copy (append) the data to the final destination upon each completion. Copying such small memory blocks is fast and does not add much overhead.

Posted 8-Oct-17 21:24pm

Jochen Arndt

Comments

NoviceCoder87 9-Oct-17 12:43pm

Thank you. 0x1015 is not an error code it is lpNumberOfBytes returned by GetQueuedCompletionStatus. So actually it didnt read 0x2000 bytes first. Data got chunked. And i was also updating len but it's not a stable method.

Now i have made it allocate each time a receive is called through HeapReallocate API but after reading your answer i will make it fixed bufer then append to final received buffer. Can you suggest how many bytes should i receive for fast transfer? or the number of requested bytes to be received doesn't effect transfer speed?

Now i have one more question that i am really don't know how to handle.

I use a code to identify each IO Completion by setting this code to example OP_WRITE for sending. OP_READ for reading. This is all set in a structure of Per IO data.

In the first WSARecv to initiate IO Reading. I set this code to OP_READ
and upon returning from GetQueuedCompletionStatus i extract this key to know what is the current completed operation using switch case. But my question is what to do in case of sending and receiving at the same time? is the returning from GetQueuedCompletionStatus sequenced? I mean if i call WSASend and there were more data to be received is the next return IO Completion from GetQueuedCompletionStatus must be sending? or it just happens randomly?

How do i send and receive at the same time? this is confusing me. How to handle this situation after GetQueuedCompletionStatus returns?. Should i create more socket from the other client to use or 1 socket is enough for both sending and receiving at the same time?

Jochen Arndt 10-Oct-17 3:24am

This can't be answered all in a comment section and requires more knowledge about the requirements (number of packets / clients, package frequency, average size of packets).

Normally you would have a header that specifies the size of the following data so that you can allocate it. To avoid multiple allocations, you can reuse existing buffers (don't (re-)allocate if existing buffer is large enough). This may be finally some kind of memory pool.

You are handling asynchronous events. So you don't get sequenced results. Which kind of data let the call return are specified in the output parameters. From these you must be able to detect which operation completed and what to do next.

With TCP, you usually use one socket for sending and receiving. Sending and receiving at the same time (even initiated from different threads) is handled by the OS when not using blocking IO.

You will have overlapped structs for send and receive. When using WSAWaitForMultipleEvents(), the return value specifies which event occured. With GetQueuedCompletionStatus() you might use the event handle of the overlapped struct.

NoviceCoder87 13-Oct-17 2:04am

Thank you for your help. I meant by multiple sockets is that if i want to send multiple files from client to server then the server has to create a dedicated socket for each file? or just use 1 socket for all operations of send? and when i add switch case after the call of GetQueuedCompletionStatus returns i use OP_WRITE , OP_READ to identify socket event. Now i don't understand how i am going to send and recv at the same time because if i am receiving i am setting this OP code to OP_READ then let's say i want to send something while that data read is still in progress so i set this OP code to OP_WRITE now what happens here is that the next OVERLAPPED structure returned from GetQueuedCompletionStatus call will have OP_WRITE code set so what will happen with the pending data that i want to receive? switch case only execute 1 block of statement so it's either read or write how to do both in switch? or i need to add this value OP_XXXX constants like OP_WRITE + OP_READ? I am not freeing OVERLAPPED structure on all IO Operations unless the client closed the connection.

Jochen Arndt 13-Oct-17 2:54am

A socket is connection related. If you have an existing connection you will usually not open another one but use that to transfer for example multiple files.

You have to create two overlapped structures: One for sending and one for receiving.

NoviceCoder87 13-Oct-17 20:06pm

Thank you for your advice. I created 2 overlapped structures now. Do i need to use Critical sections? i mean in buffer copying and totalbytesreceived modify memory values should i put them inside EnterCriticalSection LeaveCriticalSection? or its not needed? where should i use Critical section in IOCP?

Jochen Arndt 14-Oct-17 4:19am

There is no need to lock the recv buffer because you copy the the data from the buffer before calling recv again.

You might need to lock when passing data from a receive thread to the main thread of your application.

NoviceCoder87 12-Nov-17 22:06pm

Jochen thanks for your help. Can you please check my other question and provide some info if you have some ideas that can help? i have been struggling for days without success.
https://www.codeproject.com/Questions/1214871/IOCP-sharing-completion-key-instances-between-each

Jochen Arndt 13-Nov-17 3:54am

I can't really help. Therefore, I will anser here.

It seems that you need to create and store some kind of unique ID for each user. This might be part of the used protocol (e.g. upon first connect / login you send that ID back to the client and he has to provide it with all further packets).

Never use SendMessage() to pass data from one thread to the the main (GUI) thread. It is not because it would block, but because it might crash your application (concurrent access to window data). Instead of SendNotifyMessage () you might also use PostMessage().

NoviceCoder87 13-Nov-17 13:07pm

Yes it's what i called unique connection identifier or key that will be used in all connections to differentiate the user. I think this is not good to use multiple sockets for same user as you can see it gives more complexity. So i think i will do only 1 socket instead and use multiple instances of per io data instead of multiple instances of context key. So this way i will have 1 socket for all operations but multiple buffers of per io data and de allocate them at each finish and allocate new at each start opf operation.
Thank you for your help. Which is very informative.