Program blows up after unmanaged callback is called

Question

0.00/5 (No votes)

See more:

I have an c dll that I need to call from my c# application.
I have it communicating with the dll, but I am having issues with callbacks.

The c looks like this.

typedef void (CDECLCALL_CONV *ConnectedCB) (DPHandle hdl);

typedef struct
{
unsigned int size;	// sizeof(DPcbs)
ConnectedCB connectedCB;
.. more callbacks
} DPcbs;

BPS_DP_API unsigned long CDECLCALL_CONV DPSetCallBacks(DPHandle hdl, DPcbs *DPcbs);

What I have tried:

I have created a interface class that wraps the unmanaged code.
It looks like this

  [System.Runtime.InteropServices.UnmanagedFunctionPointerAttribute(System.Runtime.InteropServices.CallingConvention.Cdecl)]
  public delegate void ConnectedCB(ref DocProcHandle hdl);

  [System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
public struct DPcbs
{
  public uint size;
  public ConnectedCB connectedCB;
... more callbacks
}
[System.Runtime.InteropServices.DllImportAttribute("DocProc.dll", EntryPoint = "DPSetCallBacks", CallingConvention = System.Runtime.InteropServices.CallingConvention.Cdecl)]
    public static extern uint DPSetCallBacks(ref DocProcHandle hdl, ref DPcbs DPcbs);

Finally My c# calling code

{
  ...
  // this code is in a static class in the constructor
  // it should not be garbage collected right?
  m_events = new DPcbs();
  m_events.connectedCB = connected;
  m_events.size = Convert.ToUInt32(Marshal.SizeOf<DPcbs>());

  wrapper.DPSetCallBacks(ref m_handle, ref m_events);
  ...
}
private void connected(ref DocProcHandle hdl)
{
  // do something here
}

The application "stopped working" - no error just a crash.
But it happens AFTER the connected method is called, and before anything else.
If I do not set the connected callback it does not crash. (it seems to crash if it tries to call any of my callbacks)

I have tried everything i can think of to make sure the method does not go through the GC.
The class is static, I have made the method static. I have Alloc'ed both the method and the event.

Is there anything I am missing or doing wrong?
Any suggestions on what to try?

Posted 1-Jun-18 11:41am

lewisv

Updated 1-Jun-18 15:26pm

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Richard Andrew x64 · Accepted Answer · 2018-06-01T15:28:00

Solution 1

You can't pass the delegate to unmanaged code the way you are trying to do. If you are going to pass it as a function pointer inside a structure, you should call:

C#

IntPtr NativeFunctionPointer = Marshal.GetFunctionPointerForDelegate(connected);

Then pass NativeFunctionPointer as the function pointer.

The reason is that a managed delegate consists of more than just a function address. It has code and data like many other .NET classes, so passing its address isn't the same as passing its function entry point.

Posted 1-Jun-18 15:28pm

Richard Andrew x64

Comments

lewisv 2-Jun-18 11:21am

That makes sense so here is what I changed it to.

Changed the declaration in the struct to

public struct DPcbs
{
  public uint size;
  public IntPtr connectedCB;
... more callbacks
}

The setting the struct i have (i made the items member variables so that the GC would not get rid of them)

      m_c = new ConnectedCB(connected);
      m_p = Marshal.GetFunctionPointerForDelegate(m_c);
      m_events.connectedCB = m_p;

It fires and runs, but the same issue. It blows up directly have the connected delegate finishes.

Richard Andrew x64 2-Jun-18 13:10pm

I'm confused by your terminology. You say "it fires and runs", then you say "it blows up", "delegate finishes".

Perhaps if you could be specific. Where exactly is the thread when it "blows up?" Is it before, during or after the callback function is executed?

Do you get an error message about the stack being unbalanced?

Have you stepped through the code with the debugger?

lewisv 2-Jun-18 16:19pm

I am sorry if it was a little confusing. I have run it though the debugger.

private void connected(ref DocProcHandle hdl)
{
  // this code works fine
}  
  <--- it crashes here after it leaves my callback method and before anything else in the c# application runs

There is no error message, the program just "stops responding" then asks me if I want to debug. No exits the debugger, Yes it will open another visual studio, but then tell me it can't debug.
If I leave the callback out, the application continues with no issues. But of course I need to handle the callbacks.

I am guessing here, but it looks like by setting/calling the callback function I am messing up memory. So the application crashes outside of the c# application and inside the dll that is calling the delegate.
I know the dll works, I have a c application that is calling it and working. To make sure I have not missed anything, I am basically rebuilding the c app in c# and calling the same methods.

Thanks

Richard Andrew x64 2-Jun-18 16:28pm

If it crashes after leaving the callback, that is a very strong indicator that the stack is corrupted on the native side. I would recommend making sure that CDECL is the right calling convention for calling into managed code.

lewisv 2-Jun-18 18:01pm

It has been 10 years since I did any real work in c/c++.
But it looks like cdecl to me.

#define CDECLCALL_CONV __cdecl

typedef void (CDECLCALL_CONV *ConnectedCB) (DPHandle hdl);
BPS_DP_API unsigned long CDECLCALL_CONV getDPDevices(DPHandle devices[], unsigned long* count);

What do you mean by cdecl being right for calling "into" managed code. Is there something else I need to do in my managed c# code?

Richard Andrew x64 2-Jun-18 18:30pm

I'm sorry. I failed to notice that you properly decorated the delegate with the CallingConvention.Cdecl attribute. I was supposing that the callback might have required the stdcall convention. Like you, it's been a while since I've done native/managed interop.

What does the code that calls the callback look like?

lewisv 5-Jun-18 12:15pm

I think I have it.
Thanks for the help.

So your first post was good, I think that was one of the problems.
The second issue was dealing with a the handle that is passed into each routine.

When I opened a connection I got an IntPtr back.
IntPtr m_ptr;
m_ptr = wrapper.Open(...)
I turned that into a struct
m_handle= Marshal.PtrToStructure<docprochandle>(m_ptrp);

I was passing that handle back into the dll.
wrapper.DPSetCallBacks(ref m_handle, ref m_events);

But the wrapper did not want a DocProcHandle, it wanted an IntPtr. When I replaced that, it went though the application just fine.
wrapper.DPSetCallBacks(m_ptr, ref m_events);

Maybe it worked fine with non-callbacks, but with the callbacks because it was placed on the stack again and messing the stack up like you mentioned.