Im hoping someone can clear up a few issues as I have been at this for quite some time with conflicting results. I have a minifilter driver coded in c++ that monitors process creation and I am using FltSendMessage with communication port and I can confirm that the value is being sent correctly from the driver. The issues I am having is when I PInvoke FilterGetMessage and try to convert the intptr back to string to get the output to the VB.NET application. I am receiving nothing at all or getting large amounts of gibberish, Here is the vb.net code if anyone can help see what is going wrong I would greatly appricate it.
<DllImport("fltlib.dll")>
Shared Function FilterConnectCommunicationPort(
<MarshalAs(UnmanagedType.LPWStr)>portName As String,
options As UInteger,
context As IntPtr,
sizeOfContext As Short,
securityAttributes As IntPtr,
<Out> ByRef portHandle As SafeFileHandle) As UInteger
End Function
<DllImport("fltlib.dll")>
Shared Function FilterGetMessage(
portHandle As SafeFileHandle,
messageBuffer As IntPtr,
messageBufferSize As Integer,
<[In], [Out]> ByRef overlapped As intptr) As Boolean
End Function
<StructLayout(LayoutKind.Sequential)>
Public Structure FILTER_MESSAGE_HEADER
Public ReplyLength As UInt32
Public MessageId As UInt64
End Structure
I am calling FilterGetMessage From within a thread using a while loop as so
Dim FltSize = ((1 << 11) - 1) * 2 + 2 '4096 Same as buffer size in driver
Dim FltMessage As IntPtr = Marshal.AllocHGlobal(FltSize)
do while true
If FilterGetMessage(Communication_Port_Handle, FltMessage, FltSize, intptr.Zero) = True Then
Dim FltPointer = Marshal.PtrToStringUni(FltMessage)
Invoke(Sub() RichTextBox1.AppendText("FILTERGETMESSAGE: " & FltPointer))
Invoke(Sub() RichTextBox1.AppendText(vbCrLf))
End If
loop
I have set up a test string within the driver to report to debugview if FltSendMessage is successfull or not and there appears to be no issue with it there and also FilterGetMessage returns true in the vb.net application as there is a valid communication port handle when it connects. So I am lost now as to why I am getting nothing at all and at other times when I run the program I get gibberish.
The driver is sending the file name as wchar_t and after doing some research I found that the vb.net equivlant is of type string I also have this struct
<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Ansi)>
Public Structure node
<MarshalAs(UnmanagedType.ByValTStr, SizeConst:=BUFFER_SIZE)>
Public y As String
End Structure
Im not sure if this struct will even be of any use but thought to add it in this question if it may be of any help.
What I have tried:
I have tried using c++/cli with no luck other than returning integers and const litteral strings.
I have tried encoding with unicode, ANSI and other
Also I have checked and re checked the buffer size being sent and allocated