C# Discussion Boards - CodeProject

Re: Trouble handling DeviceIoControl to retrieve USN data

3-Jan-15 0:46

I am having trouble with DeviceIoControl() and control code FSCTL_ENUM_USN_DATA and I am also having difficulties understanding the documentation for FSCTL_ENUM_USN_DATA.

I created the structs for MFT_ENUM_DATA and USN_RECORD_V3 and passed them as I understood the documentation, as pointers.

However, the first and only USN record I get in return (the values of the USN_RECORD_V3 buffer) is only garbage. Also, the outBufferSize should be 555 (explained at the doc for USN_RECORD_V3), but if I exceed 175 bytes, I get a System.AccessViolationError.

How to calculate outBufferSize:

C++

MaximumChangeJournalRecordSize =
    ( (MaximumComponentLength - 1) * sizeof(WCHAR)
    + sizeof(USN_RECORD_V3) );

I would appreciate if you could help me finding the mistake in my code or in my thought process. In advance, a lot of thanks to you guys!

Edit: Problematic code starts below comment:

// ENUM USN DATA

Program.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Runtime.InteropServices;
using Microsoft.Win32.SafeHandles;
using FILETIME = System.Runtime.InteropServices.ComTypes.FILETIME;

public class Win32
{
    /// <summary>
    /// Represents an update sequence number (USN) change journal, its records, and its capacity.
    /// This structure is the output buffer for the FSCTL_QUERY_USN_JOURNAL control code.
    /// Minimum supported client: Windows XP [desktop apps only]
    /// Minimum supported server: Windows Server 2003 [desktop apps only]
    /// </summary>
    /// <see cref="http://msdn.microsoft.com/en-us/library/windows/desktop/aa365721%28v=vs.85%29.aspx"/>
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public struct USN_JOURNAL_DATA_V0
    {
        public UInt64 UsnJournalId;
        public Int64 FirstUsn;
        public Int64 NextUsn;
        public Int64 LowestValidUsn;
        public Int64 MaxUsn;
        public UInt64 MaximumSize;
        public UInt64 AllocationDelta;
    }

    /// <summary>
    /// Represents an update sequence number (USN) change journal, its records, and its capacity.
    /// This structure is the output buffer for the FSCTL_QUERY_USN_JOURNAL control code.
    /// Minimum supported client: Windows 8 [desktop apps only]
    /// Minimum supported server: Windows Server 2012 [desktop apps only]
    /// </summary>
    /// <see cref="http://msdn.microsoft.com/en-us/library/windows/desktop/hh802707%28v=vs.85%29.aspx"/>
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public struct USN_JOURNAL_DATA_V1
    {
        public UInt64 UsnJournalId;
        public Int64 FirstUsn;
        public Int64 NextUsn;
        public Int64 LowestValidUsn;
        public Int64 MaxUsn;
        public UInt64 MaximumSize;
        public UInt64 AllocationDelta;
        public UInt16 MinSupportedMajorVersion;
        public UInt16 MaxSupportedMajorVersion;
    }

    /// <summary>
    /// Contains the information for an update sequence number (USN) change journal version 3.0 record. The version 2.0 record is defined by the USN_RECORD_V2 structure (also called USN_RECORD structure).
    /// </summary>
    /// <see cref="http://msdn.microsoft.com/en-us/library/windows/desktop/hh802708%28v=vs.85%29.aspx"/>
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public struct USN_RECORD_V3
    {
        public UInt32 RecordLength;
        public UInt16 MajorVersion;
        public UInt16 MinorVersion;
        public Byte FileReferenceNumber;
        public Byte ParentFileReferenceNumber;
        public Int64 Usn;
        public FILETIME TimeStamp;
        public UInt32 Reason;
        public UInt32 SourceInfo;
        public UInt32 SecurityId;
        public UInt32 FileAttributes;
        public UInt16 FileNameLength;
        public UInt16 FileNameOffset;
        public Char FileName;
    }

    /// <summary>
    /// Contains information defining the boundaries for and starting place of an enumeration of update sequence number (USN) change journal records for ReFS volumes. It is used as the input buffer for the FSCTL_ENUM_USN_DATA control code.
    /// </summary>
    /// <see cref="http://msdn.microsoft.com/en-us/library/windows/desktop/hh802704%28v=vs.85%29.aspx"/>
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public struct MFT_ENUM_DATA_V1
    {
        public UInt64 StartFileReferenceNumber;
        public Int64 LowUsn;
        public Int64 HighUsn;
        public UInt16 MinMajorVersion;
        public UInt16 MaxMajorVersion;
    }

    /// <summary>
    /// Sends the 'dwIoControlCode' to the device specified by 'hDevice'.
    /// </summary>
    /// <param name="hDevice">IntPtr handle to handle to a file, directory, or device to receive 'dwIoControlCode'. Obtained by calling CreateFile.</param>
    /// <param name="dwIoControlCode">Device IO Control Code to send</param>
    /// <param name="lpInBuffer">Input buffer if required</param>
    /// <param name="nInBufferSize">Size of input buffer</param>
    /// <param name="lpOutBuffer">Output buffer if required</param>
    /// <param name="nOutBufferSize">Size of output buffer</param>
    /// <param name="lpBytesReturned">Number of bytes returned in output buffer</param>
    /// <param name="lpOverlapped">IntPtr to an 'OVERLAPPED' structure</param>
    /// <returns>'true' if successful, otherwise 'false'</returns>
    /// <see cref="http://msdn.microsoft.com/en-us/library/windows/desktop/aa363216%28v=vs.85%29.aspx"/>
    [DllImport("kernel32.dll", ExactSpelling = true, SetLastError = true, CharSet = CharSet.Auto)]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool DeviceIoControl(
        SafeHandle hDevice,
        UInt32 dwIoControlCode,
        IntPtr lpInBuffer,
        UInt32 nInBufferSize,
        out IntPtr lpOutBuffer,
        UInt32 nOutBufferSize,
        out UInt32 lpBytesReturned,
        IntPtr lpOverlapped
    );

    [DllImport("kernel32.dll", ExactSpelling = true, SetLastError = true, CharSet = CharSet.Auto)]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool DeviceIoControl(
        SafeHandle hDevice,
        UInt32 dwIoControlCode,
        IntPtr lpInBuffer,
        UInt32 nInBufferSize,
        out USN_JOURNAL_DATA_V1 lpOutBuffer,
        UInt32 nOutBufferSize,
        out UInt32 lpBytesReturned,
        IntPtr lpOverlapped
    );

    [DllImport("kernel32.dll", ExactSpelling = true, SetLastError = true, CharSet = CharSet.Auto)]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool DeviceIoControl(
        SafeHandle hDevice,
        UInt32 dwIoControlCode,
        ref MFT_ENUM_DATA_V1 lpInBuffer,
        UInt32 nInBufferSize,
        out USN_RECORD_V3 lpOutBuffer,
        UInt32 nOutBufferSize,
        out UInt32 lpBytesReturned,
        IntPtr lpOverlapped
    );

    // Use interop to call the CreateFile function. 
    // For more information about CreateFile, 
    // see the unmanaged MSDN reference library.
    [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
    public static extern SafeFileHandle CreateFile(
        string lpFileName,
        uint dwDesiredAccess,
        uint dwShareMode,
        IntPtr lpSecurityAttributes,
        uint dwCreationDisposition,
        uint dwFlagsAndAttributes,
        IntPtr hTemplateFile
    );

    /// <summary>
    /// Sets the number of bytes specified by 'size' of the memory associated with the argument 'ptr' 
    /// to zero.
    /// </summary>
    /// <param name="ptr"></param>
    /// <param name="size"></param>
    [DllImport("kernel32.dll")]
    public static extern void ZeroMemory(IntPtr ptr, int size);
}

namespace UsnTest01
{

    class Program
    {
        #region constants
        public const short FILE_ATTRIBUTE_NORMAL = 0x80;
        public const short INVALID_HANDLE_VALUE = -1;
        public const UInt32 GENERIC_READ = 0x80000000;
        public const UInt32 GENERIC_WRITE = 0x40000000;
        public const UInt32 FILE_SHARE_READ = 0x00000001;
        public const UInt32 FILE_SHARE_WRITE = 0x00000002;
        public const UInt32 CREATE_NEW = 1;
        public const UInt32 CREATE_ALWAYS = 2;
        public const UInt32 OPEN_EXISTING = 3;

        private const UInt32 FILE_DEVICE_FILE_SYSTEM = 0x00000009;
        private const UInt32 METHOD_NEITHER = 3;
        private const UInt32 METHOD_BUFFERED = 0;
        private const UInt32 FILE_ANY_ACCESS = 0;
        private const UInt32 FILE_SPECIAL_ACCESS = 0;
        private const UInt32 FILE_READ_ACCESS = 1;
        private const UInt32 FILE_WRITE_ACCESS = 2;
        public const UInt32 FSCTL_QUERY_USN_JOURNAL = (FILE_DEVICE_FILE_SYSTEM << 16) | (FILE_ANY_ACCESS << 14) | (61 << 2) | METHOD_BUFFERED;
        public const UInt32 FSCTL_ENUM_USN_DATA = (FILE_DEVICE_FILE_SYSTEM << 16) | (FILE_ANY_ACCESS << 14) | (44 << 2) | METHOD_NEITHER;
        public const UInt32 FSCTL_READ_USN_JOURNAL = (FILE_DEVICE_FILE_SYSTEM << 16) | (FILE_ANY_ACCESS << 14) | (46 << 2) | METHOD_NEITHER;
        #endregion constants

        static void Main(string[] args)
        {

            // GET VOLUME HANDLE

            SafeFileHandle handleValue = null;
            handleValue = Win32.CreateFile(
                @"\\.\C:",
                GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE,
                IntPtr.Zero,
                OPEN_EXISTING,
                0,
                IntPtr.Zero);

            if (handleValue.IsInvalid)
            {
                Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
            }

            // GET USN JOURNAL ID

            Win32.USN_JOURNAL_DATA_V1 ujd = new Win32.USN_JOURNAL_DATA_V1();
            uint lpBytesReturned;

            bool retJournalId = Win32.DeviceIoControl(
                handleValue,
                FSCTL_QUERY_USN_JOURNAL,
                IntPtr.Zero,
                0,
                out ujd,
                (uint)Marshal.SizeOf(ujd),
                out lpBytesReturned,
                IntPtr.Zero);

            Console.WriteLine("Wert ist " + ujd.UsnJournalId.ToString("X4"));



            // ENUM USN DATA

            Win32.USN_RECORD_V3 usnRecordV3;
            Win32.MFT_ENUM_DATA_V1 mftEnumDataV1 = new Win32.MFT_ENUM_DATA_V1 { StartFileReferenceNumber = 0 };

            //int mftEnumDataV1BufferSize = Marshal.SizeOf(mftEnumDataV1);
            //IntPtr mftEnumDataV1BufferPointer = Marshal.AllocHGlobal(mftEnumDataV1BufferSize);
            //Marshal.StructureToPtr(mftEnumDataV1, mftEnumDataV1BufferPointer, true);
            //Win32.ZeroMemory(mftEnumDataV1BufferPointer, mftEnumDataV1BufferSize);

            // TODO "255-1" durch "MaximumComponentLength-1" ersetzen
            uint outBufferSize = (uint)((255 - 1) * sizeof(Char) + Marshal.SizeOf(typeof(Win32.USN_RECORD_V3)));
            lpBytesReturned = 0;

            bool retUsnData = false;
            while (retUsnData != true) // while non-zero value
            {
                retUsnData = Win32.DeviceIoControl(
                    handleValue,
                    FSCTL_ENUM_USN_DATA,
                    ref mftEnumDataV1,
                    (uint)Marshal.SizeOf(mftEnumDataV1),
                    out usnRecordV3,
                    175, //max without exception is 175. Should be var `outBufferSize`.
                    out lpBytesReturned,
                    IntPtr.Zero);

                //mftEnumDataV1.StartFileReferenceNumber = usnRecordV3.RecordLength;
                //object test = Marshal.PtrToStructure(mftEnumDataV1BufferPointer, typeof(Win32.MFT_ENUM_DATA_V1));
                //Marshal.WriteInt64(mftEnumDataV1BufferPointer, usnRecordV3.RecordLength);
                //Marshal.StructureToPtr(mftEnumDataV1, mftEnumDataV1BufferPointer, true);

            }

            int r1 = Marshal.GetLastWin32Error();
            int r2 = Marshal.GetHRForLastWin32Error();

            if (retUsnData == false)
            {
                Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
            }

        }
    }
}

Richard MacCutchan3-Jan-15 0:57

Re: Trouble handling DeviceIoControl to retrieve USN data

3-Jan-15 0:57

The documentation for the output buffer states:

Mandelnuss wrote:

lpOutBuffer
A pointer to the output buffer that receives a USN followed by zero or more USN_RECORD_V2 or USN_RECORD_V3 structures.
The USN is a DWORDLONG value that represents the USN following the last record in the buffer. It can be used in a subsequent call to FSCTL_ENUM_USN_DATA to get the next set of records following those in the current output buffer.

Whereas your code is expecting just a USN_RECORD_V3. You need to add the extra space at the beginning.

Mandelnuss3-Jan-15 1:18

Re: Trouble handling DeviceIoControl to retrieve USN data

3-Jan-15 1:18

Wow, okay. I did not quite understand that part, thank you. So what I am getting is one USN and as many complete USN records as fit in the buffer? So using a struct is the wrong approach, right?

What variable type would I point to in this case? A byte-array? Or is there something better?

Richard MacCutchan3-Jan-15 1:50

Re: Trouble handling DeviceIoControl to retrieve USN data

3-Jan-15 1:50

The item at the beginning is a DWORDLONG (64 bits)[^], so you need to add that to the beginning of your structure definition. You also need to expand the structure at the end to allow for variable data in the final item. In a Win32 structure definition, when you see something like:

C++

...
  WORD          FileNameOffset;
  WCHAR         FileName[1];
} USN_RECORD_V3, *PUSN_RECORD_V3;

the FileName (last) field is indicating that it is actually of unknown length, rather than a single character, so you should reserve as much space as is expected for your results. It is better to have more space in your structure than you need, but as long as it is at least the same size as indicated in the bufferlength variable, it should work.

Mandelnuss3-Jan-15 3:53

Re: Trouble handling DeviceIoControl to retrieve USN data

3-Jan-15 3:53

Thank you so much for your help so far. I could do much progress based on your replies!

I understand that the buffer will contain the DWORDLONG, the USN_RECORD_V3 and a variable sized string (FileNameLength bytes minus the first char) in this order.

The call will return more than one USN record. While I would love to use something like this

SQL

public struct OUTBUFF{
   public ulong usn; // 8 byte
   public USN_RECORD_V3 usnRecordV3; // 47 byte
   [MarshalAs(UnmanagedType.ByValArray, SizeConst = 508)]
   public byte[] fileName; // max 508
}

succeeding records would be written in OUTBUFF.fileName. A solution would be to just have a byte array for all returned USN records

XML

<pre lang="sql">public struct OUTBUFF{
   public ulong usn; // 8 byte
   [MarshalAs(UnmanagedType.ByValArray, SizeConst = 555)]
   public byte[] recordData; // max 508+47
}</pre>

and then manually split the OUTBUFF.recordData at the offsets calculated by USN_RECORD_V3.RecordLength

While this would not be difficult to write, do you maybe know of a better way to handle this? I am still new to the matter and keen on learning good ways of working with that stuff.

Richard MacCutchan3-Jan-15 4:46

Re: Trouble handling DeviceIoControl to retrieve USN data

3-Jan-15 4:46

Mandelnuss wrote:
and then manually split the OUTBUFF.recordData at the offsets calculated

That is the general way to do it, even in unmanaged code (C++), so you are definitely on the right track. Thumbs Up | :thumbsup:

Mandelnuss3-Jan-15 5:20

3-Jan-15 5:20

Thanks for the help Richard!

Logic of prime numbers

Member 113371082-Jan-15 12:44

Member 11337108

2-Jan-15 12:44

if (value % 2 == 0) { x++; break; }
if (value % i == 0) { x++; break; }

Heres my problem with this. I am not understanding the logic and I need help with this. I would really appreciate your feedback.

Just as an example.
suppose i input value=5
i is going to check 1,3,5.
so 5%1= Remainder 0
5%3= Remainder 2
5%5= Remainder 0
Well since 5%3 has a remainder, doesnt this disqualify it? Obviously we all know that five is a prime number but what is the computer checking with the module??

Thank you guys so much and I hope you guys can teach me the logic behind this.

BillWoodruff2-Jan-15 13:14

2-Jan-15 13:14

The modulo operator "%" in .NET C# returns the integral remainder of a number divided by (modulo) the divisor.

Any number where the modulo result is zero only when the divisor is the same as the dividend is prime (leaving out the obvious fact that every number is divisible by one).

A way that is perhaps useful to visualize this is:

int dividend = 5;

int divisor = 2;

int quotient = 5 / 2; // returns 2 since integer division in C# discards the remainder

int remainder = 5 % 2; // return 1

bool realityCheck = dividend == (quotient + remainder);

«A man will be imprisoned in a room with a door that's unlocked and opens inwards ... as long as it does not occur to him to pull rather than push» Wittgenstein

modified 3-Jan-15 6:41am.

Richard MacCutchan2-Jan-15 22:03

2-Jan-15 22:03

BillWoodruff wrote:
int remainder = 5 % 2; // return 3

Really?

Pete O'Hanlon2-Jan-15 23:23

Pete O'Hanlon

2-Jan-15 23:23

If we redesign the modulo to actually mean subtract, then yes.

BillWoodruff3-Jan-15 0:42

3-Jan-15 0:42

Oh, my bad. I plead insomnia. Corrected. thanks, Bill

«A man will be imprisoned in a room with a door that's unlocked and opens inwards ... as long as it does not occur to him to pull rather than push» Wittgenstein

Richard MacCutchan3-Jan-15 0:47

3-Jan-15 0:47

I usually plead senility. Frown | :(

BillWoodruff3-Jan-15 11:57

3-Jan-15 11:57

In your case, "senility" I seriously doubt ! How many moons have you ? I have 881.

«A man will be imprisoned in a room with a door that's unlocked and opens inwards ... as long as it does not occur to him to pull rather than push» Wittgenstein

Richard MacCutchan3-Jan-15 21:05

3-Jan-15 21:05

Not sure whether you divide that by 12 or 13, but I'm a baby-boomer, born in '45.

BillWoodruff3-Jan-15 21:21

3-Jan-15 21:21

Then I am at least 15~16 lunes ahead of you, biologically, which is a small comfort given how much your intellect is so far ahead of mine.

«A man will be imprisoned in a room with a door that's unlocked and opens inwards ... as long as it does not occur to him to pull rather than push» Wittgenstein

Richard MacCutchan3-Jan-15 22:02

3-Jan-15 22:02

BillWoodruff wrote:
how much your intellect is so far ahead of mine.

Most unlikely. There are more things in heaven and I earth that I don't know, compared to the things that I do. And much of my 'intellect' is actually generated by Google.

SledgeHammer012-Jan-15 14:16

SledgeHammer01

2-Jan-15 14:16

value % whatever == 0 is checking for *no* remainder when divided by whatever.

Your algorithm should not be testing against 1 since every number is divisible by 1. There is a more optimized "standard" algorithm that increments by 6 on each iteration rather then 1. But that's about as efficient as you'll get. There isn't any efficient publicly known algorithm. Maybe the CIA spooks have one though LOL...

Being able to easily factor prime numbers would render all forms of cryptography useless.

Member 113371082-Jan-15 14:19

Member 11337108

2-Jan-15 14:19

int num1;

Console.WriteLine("Accept number:");
num1 = Convert.ToInt32(Console.ReadLine());
if (num1 == 0 || num1 == 1)
{
    Console.WriteLine(num1 + " is not prime number");
    Console.ReadLine();
}
else
{
    for (int a = 2; a <= num1 / 2; a++)
    {
        if (num1 % a == 0)
        {
            Console.WriteLine(num1 + " is not prime number");
            return;
        }

    }
    Console.WriteLine(num1 + " is a prime number");
    Console.ReadLine();
}

Heres my question.

looking at the if statement
if (num1%a==0)
num1 is not a prime. I am having a tough time understanding why that is.

SledgeHammer012-Jan-15 14:32

SledgeHammer01

2-Jan-15 14:32

Correct. If a number % anotherNumber == 0, (assuming anotherNumber is not 1 or the number itself), its not a prime.

6 % 3 = 0 means that 6 is divisible by 3, so 6 is not a prime.

Member 113371082-Jan-15 15:06

Member 11337108

2-Jan-15 15:06

Console.Write("Enter a Number : ");
           int num;
           num = Convert.ToInt32(Console.ReadLine());
           int k;
           k = 0;
           for (int i = 1; i <= num; i++)
           {
               if (num % i == 0)
               {
                   k++;
               }
           }
           if (k == 2)
           {
               Console.WriteLine("Entered Number is a Prime Number and the Largest Factor is {0}",num);
           }
           else
           {
               Console.WriteLine("Not a Prime Number");
           }
           Console.ReadLine();

you seem to be very smart. so heres one last question for you. In this situation, what does the k stand for? why is it that if k=2 then the number that was entered is prime?