How can I read a file into unmanaged memory using C#?

Question

0.00/5 (No votes)

See more:

I'm using Net 5 (C#).

I need to read a file into unmanaged memory (and write unmanaged memory to a file).

The only way I know is to use Windows API 'CreateFile' to get a handle to the file and then use ```ReadFile``` or ```WriteFile``` imported from ```Kernel32```.

This works directly in Windows.

It fails when running in a Linux Docker container under Windows 10 Pro.

It fails with:

Unable to load shared library 'kernel32' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libkernel32: cannot open shared object file: No such file or directory

How can I accomplish the read/write inside a Linux Docker container?

What I have tried:

Building a Docker image via:

FROM mcr.microsoft.com/dotnet/runtime:5.0 AS base

FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS base

None of the above worked.

Posted 2-Jun-21 16:02pm

david marcus

Updated 9-Jun-21 13:08pm

Add a Solution

Comments

Richard Deeming 3-Jun-21 4:05am

The unmanaged APIs for Windows and Linux are completely different. You'll need to write different code for each platform you need to support.

Why are you using unmanaged memory? Perhaps if you explain what you're actually trying to achieve, somebody might be able to suggest a better solution.

[no name] 3-Jun-21 11:46am

Yes. I also thought: do they let you stick an OS into a Docker container? Isn't that a VM then? (I don't know Docker)

david marcus 9-Jun-21 19:04pm

The need for unmanaged memory comes from the data organization in the app. It is a memory-based DB that holds about 2.5M records per 1 GB of memory. Each record consists of many fields; some are scalar values, some fields are arrays. To get this many records per 1 GB of memory, the records contain inline values (NO managed objects - just a C struct with primitive values).

If these records had to be real .Net objects, the amount of memory required would be 10X, at least!

In any event, the real question is NOT why the app is using unmanaged memory (which IS supported in .Net 5), but rather how to read from a file (using .Net 5) into unmanaged memory.

After more research I found a solution:

I use File.Open(...) to get a FileStream object to the file. I then use Read(ReadOnlySpan<byte> buffer) to get it to read directly into unmanaged memory.

Thanks for trying to help.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

david marcus · Accepted Answer · 2021-06-09T13:08:00

Solution 1

Use: var fs = File.Open(...)

to get a FileStream object to the file.

Use: fs.Read(new ReadOnlySpan(pBuffer)) ;

where: byte* pBuffer; //-- pointer to unmanaged memory

Posted 9-Jun-21 13:08pm

david marcus