Hardware & Devices

5-Mar-08 14:04

I am posting this on the device driver forum because I think that this a Windows internal question and this about as a close of an area that I am going get

In Windows 32 bit were pointers are 32 bits there can conceptually be 2 process with the same address pointer e.g.

NorePad and outlook can both reference an address/pointer 12345678 and Windows/The OS would know to which process this pointer/address belongs

I was just wondering how this done is there some kind of selector/register (for lack of a better term) which qualfies an address/pointer

thankx

JudyL_MD6-Mar-08 2:26

6-Mar-08 2:26

ForNow wrote:
I was just wondering how this done is there some kind of selector/register (for lack of a better term) which qualfies an address/pointer

In the sense that you mean, given your questions in the other forum, the answer is NO.

In general terms, the OS internally maintains a set of page table mappings for each process that translate from the process's virtual address space to an actual physical address space. When a given process is scheduled for execution, the "current" page table pointers are adjusted to reference the page tables for that process. A virtual address goes in and a physical address comes out.

All the gory details (are there are lots of them!) can be found in Microsoft Windows Internals, 4th edition by Russinovich and Solomon

Judy

ForNow6-Mar-08 2:47

6-Mar-08 2:47

I have the book must be chapter 7 memory mangement

Any to way to access these page table pointer ??

Anyway thankx I read up

Thankx again

JudyL_MD6-Mar-08 2:56

6-Mar-08 2:56

ForNow wrote:
I have the book must be chapter 7 memory mangement

That's the one - longest chapter in the entire book

ForNow wrote:
Any to way to access these page table pointer ??

Not in any documented way. There are few things scarier than someone messing with the internals of the memory manager - the potential for wreaking havoc is immense.

Have fun!
Judy

ForNow6-Mar-08 6:09

6-Mar-08 6:09

Just an FYI

I worked as contractor for IBM in poughkeepise on IOS (Input Output Supervisor)component of MVS for a couple of years that deals a lot with Hardware of the MainFrame

Guess I am used getting to the heart of the matter

P.S.

I thought I could load some value (selector) in a register and have it qualify the 32 bit address I was looking at

Anyway thankx

Luc Pattyn6-Mar-08 6:54

Luc Pattyn

6-Mar-08 6:54

Hi,

the virtual-to-physical address translation is handled by the operating system, and is
not accessible by a user app; I trust you will need a driver for whatever it is you are
trying to do, hence also a DDK (Driver Development Kit), and that will offer you functions
to do all kinds of translations, effectively keeping the details out of your reach.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

This month's tips:
- before you ask a question here, search CodeProject, then Google;
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get;
- use PRE tags to preserve formatting when showing multi-line code snippets.

ForNow6-Mar-08 13:13

6-Mar-08 13:13

thankx

Randor 7-Mar-08 5:54

Randor

7-Mar-08 5:54

On XP SP1 and XP SP2 if the virtual address is (addr < 0x80000000 || addr >= 0xA0000000) then you can attempt to mask the address with 0x1FFFF000 like this:

PHYSICAL_ADDRESS addr;
addr.QuadPart = (ULONGLONG)addr.QuadPart & 0x1FFFF000;

The problem is if the virtual address is (0x80000000 <= addr && addr < 0xA0000000) then you cannot reliably calculate the physical address from ring-3 and must use a driver and call MmGetPhysicalAddress() its possible although with a probability of error.

And it gets worse with Vista and Windows 2003 SP1, because access to \Device\PhysicalMemory is disabled!

-David Delaune

Luc Pattyn7-Mar-08 6:23

Luc Pattyn

7-Mar-08 6:23

Hi, you may want to repost that in such a way that the OP gets a notification...

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Mike Dimmick7-Mar-08 15:56

Mike Dimmick

7-Mar-08 15:56

In normal execution, the processor handles translation from virtual to physical addresses completely automatically. The processor has a Translation Look-aside Buffer (TLB) which performs this operation. This is often described as an array of mappings that the processor 'searches' in one cycle, but is really a circuit in which the virtual address goes in the top and the physical address pops out of the bottom, if present.

If the required mapping is not in the TLB, the processor searches the page table. The control register CR3 points to the base of the page directory for the process. The Page Directory consists of Page Directory Entries which point to Page Tables. The Page Table Entries point to physical memory. If a valid Page Table Entry is found, the TLB is automatically updated and execution continues; otherwise, a Page Fault exception is raised to the OS which either fixes up the TLB directly and continues, fixes up the page table and continues (this might be the case where the data is already in RAM), reads in the code or data from disk (Windows is demand-paged, it doesn't read code from executables until it's actually called), or generates an Access Violation software exception.

Each process has its own Page Directory. When the OS switches from one thread to another, if the new thread is from a different process, it reloads the CR3 register with the address of the page directory for the new process. That automatically clears the TLB. (Page Table Entries can be marked Global; those are not cleared from the TLB.) Windows keeps the page directory base address information for each process in a data structure about that process.

The page table entry can be marked valid or invalid. A Valid page table entry is handled by the processor. If marked invalid (bit 0 = 0), the OS can (and does) store anything it likes in the other 31 bits. (To simplify things I'm talking about how 32-bit Windows worked on most machines before XP SP2!)

In some cases, particularly when memory has just been allocated, the PTE might not yet contain the necessary information. In addition to the page tables, Windows also stores information about all the virtual addresses that a process has allocated in a tree structure of Virtual Address Descriptors. It tracks what physical memory location belongs to which page table entry through the Page Frame Number Database, to update page tables appropriately when it has to reuse physical memory locations for something else.

DoEvents: Generating unexpected recursion since 1991

ForNow8-Mar-08 14:51

8-Mar-08 14:51

Thankx for the Info

A couple of quick questions in 32 bit Windows is the concept of segment offset still valid ?? however the offset is 32 bits

Anyway to access the CRs ??? or it all strictly hardware ???

Thankx

Mike Dimmick9-Mar-08 3:26

Mike Dimmick

9-Mar-08 3:26

In 32-bit protected mode the segment registers still work. The segment registers themselves are 16 bits wide and point to segment descriptors which have 32-bit base and limit fields. The base and limit fields are interpreted as virtual addresses. The segment registers are really only for aliasing and setting protection information.

Windows doesn't bother with segmented addressing, bar one case: it uses the FS segment register to point to the thread's Thread Information Block. This means code can use FS:[0] to point to the TIB throughout, without having to perform another lookup to find out where the TIB is. All the other segment registers point to segments (of the correct type) which have a base of 0 and a limit of 0xFFFFFFFF, that is, they can reference any address in the system. SS, DS, ES and GS are all set to the same value, while CS points to a different selector as selectors differentiate between code and data.

For exception handling, 32-bit Windows uses the TIB to carry a chain of exception handlers, so if you look at disassembly of a function which has an exception handler, you'll see instructions referring to FS:[0].

In 64-bit long mode (x64), AMD decided to effectively remove the legacy segment registers apart from FS and GS. (Some operating systems used GS, I think.) The legacy segment registers still have to be set to point to segment descriptors but the base and limit aren't checked. The instructions that used to manipulate some of these registers (e.g. PUSH CS) have been removed. (Currently the opcodes do nothing but are reserved for future use.) The MOV CS,rnn instructions are now the only way of loading these registers.

The processor includes MOV CRn,rr and MOV rr,CRn instructions but they can only be executed at privilege level ('ring') 0, which means in kernel mode. I'd strongly recommend not trying to change them as the OS will get very confused. There are device driver APIs documented for mapping a user-mode buffer into the system address space if necessary. If you want to share some address space between processes, use a file mapping object (CreateFileMapping).

For a reference on all this information, you can try volume 3 of the Intel Architecture Software Developer's Manuals[^]. It's hard going but this really is the master reference.

DoEvents: Generating unexpected recursion since 1991

ForNow9-Mar-08 7:50

Device Firmware Upgrade (DFU)

9-Mar-08 7:50

You have beeb very helpfull so the segement registers CS,DS etc..
are still 16 bit pointing to segemnt discriptors

EBX and EDX are 32 bit registers

In order to really understand I should write some X86 assembler programs My bacgound is In Mainframe 370 Assembler internals etc
So it shouldn't be that difficult

Sometimes in Visual Studio when I do a watch on data name I get this error CXXX017 it cann't evaluate the expression but when I go into disassembly mode I can see the value that get loaded so knowing masm is helpfull

On last question the CS registers is never really primed or loaded on X86 systems when process get control initially it get intialized to the first procedure and the value only changes with a Call inst/macro right ??

Thankx again

Roger Stoltz4-Mar-08 22:39

Roger Stoltz

4-Mar-08 22:39

Anyone know of some kind of generic Win32 driver for DFU?

The protocol is standardized even though the format of the downloaded files may differ, so it should be possible to create a generic driver for it and modify the .inf-file with the correct VID and PID for the device.

This is what I'm talking about:
http://www.usb.org/developers/devclass_docs/DFU_1.1.pdf[^]

"It's supposed to be hard, otherwise anybody could do it!" - selfquote
"High speed never compensates for wrong direction!" - unknown

System Hang while using pen drive

yogesh_kumar_agarwal4-Mar-08 20:18

yogesh_kumar_agarwal

4-Mar-08 20:18

Hi all

when i use my pendrive on my system it will hang but the same pen drive works fine in my friends pc we both have the same antivirus protection enabled and there is no virus in my pc .

What would thye cause for the same

Yogesh Agarwal

Re: System Hang while using pen drive

M.Nouri10-Mar-08 4:13

M.Nouri

10-Mar-08 4:13

Hi Yogesh
if you installel driver for this. you must uninstall dirver then retry to install for this.

Build a DLL with DDK

Member 41248731-Mar-08 3:41

1-Mar-08 3:41

Hi guys,

I'm trying to build a DLL using the DDK.
My Sources file is:

----- START SOURCES FILE -----
TARGETNAME = VirtualDiskTools
TARGETPATH = obj
TARGETTYPE = DYNLINK

USE_MSVCRT=0

INCLUDES = C:\WinDDK\2600.1106\inc\ddk\wxp
LIBS = %BUILD%\lib

SOURCES = VirtualDiskTools.c
----- END SOURCES FILE -----

And the MAKEFILE contains only this line:

!INCLUDE $(NTMAKEENV)\makefile.def

When I try to compile the project through Visual Studio, using the ddkbuild tool, I get the following error:

1>NMAKE : U1073: don't know how to make 'VirtualDiskTools.def'

I should change something in my sources file, maybe?

Thanks in advance

JudyL_MD3-Mar-08 2:23

3-Mar-08 2:23

OK, I'll ask the obvious - do you have a VirtualDiskTools.def and is it located in the source directory?

Judy

Member 41248733-Mar-08 8:14

3-Mar-08 8:14

Hi Judy,

no, I don't have any VirtualDiskTools.def in my source directory. The strange thing is that I can get a correct build of another project without a def file, compiled using TARGETTYPE=DRIVER (instead of TARGETTYPE=DYNLINK).

Maybe, compiling with DYNLINK, I need to create a def file?
So, how can I create VirtualDiskTools.def in my source directory?

Thanks for your reply

JudyL_MD3-Mar-08 8:43

3-Mar-08 8:43

Member 4124873 wrote:
Maybe, compiling with DYNLINK, I need to create a def file?
So, how can I create VirtualDiskTools.def in my source directory?

I agree that you need to create a .DEF file - in my quick search, I didn't find a single sources file that created a DYNLINK that didn't have an accompanying DEF file.

A DEF file is a plain text file that lists the exported functions from your DLL. A bare-bones file would be like:

LIBRARY YourNameHere
EXPORTS
FuncName1
FuncNAme2

For all the gory details, see Module Definition File[^]

Judy

Member 412487322-Mar-08 0:18

22-Mar-08 0:18

Maybe I'm too late, I'm sorry Judy.

I tried using your instructions. Please, confirm me the follow:

In my def file I define the name that I want for my DLL (LIBRARY), under the EXPORTS section, I can define a list of functions declared in my DDL source code, that can be accessible from external applications that are referencing the DLL. Is it right?

Because I tried to add into the EXPORTS section a function name (FileDiskCreateDevice) but when I try to compile, I get a lot of errors like:

Error 48 fatal error LNK1120: 23 unresolved externals objchk_wxp_x86\i386\VirtualDiskTools.dll

Error 17 error LNK2019: unresolved external symbol _KeGetCurrentThread@0 referenced in function _FileDiskDeviceControl@8 virtualdisktools.obj

Error 41 error LNK2019: unresolved external symbol _KeGetCurrentThread@0 referenced in function _FileDiskDeviceControl@8 virtualdisktools.obj

Error 11 error LNK2019: unresolved external symbol __imp__SeTokenType@4 referenced in function _FileDiskDeleteDevice@4 virtualdisktools.obj

What's wrong?

Thanks in advance.

JudyL_MD24-Mar-08 4:05

24-Mar-08 4:05

Member 4124873 wrote:
What's wrong?

Exactly what the error message says. You've used the functions KeGetCurrentThread and SeTokenType without including the implementations of them. You need to include the libraries for the functions in your sources file. I believe the first is in ntdll.lib but I'm not sure where the second one is since I've never done stuff that needed the IFS kit before. Look in an IFS example and see which libraries are included in that source file.

When I look up SeTokenType in the ddk documentation, it says that routine is reserved for system use. You should be using the recommended alternative SeQueryInformationToken.

Judy

Member 412487324-Mar-08 4:43