Address Windowing Extensions ( AWE ) technology on Windows platforms with Intel CPUs

[SergeyKostrov]

Hi everybody, It is a well known problem ( or limitation ) that a 32-bit application on some 32-bit Windows platforms can not allocate more than 2GB of memory. However, some 32-bit Windows platforms, for example Windows XP Professional or Windows Server 2003, support Address Windowing Extensions ( AWE ) technology ( designed by Microsoft ) and a 32-bit application could allocate more than 2GB of memory. Please take into account that Intel CPUs should have support for a Physical Address Extensions ( PAE ) feature. Enclosed is a very simple test-case in 'AweTestAppV1.zip' ( Visual Studio 2005 and up ) that tries to allocate different amounts of memory and allows to verify if a 32-bit application could allocate more than 2GB of memory on Windows platforms that support AWE. Notes: - An Intel CPU PAE feature is supported even on Intel Pentium(R) 4 CPU - A 32-bit Windows platform must support AWE ( for example, Windows XP Home doesn't support AWE ) - Please take a look at a small Readme.txt file for some technical details - For more information about AWE technology visit: http://msdn.microsoft.com/en-us/library/windows/hardware/gg487508.aspx Best regards, Sergey

[SergeyKostrov]

>>... >>What is 4GT? >>http://technet.microsoft.com/en-us/library/cc786709(WS.10).aspx I also recommend to look at these articles: How 4GT Works http://technet.microsoft.com/en-us/library/cc784475(v=ws.10).aspx 4GT Technical Reference http://technet.microsoft.com/en-us/library/cc778496(v=ws.10).aspx 4GT Tools and Settings http://technet.microsoft.com/en-us/library/cc737977(v=ws.10).aspx PAE x86 Technical Reference http://technet.microsoft.com/en-us/library/cc728455(v=ws.10).aspx

[Bernard]

>>>Iliya, please don't spend too much time on it unless you're really interested in AWE technology.>>> Why not:) I'm interesting in system programming.Sometimes I spent quite a lot of time with the kernel debugger while I'm digging into Windows internals. It is very interesting when you can see various critical data structures like KDPC(kernel deferred procedure calls) or PCR(processor control region) beign parsed and exposed by debugger.Now i'm working on kernel module which will be able to perform various manipulations on those structures.

[Bernard]

>>>I also recommend to look at these articles:>>> Thanks I would also recommend you to read "Windows Internals" books.

[SergeyKostrov]

>>Today I will create another VM with Win Server 2003 and extensive testing will be performed. What is your progress with installing Windows Server 2003 in the VM?

[Bernard]

Has been already installed,soon I will post the results.

[SergeyKostrov]

Hi Iliya, >>...Has been already installed, soon I will post the results... Just to clarify the task: At least 2.5GB of memory has to be allocated for the 32-bit test application ( source codes are provided ) on Windows Server 2003 ( 32-bit ). Let us know as soon as the test is completed. Best regards, Sergey

[Bernard]

>>>Just to clarify the task: At least 2.5GB of memory has to be allocated for the 32-bit test application ( source codes are provided ) on Windows Server 2003 ( 32-bit ). Let us know as soon as the test is completed.>>> Thanks. Soon I will post the results.(I'm quite busy with my projects and learning:)) If you are interested I'm posting an update to my "intrinsincs" project.Please look at my sine functions thread.

[Bernard]

@Sergey I'm switching to Win XP Pro because I have a stability issues with the Win Server 2003. Today I will run your code. Sorry for beign late with the tests.I was quite busy last few days. Btw.I created a new thread in "AVX and CPU instructions forum" where I described a strange issue related to inline assembly code. If you are interested please look at the code. Thanks in advance.

[SergeyKostrov]

Hi Iliya, Thanks for the update. >>I'm switching to Win XP Pro because I have a stability issues with the Win Server 2003. Do you mean when it is installed in VMware Workstation 7? >>Btw.I created a new thread in "AVX and CPU instructions forum" where I described a strange issue related to inline assembly code. >>If you are interested please look at the code. I saw and I'll take a look at it during the weekend.

[Bernard]

>>>Do you mean when it is installed in VMware Workstation 7?>>> Yes I have 5 VMs 4 of them are Win OSs(3 Win Server 2008 and 1 Win Server 2003) and one is Linux hacking distro Backtrack 4. Now those VMs are used to test downloaded malware infections with the help of static and dynamic analysis. Regarding Win Server 2003 stability issue it probabaly stems from my kernel debugging experiments where I usually try to perform modification of crucial data structures. >>>I saw and I'll take a look at it during the weekend.>>> Thank you.

[SergeyKostrov]

Here is update... I have some progress and finally I'm able to see how it works. However, it will take some time before a final decision about applicability of AWE-based solution will be made. There are lots of concerns and the most important is AWE is Not a portable solution.

[Bernard]

>>>Here is update... I have some progress and finally I'm able to see how it works. However, it will take some time before a final decision about applicability of AWE-based solution will be made. There are lots of concerns and the most important is AWE is Not a portable solution.>>> I can see that you were able to allocate 100% of the requested pages.Was your previous AWE mem allocation tests so successful?

[SergeyKostrov]

>>...Was your previous AWE mem allocation tests so successful? No because only yesterday I finally created a right environment for AWE tests. I'm still verifying results and sources because there are many questions. I will also need to spend more time on documentation.

[SergeyKostrov]

By the way, a long vertical red line is not an allocation of memory pages. It is some kind of moving cursor.

[Bernard]

>>>No because only yesterday I finally created a right environment for AWE tests. I'm still verifying results and sources because there are many questions. I will also need to spend more time on documentation.>>> Can you send me for testing your source code? I need to download 32-bit Win 2003 OS and I will test your AWE code.

[SergeyKostrov]

>>...Can you send me for testing your source code? Yes, of course. I've created two more little projects ( there are 3 in total now ) but the last one is in a very raw-state. I will upload all updated projects in about one week or later. I also would like to mention that I really underestimated efforts related to AWE tests. It took more than a couple of days to create a right environment in a VMware Player 5.0.1 ( build-894247 ). In overall, I clearly see that AWE is a more advanced version of a 20+-year-old Expanded Memory Specification ( EMS ) technology for 16-bit operating systems. In the past, Microsoft and Intel had problems with address space limitations twice: - it wasn't enough for applications to have a 20-bit address space in a 16-bit MS-DOS & Windows 2.x/3.x ( without Win32s extension ) - it wasn't enough for applications to have a 32-bit address space in a 32-bit Windows

[Bernard]

>>>In overall, I clearly see that AWE is a more advanced version of a 20+-year-old Expanded Memory Specification ( EMS ) technology for 16-bit operating systems. In the past, Microsoft and Intel had problems with address space limitations twice:>>> It should be more advanced partly because of more advanced and capable hardware. >>>Yes, of course. I've created two more little projects ( there are 3 in total now ) but the last one is in a very raw-state. I will upload all updated projects in about one week or later. I also would like to mention that I really underestimated efforts related to AWE tests. It took more than a couple of days to create a right environment in a VMware Player 5.0.1 ( build-894247 ).>>> Thank you. Regarding VMware player why do not you use VWware workstation?

[SergeyKostrov]

>>...why do not use VWware workstation?

It is up to you what VMware version should be used on your computer(s). I don't want to start a discussion about advantages or disadvantages of different versions of VMware.

[Bernard]

Sergey Kostrov wrote:

>>...why do not use VWware workstation?

It is up to you what VMware version should be used on your computer(s). I don't want to start a discussion about advantages or disadvantages of different versions of VMware.

I also do not want to talk about the advantages and disadvantages of the variuos versions of VMware software.For my needs Workstation version is must for  me.

[Bernard]

iliyapolak wrote:

Quote:

Sergey Kostrovwrote:

>>...why do not use VWware workstation?

It is up to you what VMware version should be used on your computer(s). I don't want to start a discussion about advantages or disadvantages of different versions of VMware.

I also do not want to talk about the advantages and disadvantages of the variuos versions of VMware software.For my needs Workstation version is must for  me.

My future plan includes usage of 10-12 virtualized 0S's simlating network segment composed from the various varsion of patched and unpatched Windows OS's I would like to test malware propagation and infection.I have already downloaded 3 DVD filled with various malware.

[SergeyKostrov]

>>My future plan includes usage of 10-12 virtualized 0S's simlating network segment composed from the various varsion of >>patched and unpatched Windows OS's I would like to test malware propagation and infection. I have >>already downloaded 3 DVD filled with various malware. Iliya, I'm very sorry but the subject of the thread is Address Windowing Extensions ( AWE ) technology on Windows platforms with Intel CPUs and I really don't understand why you're continuing discussions on absolutely unrelated subjects like malware propagation and infection. You could create a new thread in http://software.intel.com/en-us/forums/watercooler-catchall for any subjects which do not violate Intel's Forums Policies. Once again, PLEASE do not "pollute" threads with unrelated subjects and I know that sometimes it is not an easy thing because we're on a forum. Thanks in advance. Best regards, Sergey