UWIN 32 and 64 bit native support

UWIN, starting with version 5.0, supports both 32 and 64 bit native processes. On 32 bit windows you must run 32 bit UWIN, but on 64 bit windows you may run 32 and/or 64 bit UWIN.


Background

32 bit binary objects, executables, libraries and DLLs have different magic numbers and headers than the 64 bit counterparts. Programs that poke around binary headers must be aware of this. 32 bit executables require 32 bit execution time and run time DLLs and 64 bit executables require 64 bit execution time and run time DLLs. This has implications on runtime plugin modules and on a UWIN implementation that supports 32 and 64 bit processes. In particular, UWIN must provide both 32 and 64 bit versions of posix.dll and ast54.dll.

Only 32 bit processes are supported on 32 bit windows. However, both 32 and 64 bit processes may run on 64 bit windows.

The native windows execution time DLL path search always looks in the directory of the executable first. Since the default executable and DLL installation directories are the same in UWIN (i.e., the bin directory; the ast nmake base rules handle this automatically), and 32 bit and 64 bit executables are installed in separate windows directories, the native windows execution time DLL path search always locates the appropriate 32/64 bit DLLs.

The windows HANDLE type size is 32 bits on 32 bit windows and 64 bits on 64 bit windows, but only the low order 32 bits are used in 64 bit windows. Furthermore, handles with the same low order 32 bits may be exchanged between 32 and 64 bit processes. If you think about it, this is really the only way windows could support 32 executing 64 bit processes and vice versa. The implications for UWIN are that access to persistent handle stores must account for the 32 vs 64 bit HANDLE size difference.


WOW 32/64

WOW (Windows on Windows) is MicroSoft's terminology for 32 bit support on 64 bit windows. Its easy to grouse about the other guys from your cozy 40 year old sandbox, but windows handling of 32 vs. 64 bit windows and 32 vs. 64 bit applications is an inconceivable mess.

The default case for 32 or 64 bit processes is reasonable:

  • System executables and DLLs are in 'C:\Windows\System32'.
  • Registry entries are in 'HKLM\...'.
However, certain registry keys have a 32 bit and 64 bit view. In some cases the views are separate but maintained consistent by windows, and in other cases they are completely separate.

Software installation directories start getting messy:

  • On 32 bit windows 32 bit applications are installed in 'C:\Program Files'.
  • On 64 bit windows 32 bit applications are installed in 'C:\Program Files (x86)' and 64 bit applications are installed in 'C:\Program Files'.
This means that an installer must change its installation directories for 32 and 64 bit windows.

The Visual C installation adds more complications:

  • 32 bit compiler and SDK binaries and libraries are installed in these directories:
    C:\Program Files (x86)\Microsoft Visual Studio *.*\VC\bin
    C:\Program Files (x86)\Microsoft Visual Studio *.*\VC\lib
    C:\Program Files\Microsoft SDKs\Windows\v*.*\bin
    C:\Program Files\Microsoft SDKs\Windows\v*.*\lib
    
  • 64 bit compiler and SDK binaries and libraries are installed in these directories:
    C:\Program Files (x86)\Microsoft Visual Studio *.*\VC\bin\(amd64|x64)
    C:\Program Files (x86)\Microsoft Visual Studio *.*\VC\lib\(amd64|x64)
    C:\Program Files\Microsoft SDKs\Windows\v*.*\bin\x64
    C:\Program Files\Microsoft SDKs\Windows\v*.*\lib\x64
    
Note that MicroSoft VC breaks convention by installing 64 bit executables in 'C:\Program Files (x86)'.

The fun starts when 32 bit processes attempt to access 64 bit files and registry keys:

  • The 64 bit 'C:\Windows\System32' is 'C:\Windows\SysNative'.
  • 64 bit registry keys must be accessed using the KEY_WOW64_64KEY flag in the Reg*() API. There is no registry path name mechanism so this requires application level recoding.
  • The 64 bit MicroSoft VC paths are listed above.
More (and different) fun when a 64 bit processes attempt to access 32 bit files and registry keys:
  • The 32 bit 'C:\Windows\System32' is 'C:\Windows\SysWOW64'.
  • 'C:\Windows\System32' is for 64 bit executables and DLLs, but 'C:\Windows\SysNative' is an invalid path for 64 bit processes.
  • 32 bit registry keys must be accessed using the KEY_WOW64_32KEY flag in the Reg*() API. There is a registry path name mechanism: all 32 bit keys are placed in the Wow6432Node subkey, but all MicroSoft documentation warns against using it.
  • The 32 bit MicroSoft VC paths are listed above.
Several API calls are provided to control System32 redirection: GetSystemWow64Directory, Wow64DisableWow64FsRedirection, Wow64EnableWow64FsRedirection, and Wow64RevertWow64FsRedirection. These set a thread-global state, and improper usage can cause system DLL load failures.

The upshot of all of this is there is no predictable mechanism or convention for handling 32 and 64 bit objects. Each situation introduces a new naming convention or API. In some cases naming conventions may allow applications to work around 32/64 incompatibilities. New APIs, however, require applications to be recoded.


UWIN solution

UWIN provides one path-based mechanism for accessing 32 and 64 bit objects from 32 and 64 bit processes. There are two virtual root directories, /32 and /64, that may be prepended to any path. /32/... accesses the 32 bit specific version of the path if any, otherwise the generic path with /32 omitted. Similarly, /64/... accesses the 64 bit specific version of the path if any, otherwise the generic path with /64 omitted. By default, if a path does not have a /32 or /64 prefix, then for 32 bit processes the path is first checked with a /32 prefix and if not found is then checked without the prefix. For 64 bit processes the path is first checked with a /64 prefix and if not found is then checked with a /32 prefix. The 64 bit mapping supports a smooth migration from 32 to 64 bit executables: if a 64 bit executable is not available then the corresponding 32 bit executable is used. Because MicroSoft does not freely distribute 64 bit versions of some of its 32 bit libraries, some applications in 64 bit UWIN packages are 32 bits.

For example, in a 32 bit process /32/bin/cat and /bin/cat are equivalent and /64/bin/cat accesses the 64 bit cat(1) executable. Similarly, in a 64 bit process /64/bin/cat and /bin/cat are equivalent and /32/bin/cat accesses the 32 bit cat(1) executable. In UWIN /bin/locate is a shell script, so the paths /bin/locate, /32/bin/locate, and /64/bin/locate all point to the same file for both 32 and 64 bit processes.

In practice the /32 and /64 prefixes are mainly used in the UWIN package installation scripts (even the 64 bit installation packages may have 32 bit components) and in the cc(1) command to handle the -m32 and -m64 options.

These UWIN directories may have /32 and /64 bit variants:

/msdev
/reg
/sys
/usr
/var
/32 or /64 prefixes on any other path are ignored.

Because UWIN supports both 32 and 64 bit windows it completely bypasses windows registry WOW redirection for its own keys and places all UWIN specific keys in the native registry. This means that the UWIN registry pathnames point to the same physical data across 32 and 64 bit windows. Otherwise the /32 and /64 prefixes apply to other /reg/... paths.

On 64 bit systems /proc contains both 32 and 64 bit processes. *32 is appended to the simple base name of 32 bit processes. Otherwise 32 and 64 bit processes behave the same (they should, or its a UWIN implementation or documentation bug).

Scripts may use the -i option to uname(1), which lists process-bits/windows-bits, to check process and windows bittedness. At present the possible values are 32/32, 32/64 and 64/64. Finally, winpath(1) lists the underlying windows path for any UNIX or windows path.


Viewpathing

UWIN provides a process global viewpath that changes the default path mapping for 32 and 64 bit processes. This shell builtin command, from a 32 bit process, causes path lookup to search for 64 bit binaries first:
vpath / /64
And this command, from a 64 bit process, causes path lookup to search for 32 bit binaries first:
vpath / /32
The former is how we first built 64 bit UWIN from 32 bit UWIN. Finally, this command clears the current viewpath, if any:
vpath - /
Recall that UWIN takes advantage of the windows default execution time DLL search so vpath does not (cannot) affect the search.


Implementation Details

There is a bit of magic under the UWIN hood to get all of this to work.

For convenience, the inode number for /32 is 32 and /64 is 64. Therefore by default the inode number for / is 32 for 32 bit processes and 64 for 64 bit processes.

In addition to the default mounted directories listed by df(1), UWIN reserves the directories /u32 and /v32. On 64 bit windows these directories are 32 bit shadows of /usr and /var respectively, and contain the 32 bit binary versions of the 64 bit counterparts in /usr and /var. This is the one peek behind the curtain. Users are encouraged to use the portable /32/usr and /32/var instead of the implementation artifacts /u32 and /v32. The names /u32 and /v32 were carefully chosen; by changing only 2 characters in a copy of a path name (/usr <=> /u32 or /var <=> /v32) UWIN can easily check for the 32 bit shadow of a generic path.

A little bit more work was required to get readdir(3) to eliminate duplicates when reading directories like /usr/bin that may be shadowed by /32/usr/bin or /64/usr/bin. The bit specific binary versions are always returned, and any shadowed version in the generic directory is ignored.

Finally, the creation or renaming of any file in a /32 or /64 mapped directory is checked for 32 or 64 bit binary magic and is placed in the appropriate physical directory with the .exe appended as needed.


June 01, 2013