AT&T AST Google Summer of Code Project

The project proposal is 2 main parts:
  1. to support FUSE file system instances on Windows-NT (via UWIN)
  2. to make UWIN fully compatible with the GNU compiler via minGW
The project will combine four OpenSource projects: AT&T AST software, UWIN (Unix for Windows), the FUSE file system, and minGW. All parts of the project are open source: AST and UWIN are covered by the Eclipse license; FUSE is covered by the Lesser GPL; minGW is covered by the GPL and its own MIT-style license.


Background

The AT&T AST Toolkit contains around 150 OpenSource commands as well as ksh93. UWIN, Unix for Windows, provides a UNIX environment on Windows. UWIN is Open Source and is similar to the OpenSource Cygwin project except UWIN is somewhat faster and provides full support for both 32 and 64 bit processes; all processes, 32 and 64 bit, UWIN and Windows, are in the same process namespace. UWIN also has different software portability goals in that it strives to provide source compatibility without the need for #ifdef __UWIN modifications.


(1) FUSE filesystems on UWIN

The goal of this project is to implement the FUSE filesystem API on UWIN. Preferably any FUSE filesystem instance implementation for UNIX would be used unchanged (and un-#ifdef'd), so the bulk of the work would be to implement FUSE in UWIN in such a way that UNIX compatible FUSE file system type implementations can be built and run without modification on UWIN.

The FUSE filesystem test case will be a FUSE implementation of the AT&T 3D filesystem. The 3D file system is a file system enhancement that allows one directory tree to be mounted on top of another. The top and bottom trees are called layers. A 3D mount forms a view that is the union of the top and bottom layers. If a file or directory resides in both the top and bottom layers then only the one in the top layer is visible (the top layer file covers the bottom layer file.) The bottom layer is copy-on-write so that any changes made to a file named by a path in the top layer that physically resides in the bottom layer causes the file to first be copied to the top layer before it is modified.

Pairwise 3D mounts can be combined to form views with multiple layers. The virtual directory named ... refers to the directory in the next layer. In a view with 2 layers, . names the top layer, ... names the next layer, and .../... does not exist. Thus ... can be used to determine the number of layers in the current view. ... can also be used to refer to covered files in lower layers in the view. For example, if the current view has 3 layers, 1 (top), 2 (middle), and 3 (bottom), and the file foo physically resides in layers 1 and 3, then ./foo names the file in layer 1, and .../foo and .../.../foo name the file in layer 3; if foo physically resides in layers 1 and 2, then ./foo names the file in layer 1, .../foo names the file in layer 2, and .../.../foo does not exist.

The 3D file system is very useful for large software development projects where an individual user needs to test and debug changes that won't immediately affect other users in the project. Instead of making a complete individual copy of the code, each user can simply create an empty directory, 3D mount it on top of the base code directory tree, and then repeat the typical edit build debug test development cycle in the top layer. When the development cycle is complete the top layer will contain all changes and the bottom layer will remain untouched. Large software projects in AT&T typically have views of 3 to 6 layers.

The original 3D file system is part of AST. ksh93 supports the vpath built-in which mounts a directory on top of another. The original implementation uses a shared library to intercept all pathname system calls and is thus a per-process entity. This has become to difficult to support, so recently we implemented 3D as a FUSE file system type. FUSE 3D supports both per-user and system-wide mounts which simplifies administration for large projects on UNIX systems that support FUSE. The same functionality, however, is currently unavailable on Windows.


(2) minGW on UWIN

UWIN provides a POSIX portability layer on top of Windows NT, 7 and 8 for 23 and 64 bit applications. A large part of the protability base is the AST toolkit; other parts are filled in by OpenSource applications and libraries like ssh, perl, and X11.

Currently the distributed UWIN binaries are built by the freely available (but not OpenSource) MSVC compiler. Providing full OpenSource compiler support will open up UWIN to a wider portion of the OpenSource community. The work will involve UWIN header, library and cc(1) wrapper command modifications, building and testing UWIN itself using the OpenSource compiler, and in addition extensive testing of OpenSource applications built with the OpenSource compiler.


Candidates

The ideal GSOC candidate for part (1) would be familiar with AST software, UWIN, and have some knowledge of the FUSE file system. The ideal GSOC candidate for part (2) would be familiar with AST software, UWIN, and minGW or GCC.

David Korn    dgkorn@gmail.com
Glenn Fowler    glenn.s.fowler@gmail.com


March 29, 2013