sciencething.org

Geekthings

Home

Where to Go Now.

I last worked on UVFS sometime in 2003 or 2004. After that it got handed on to Interwoven who have continued to develop it. Thank you ex-employer of mine. It has it's own Sourceforge project now: UVFS. I haven't tried using it since I last worked for Interwoven, but it was just updated on March 31, 2008. I assume it works considerably better than when I last worked on it, and that it behaves with recent kernels. I should also point out fuse, which is in the main kernel code-line. Very nifty, and has ports to several OSs including, perhaps, Windows. The rest of the information about UVFS is of historical interest only.

What is it?

UVFS is a user-space filesystem kit for linux. It consists of a compile time configurable kernel module which communicates with a user space program which implements a filesystem. Other such kits already exist: userfs, which seems to be dead, PODFUK, and fuse. I've used userfs in the past and was disappointed at how easy it was to crash. I haven't tried PODFUK because the last time I downloaded it, it wouldn't compile. I've used fuse and it works and it has a clean and simple user-space interface for writing filesystems.

UVFS distinguishes itself by its goals, which are to translate as much of the kernel vfs layer as possible to the userspace interface, be as fast as possible and to amuse me. I plan on actively maintaining and extending UVFS for the foreseeable future.

UVFS is known to work on 2.4.13 and 2.4.17 kernels, and should probably work on other 2.4.x kernels with, perhaps, some fixups. Somewhere in the 2.4.x series of kernels the MODULE_LICENCE macro was introduced. If you build uvfs against an earlier kernel you will need to comment out the MODULE_LICENSE use in kernel/init.c. I have no idea if UVFS would work with 2.5.x kernels. Probably it won't

How to use UVFS.

UVFS comes with a sample filesystem, pmfs. Pmfs is an in memory filesystem which saves and loads its contents to and from a plain file.

Here is how to build the standard uvfs distribution. cd into the uvfs directory. Edit kernel_pmfs/Makefile to reflect whether you are building for an SMP machine. (Remove the -D__SMP__ from the CFLAGS line at the tops of the Makefile if you're on a UP system). Also modify the kernel include path to match that of your setup. Run make depend. Run make.

To run pmfs, become root, cd into the kernel_pmfs directory and do an insmod -x pmfs.o. Change into the pmfs directory and run ./pmfs 2>log followed by a ../uvfs_utils/uvfs_mount pmfs /path/to/backing.file /mountpoint. You can repeat the uvfs_mount invocation multiple times with different backing files. The backing file does not have to exist. At mount time if the specified backing file does not exist pmfs creates a new empty filesystem. Any time you umount a pmfs instance and there are no more outstanding mounts of that instance, pmfs writes the contents of that instance to its backing file.

Now that you have pmfs running please try to break it. If you do break it drop me a line telling me how you did it.

To shut down the pmfs filesystem unmount it, and run ../uvfs_utils/uvfs_signal /proc/fs/pmfs shutdown followed by a rmmod pmfs.

How to write your own filesystem.

Create a new directory containing symlinks to uvfs/kernel/*.[ch]. Copy the Makefile from the pmfs directory to this new directory and edit the new copy. First set the name of the target .o file. Next, edit the CFLAGS line. Set the name of both the filesystem and character device by setting -DUVFS_FS_NAME=. Set options using the following flags:

  • UVFS_IMPL_DELETE_INODE causes the kernel module to make delete_inode calls to the user space filesystem. To write a strictly correct unix filesystem you should turn this option on. Files should continue to exist until the kernel tells the filesystem that the file is no longer referenced. The delete_inode VFS call is how the kernel tells a filesystem that a file is no longer referenced. A good many unix programs expect to be able to unlink a file and continue to use an open descriptor to that file. (It's a way for an application to use an invisible temp file, for instance.) That being said, this behavior is one of the more disgusting design decisions in the original unix filesystem, and you should feel free to implement filesystems that don't have this behavior.
  • UVFS_IMPL_READ_WRITE causes the kernel driver to implement the file_read and file_write VFS calls. With UVFS_IMPL_READ_WRITE turned on regular file reads and writes bypass the page cache. This is useful, for example, if your filesystem already has its file data buffered. This is the case for pmfs or for a filesystem that uses another local filesystem to store it's data. If your filesystem does not do it's own buffering then you should turn off UVFS_IMPL_READ_WRITE.
  • UVFS_IMPL_SYMLINK causes the kernel driver to implement the symlink, readlink and followlink VFS methods. For a normal unix filesystem you should implement these. You can turn it off if you are writing a filesystem that must interoperate with a symlinkless filesystem.
  • UVFS_IMPL_MKNOD causes the kernel driver to implement the mknod VFS method. For best unix filesystem verisimilitude implement this. For many filesystems this is unnecessary.
  • UVFS_IMPL_LINK causes the kernel driver to implement the link VFS method. Again for best unix filesystem verisimilitude implement this. If you want to make the world a better place, though, don't implement link. Hardlinks (along with link counts and publicly visible inode numbers) are another of the (thankfully few) disgusting unix design decisions and should therefore be discouraged.

As UVFS gets stabilized I intend to provide a complete reference, and an in depth HOWTO. For now you'll have to make do with the following.

 

Follow what the example filesystem does to implement your user-space filesystem. In brief a user space filesystem (1) opens /proc file associated with the filesystem (/proc/fs/pmfs for pmfs e.g.) (2) goes into a loop calling read on the file to get a filesystem request, processing that request and calling write on the file to reply to the request. The filesystem knows it should shutdown when it gets a UVFS_SHUTDOWN filesystem request. The file kernel/protocol.h contains all of the filesystem requests that a user-space filesystem might have to implement.

A user-space filesystem can be multithreaded. By default the example filesystem, pmfs, runs in with two processing threads. Pmfs can compiled in single thread mode by removing the -DPMFS_THREADED flag from the pmfs Makefile. The kernel driver for UVFS simply queues filesystem requests. Any number of user space threads can handle items off the queue by simply doing read, process, write in a loop. On my dual 1.3GHZ Athlon system pmfs does 17000 operations per second under heavy concurrent load with two threads and 11000 operation per second with only one thread.

Known Problems

If UVFS_IMPL_DELETE_INODE is turned on and a userspace filesystem calls rmdir on a foreign filesystem while processing a rmdir request, the system can deadlock. The workaround is to not turn on UVFS_IMPL_DELETE_INODE if your filesystem fits this description. If your filesystem absolutely has to have UVFS_IMPL_DELETE_INODE turned on there is a workaround in the kernel portion of version 0.2 or earlier of UVFS.

In low VM situations UVFS filesystems can cause strange kernel oops's. I haven't been able to determine what the cause is but it may be related to a possible deadlock situation pointed out to me by Pavel Macek: A UVFS filesystem could be swapped out when a writepage for that filesystem comes through. These oops's don't appear to be, on the surface, deadlocks, but one never knows. The workaround for this problem is not to run systems under so much load that they swap. ;)

Under heavy concurrent stress pmfs will very infrequently delete files it shouldn't. This seem to be caused by some kind of dcache corruption which causes rm -r to delete files in the parent directory of the directory that it is supposed to be deleting files in. This is obviously a serious problem. I'm working on it diligently. However UVFS is stable enough under normal loads to use for new filesystem development. BTW this problem isn't new. It almost certainly existed in earlier version of UVFS. Update February 22, 2002: I haven't been able to reproduce this for the last week. I may have inadvertantly fixed this problem while doing other things.

Licensing

The kernel driver is licenced under the GPL which should be included in this distribution. The user-space interface as embodied by kernel/protocol.h and the rest of the code are licensed under the LGPL which should also be included with this distribution. My intent is that anyone is allowed to cut and paste from the example filesystem and utilities, even if it's for a commercial product. If you change the substance of the example filesystem and utilities you should feel obliged to make those changes available for free.

The latest version of UVFS should always be available here.

Questions, comments, bug fixes, new filesystems are welcome.

Contact Britt Park at britt@sciencething.org