     [file version.texi not included]

The GNU Hurd
************

This file documents the GNU Hurd kernel component.  This edition of the documentation was last updated for version VERSION of the Hurd.

Copyright © 1994, 1996, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007, 2008, 2009 Free Software Foundation, Inc.

     Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

     Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

     Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions.

Introduction
************

The GNU Hurd (The name Hurd stands for ``Hird of Unix-Replacing Daemons.''  The name Hird stands for ``Hurd of Interfaces Representing Depth.'') is the GNU Project's replacement for the Unix kernel. The Hurd is a collection of servers that run on the Mach microkernel to implement file systems, network protocols, file access control, and other features that are normally implemented by the Unix kernel or similar kernels (such as Linux).

Audience
========

This manual is designed to be useful to everybody who is interested in using, administering, or programming the Hurd.

If you are an end-user and you are looking for help on running the Hurd, the first few chapters of this manual describe the essential parts of installing, starting up, and shutting down a Hurd workstation.  If you need help with a specific program, the best way to use this manual is to find the program's name in the index and go directly to the appropriate section.  You may also wish to try running PROGRAM --help, which will display a brief usage message for PROGRAM (see Foundations).

The rest of this manual is a technical discussion of the Hurd servers and their implementation, and would not be helpful until you want to learn how to modify the Hurd.

This manual is organized according to the subsystems of the Hurd, and each chapter begins with descriptions of utilities and servers that are related to that subsystem.  If you are a system administrator, and you want to learn more about, say, the Hurd networking subsystem, you can skip to the networking chapter (see Networking), and read about the related utilities and servers.

Programmers who are interested in learning how to modify Hurd servers, or write new ones, should begin by learning about a microkernel to which the Hurd has been ported (currently only GNU Mach) and reading Foundations.  You should then familiarize yourself with a subsystem that interests you by reading about existing servers and the libraries they use.  At that point, you should be able to study the source code of existing Hurd servers and understand how they use the Hurd libraries.

The final level of mastery is learning the about the RPC interfaces which the Hurd libraries implement.  The last section of each chapter describes any Hurd interfaces used in that subsystem.  Those sections assume that you are perusing the referenced interface definitions as you read.  After you have understood a given interface, you will be in a good position to improve the Hurd libraries, design your own interfaces, and implement new subsystems.

Features
========

The Hurd is not the most advanced operating system known to the planet (yet), but it does have a number of enticing features:

it's free software
     Anybody can use, modify, and redistribute it under the terms of the GNU General Public License (see Copying).  The Hurd is part of the GNU system, which is a complete operating system licensed under the GPL.

it's compatible
     The Hurd provides a familiar programming and user environment.  For all intents and purposes, the Hurd is a modern Unix-like kernel.  The Hurd uses the GNU C Library, whose development closely tracks standards such as ANSI/ISO, BSD, POSIX, Single Unix, SVID, and X/Open.

it's built to survive
     Unlike other popular kernel software, the Hurd has an object-oriented structure that allows it to evolve without compromising its design. This structure will help the Hurd undergo major redesign and modifications without having to be entirely rewritten.

it's scalable
     The Hurd implementation is aggressively multithreaded so that it runs efficiently on both single processors and symmetric multiprocessors. The Hurd interfaces are designed to allow transparent network clusters (collectives), although this feature has not yet been implemented.

it's extensible
     The Hurd is an attractive platform for learning how to become a kernel hacker or for implementing new ideas in kernel technology.  Every part of the system is designed to be modified and extended.

it's stable
     It is possible to develop and test new Hurd kernel components without rebooting the machine (not even accidentally).  Running your own kernel components doesn't interfere with other users, and so no special system privileges are required.  The mechanism for kernel extensions is secure by design: it is impossible to impose your changes upon other users unless they authorize them or you are the system administrator.

it exists
     The Hurd is real software that works Right Now.  It is not a research project or a proposal.  You don't have to wait at all before you can start using and developing it.

Overview
========

An operating system kernel provides a framework for programs to share a computer's hardware resources securely and efficiently.  This framework includes mechanisms for programs to communicate safely, even if they do not trust one another (see Ports Library).

The GNU Hurd divides up the work of the traditional kernel, and implements it in separate programs, or kernel servers.  The Hurd formally defines the communication protocols that each of the servers understands, so that it is possible for different servers to implement the same interface.

The GNU C Library provides a POSIX environment on the Hurd, by translating standard POSIX system calls into interactions with the appropriate Hurd server.

History
=======

Richard Stallman (RMS) started GNU in 1983, as a project to create a complete free operating system.  In the text of the GNU Manifesto, he mentioned that there is a primitive kernel.  In the first GNUsletter, Feb. 1986, he says that GNU's kernel is TRIX, which was developed at the Massachusetts Institute of Technology.

By December of 1986, the Free Software Foundation (FSF) had ``started working on the changes needed to TRIX'' [Gnusletter, Jan. 1987]. Shortly thereafter, the FSF began ``negotiating with Professor Rashid of Carnegie-Mellon University about working with them on the development of the Mach kernel'' [Gnusletter, June, 1987].  The text implies that the FSF wanted to use someone else's work, rather than have to fix TRIX.

In [Gnusletter, Feb. 1988], RMS was talking about taking Mach and putting the Berkeley Sprite filesystem on top of it, ``after the parts of Berkeley Unix... have been replaced.''

Six months later, the FSF is saying that ``if we can't get Mach, we'll use TRIX or Berkeley's Sprite.''  Here, they present Sprite as a full-kernel option, rather than just a filesystem.

In January, 1990, they say ``we aren't doing any kernel work.  It does not make sense for us to start a kernel project now, when we still hope to use Mach'' [Gnusletter, Jan. 1990].  Nothing significant occurs until 1991, when a more detailed plan is announced:

     ``We are still interested in a multi-process kernel running on top of
     Mach. The CMU lawyers are currently deciding if they can release Mach
     with distribution conditions that will enable us to distribute it. If
     they decide to do so, then we will probably start work. CMU has
     available under the same terms as Mach a single-server partial Unix
     emulator named Poe; it is rather slow and provides minimal
     functionality. We would probably begin by extending Poe to provide full
     functionality. Later we hope to have a modular emulator divided into
     multiple processes.'' [Gnusletter, Jan. 1991].

RMS explains the relationship between the Hurd and Linux in http://www.gnu.org/software/hurd/hurd-and-linux.html, where he mentions that the FSF started developing the Hurd in 1990.  As of [Gnusletter, Nov. 1991], the Hurd (running on Mach) is GNU's official kernel.

GNU General Public License
==========================

     [file gpl.texinfo not included]

Bootstrap
*********

Bootstrapping (The term bootstrapping refers to a Dutch legend about a boy who was able to fly by pulling himself up by his bootstraps.  In computers, this term refers to any process where a simple system activates a more complicated system.) is the procedure by which your machine loads the microkernel and transfers control to the Hurd servers.

Bootloader
==========

The bootloader is the first software that runs on your machine. Many hardware architectures have a very simple startup routine which reads a very simple bootloader from the beginning of the internal hard disk, then transfers control to it.  Other architectures have startup routines which are able to understand more of the contents of the hard disk, and directly start a more advanced bootloader.

GNU GRUB (The GRand Unified Bootloader, available from http://www.gnu.org/software/grub/.) is the GNU bootloader. GNU GRUB provides advanced functionality, and is capable of loading several different kernels (such as Linux, DOS, and the *BSD family).

From the standpoint of the Hurd, the bootloader is just a mechanism to get the microkernel running and transfer control to the Hurd servers. You will need to refer to your bootloader and microkernel documentation for more information about the details of this process.

Server Bootstrap
================

As it can be seen from the GNU GRUB configuration, the Hurd is bootstrapped by starting the GNU Mach microkernel and two programs: the root filesystem and the exec server.

The --multiboot-command-line option tells the file system server that it is a root filesystem, which triggers it to run /hurd/init as PID 1.  /hurd/init starts the /hurd/proc and /hurd/auth servers.  After the servers are launched /hurd/init starts the /libexec/runsystem.sh script to finish booting.

After the Hurd has been booted, other sets of core Hurd servers can be started in parallel.  They're called sub-Hurds or neighbor Hurds (see Recursive Bootstrap).

Recursive Bootstrap
-------------------

The boot program can be used to start a set of core Hurd servers while another Hurd is already running.  You will rarely need to do this, and it requires superuser privileges to control the new Hurd (or allow it to access certain devices), but it is interesting to note that it can be done.

Usually, you would make changes to only one server, and simply tell your programs to use it in order to test out your changes.  This process can be applied even to the core servers.  However, some changes have far-reaching effects, and so it is nice to be able to test those effects without having to reboot the machine.

Here are the steps you can follow to test out a new set of servers:

 1. Create a pseudo-root device.  Usually, you would do this by creating a new partition under your old Hurd, and initializing it with your favorite filesystem format.  boot understands the regular libstore options (FIXME xref), so you may use a file or other store instead of a partition.

          $ dd if=/dev/zero of=my-partition bs=1024k count=400
          400+0 records in
          400+0 records out
          $ mke2fs ./my-partition
          mke2fs 1.18, 11-Nov-1999 for EXT2 FS 0.5b, 95/08/09
          my-partition is not a block special device.
          Proceed anyway? (y,n) y
          Filesystem label=
          OS type: GNU/Hurd
          Block size=1024 (log=0)
          Fragment size=1024 (log=0)
          102400 inodes, 409600 blocks
          20480 blocks (5.00%) reserved for the super user
          First data block=1
          50 block groups
          8192 blocks per group, 8192 fragments per group
          2048 inodes per group
          Superblock backups stored on blocks:
                  8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

          Writing inode tables: done
          Writing superblocks and filesystem accounting information: done
          $

 2. Copy the core servers, C library, your modified programs, and anything else you need onto the pseudo-root.

          $ settrans -c ./my-root /hurd/ext2fs -r `pwd`/my-partition
          $ fsysopts ./my-root --writable
          $ cd my-root
          $ tar -zxpf /pub/debian/FIXME/gnu-20000929.tar.gz
          $ cd ..
          $ fsysopts ./my-root --readonly

 3. Create a new boot script (FIXME xref).

 4. Run boot.

          $ boot -D ./my-boot ./my-boot/boot/servers.boot ./my-partition
          [...]

 5. Here is an example using a hard drive that already has a GNU/Hurd system installed on an ext2 filesystem on /dev/hd2s1.

          $ settrans /mnt /hurd/ex2fs --readonly /dev/hd2s1
          $ boot -d -D /mnt -I /mnt/boot/servers.boot /dev/hd2s1

 6. See Invoking boot for help with boot.

Note that it is impossible to share microkernel devices between the two running Hurds, so don't get any funny ideas.  When you're finished testing your new Hurd, then you can run the halt or reboot programs to return control to the parent Hurd.

If you're satisfied with your new Hurd, you can arrange for your bootloader to start it, and reboot your machine.  Then, you'll be in a safe place to overwrite your old Hurd with the new one, and reboot back to your old configuration (with the new Hurd servers).

Boot Scripts
------------

Boot Scripts are used to boot further Hurd systems in parallel to the first, and are parsed by boot to boot a sub-Hurd.

In that script, the variables HOST-PORT and DEVICE-PORT are integers which represent the microkernel host and device ports, respectively (and are used to initialize the ${host-port} and ${device-port} boot script variables).  If these ports are not specified, then it is assumed that the Hurd is already running, and the current ports will be fetched from the procserver.

ROOT-NAME is the name of the microkernel device that should be used as the Hurd bootstrap filesystem.

FIXME: finish

Invoking boot
-------------

Usage: boot [OPTION...] BOOT-SCRIPT DEVICE...

--kernel-command-line=COMMAND LINE
-c
     Simulated multiboot command line to supply.

--pause
-d
     Pause for user confirmation at various times during booting.

--boot-root=DIR
-D
     Root of a directory tree in which to find the files specified in BOOT-SCRIPT.

--interleave=BLOCKS
     Interleave in runs of length BLOCKS.

--isig
-I
     Do not disable terminal signals, so you can suspend and interrupt the boot program itself, rather than the programs running in the booted system.

--layer
-L
     Layer multiple devices for redundancy.

--single-user
-s
     Boot into single user mode.

--store-type=TYPE
-T
     Each DEVICE names a store of type TYPE.

Mandatory or optional arguments to long options are also mandatory or optional for any corresponding short options.

If neither --interleave or --layer is specified, multiple DEVICEs are concatenated.

Shutdown
========

FIXME: finish

Foundations
***********

Every Hurd program accepts the following optional arguments:

--help
     Display a brief usage message, then exit.  This message is not a substitute for reading program documentation; rather, it provides useful reminders about specific command-line options that a program understands.

--version
     Output program version information and exit.

The rest of this chapter provides a programmer's introduction to the Hurd.  If you are not a programmer, then this chapter will not make much sense to you... you should consider skipping to descriptions of specific Hurd programs (see Audience).

The Hurd distribution includes many libraries in order to provide a useful set of tools for writing Hurd utilities and servers.  Several of these libraries are useful not only for the Hurd, but also for writing microkernel-based programs in general.  These fundamental libraries are not difficult to understand, and they are a good starting point, because the rest of the Hurd relies upon them quite heavily.

Threads Library
===============

All Hurd servers and libraries are aggressively multithreaded in order to take full advantage of any multiprocessing capabilities provided by the microkernel and the underlying hardware.  The Hurd threads library, libthreads, contains the default Hurd thread implementation, which is declared in <cthreads.h>.

Currently (April 1998), the Hurd uses cthreads, which have already been documented thoroughly by CMU.  Eventually, it will be migrated to use POSIX pthreads, which are documented in a lot of places.

Every single library in the Hurd distribution (including the GNU C library) is completely thread-safe, and the Hurd servers themselves are aggressively multithreaded.

Ports Library
=============

Ports are communication channels that are held by the kernel.

A port has separate send rights and receive rights, which may be transferred from task to task via the kernel.  Port rights are similar to Unix file descriptors: they are per-task integers which are used to identify ports when making kernel calls.  Send rights are required in order to send an RPC request down a port, and receive rights are required to serve the RPC request.  Receive rights may be aggregated into a single portset, which serve as useful organizational units.

In a single-threaded RPC client, managing and categorizing ports is not a difficult process.  However, in a complex multithreaded server, it is useful to have a more abstract interface to managing portsets, as well as maintaining server metadata.

The Hurd ports library, libports, fills that need.  The libports functions are declared in <hurd/ports.h>.

Buckets and Classes
-------------------

The libports bucket is simply a port set, with some metadata and a lock.  All of the libports functions operate on buckets.

 -- Function: struct port_bucket * ports_create_bucket (void)
     Create and return a new, empty bucket.

A port class is a collection of individual ports, which can be manipulated conveniently, and have enforced deallocation routines. Buckets and classes are entirely orthogonal: there is no requirement that all the ports in a class be in the same bucket, nor is there a requirement that all the ports in a bucket be in the same class.

 -- Function: struct port_class ports_create_class (void (*CLEAN_ROUTINE) (void *PORT), void (*DROPWEAK_ROUTINE) (void *PORT))
     Create and return a new port class.  If nonzero, CLEAN_ROUTINE will be called for each allocated port object in this class when it is being destroyed.  If nonzero, DROPWEAK_ROUTINE will be called to request weak references to be dropped.  (If DROPWEAK_ROUTINE is null, then weak references and hard references will be identical for ports of this class.)

Once you have created at least one bucket and class, you may create new ports, and store them in those buckets.  There are a few different functions for port creation, depending on your application's requirements:

 -- Function: error_t ports_create_port (struct port_class *CLASS, struct port_bucket *BUCKET, size_t SIZE, void *RESULT)
     Create and return in RESULT a new port in CLASS and BUCKET; SIZE bytes will be allocated to hold the port structure and whatever private data the user desires.

 -- Function: error_t ports_create_port_noinstall (struct port_class *CLASS, struct port_bucket *BUCKET, size_t SIZE, void *RESULT)
     Just like ports_create_port, except don't actually put the port into the portset underlying BUCKET.  This is intended to be used for cases where the port right must be given out before the port is fully initialized; with this call you are guaranteed that no RPC service will occur on the port until you have finished initializing it and installed it into the portset yourself.

 -- Function: error_t ports_import_port (struct port_class *CLASS, struct port_bucket *BUCKET, mach_port_t PORT, size_t SIZE, void *RESULT)
     For an existing receive right, create and return in RESULT a new port structure; BUCKET, SIZE, and CLASS args are as for ports_create_port.

Port Rights
-----------

The following functions move port receive rights to and from the port structure:

 -- Function: void ports_reallocate_port (void *PORT)
     Destroy the receive right currently associated with PORT and allocate a new one.

 -- Function: void ports_reallocate_from_external (void *PORT, mach_port_t RECEIVE)
     Destroy the receive right currently associated with PORT and designate RECEIVE as the new one.

 -- Function: void ports_destroy_right (void *PORT)
     Destroy the receive right currently associated with PORT.  After this call, ports_reallocate_port and ports_reallocate_from_external may not be used.

 -- Function: mach_port_t ports_claim_right (void *PORT)
     Return the receive right currently associated with PORT.  The effects on PORT are the same as in ports_destroy_right, except that the receive right itself is not affected.  Note that in multi-threaded servers, messages might already have been dequeued for this port before it gets removed from the portset; such messages will get EOPNOTSUPP errors.

 -- Function: error_t ports_transfer_right (void *TOPT, void *FROMPT)
     Transfer the receive right from FROMPT to TOPT. FROMPT ends up with a destroyed right (as if ports_destroy_right were called) and TOPT's old right is destroyed (as if ports_reallocate_from_external were called).

 -- Function: mach_port_t ports_get_right (void *PORT)
     Return the name of the receive right associated with PORT.  The user is responsible for creating an ordinary send right from this name.

Port Metadata
-------------

It is important to point out that the PORT argument to each of the libports functions is a void * and not a struct port_info *.  This is done so that you may add arbitrary meta-information to your libports-managed ports.  Simply define your own structure whose first element is a struct port_info, and then you can use pointers to these structures as the PORT argument to any libports function.

The following functions are useful for maintaining metadata that is stored in your own custom ports structure:

 -- Function: void * ports_lookup_port (struct port_bucket *BUCKET, mach_port_t PORT, struct port_class *CLASS)
     Look up PORT and return the associated port structure, allocating a reference.  If the call fails, return zero.  If BUCKET is nonzero, then it specifies a bucket to search; otherwise all buckets will be searched.  If CLASS is nonzero, then the lookup will fail if PORT is not in CLASS.

 -- Function: error_t ports_bucket_iterate (struct port_bucket *BUCKET, error_t (*FUN) (void *PORT))
     Call FUN once for each port in BUCKET.  No guarantee is made about the order of iteration, which might vary from call to call.  If FUN returns an error, then no further calls to FUN are made for any remaining ports, and the return value of FUN is returned from ports_bucket_iterate.

Port References
---------------

These functions maintain references to ports so that the port information structures may be freed if and only if they are no longer needed.  It is your responsibility to tell libports when references to ports change.

 -- Function: void ports_port_ref (void *PORT)
     Allocate a hard reference to PORT.

 -- Function: void ports_port_deref (void *PORT)
     Drop a hard reference to PORT.

 -- Function: void ports_no_senders (void *PORT, mach_port_mscount_t MSCOUNT)
     The user is responsible for listening for no senders notifications; when one arrives, call this routine for the PORT the message was sent to, providing the MSCOUNT from the notification.

 -- Function: int ports_count_class (struct port_class *CLASS)
     Block creation of new ports in CLASS.  Return the number of ports currently in CLASS.

 -- Function: int ports_count_bucket (struct port_bucket *BUCKET)
     Block creation of new ports in BUCKET.  Return the number of ports currently in BUCKET.

 -- Function: void ports_enable_class (struct port_class *CLASS)
     Permit suspended port creation (blocked by ports_count_class) to continue.

 -- Function: void ports_enable_bucket (struct port_bucket *BUCKET)
     Permit suspended port creation (blocked by ports_count_bucket) to continue.

Weak references are not often used, as they are the same as hard references for port classes where DROPWEAK_ROUTINE is null. See Buckets and Classes.

 -- Function: void ports_port_ref_weak (void *PORT)
     Allocate a weak reference to PORT.

 -- Function: void ports_port_deref_weak (void *PORT)
     Drop a weak reference to PORT.

RPC Management
--------------

The rest of the libports functions are dedicated to controlling RPC operations.  These functions help you do all the locking and thread cancellations that are required in order to build robust servers.

 -- Typedef: typedef int (*ports_demuxer_type) (mach_msg_header_t *INP, mach_msg_header_t *OUTP)
     Type of MiG demuxer routines.

 -- Function: error_t ports_begin_rpc (void *PORT, mach_msg_id_t MSG_ID, struct rpc_info *INFO)
     Call this when an RPC is beginning on PORT.  INFO should be allocated by the caller and will be used to hold dynamic state.  If this RPC should be abandoned, return EDIED; otherwise we return zero.

 -- Function: void ports_end_rpc (void *PORT, struct rpc_info *INFO)
     Call this when an RPC is concluding.  The arguments must match the ones passed to the paired call to ports_begin_rpc.

 -- Function: void ports_manage_port_operations_one_thread (struct port_bucket *BUCKET, ports_demuxer_type DEMUXER, int TIMEOUT)
     Begin handling operations for the ports in BUCKET, calling DEMUXER for each incoming message.  Return if TIMEOUT is nonzero and no messages have been received for TIMEOUT milliseconds.  Use only one thread (the calling thread).

 -- Function: void ports_manage_port_operations_multithread (struct port_bucket *BUCKET, ports_demuxer_type DEMUXER, int THREAD_TIMEOUT, int GLOBAL_TIMEOUT, void (*HOOK) (void))
     Begin handling operations for the ports in BUCKET, calling DEMUXER for each incoming message.  Return if GLOBAL_TIMEOUT is nonzero and no messages have been received for GLOBAL_TIMEOUT milliseconds.  Create threads as necessary to handle incoming messages so that no port is starved because of sluggishness on another port.  If THREAD_TIMEOUT is nonzero, then individual threads will die off if they handle no incoming messages for LOCAL_TIMEOUT milliseconds.  If non-null, HOOK will be called in each new thread immediately after it is created.

 -- Function: error_t ports_inhibit_port_rpcs (void *PORT)
     Interrupt any pending RPC on PORT.  Wait for all pending RPCs to finish, and then block any new RPCs starting on that port.

 -- Function: error_t ports_inhibit_class_rpcs (struct port_class *CLASS)
     Similar to ports_inhibit_port_rpcs, but affects all ports in CLASS.

 -- Function: error_t ports_inhibit_bucket_rpcs (struct port_bucket *BUCKET)
     Similar to ports_inhibit_port_rpcs, but affects all ports in BUCKET.

 -- Function: error_t ports_inhibit_all_rpcs (void)
     Similar to ports_inhibit_port_rpcs, but affects all ports whatsoever.

 -- Function: void ports_resume_port_rpcs (void *PORT)
     Reverse the effect of a previous ports_inhibit_port_rpcs for this PORT, allowing blocked RPCs to continue.

 -- Function: void ports_resume_class_rpcs (struct port_class *CLASS)
     Reverse the effect of a previous ports_inhibit_class_rpcs for CLASS.

 -- Function: void ports_resume_bucket_rpcs (struct port_bucket *BUCKET)
     Reverse the effect of a previous ports_inhibit_bucket_rpcs for BUCKET.

 -- Function: void ports_resume_all_rpcs (void)
     Reverse the effect of a previous ports_inhibit_all_rpcs.

 -- Function: void ports_interrupt_rpcs (void *PORT)
     Cancel (with thread_cancel) any RPCs in progress on PORT.

 -- Function: int ports_self_interrupted (void)
     If the current thread's RPC has been interrupted with ports_interrupt_rpcs, return nonzero and clear the interrupted flag.

 -- Function: error_t ports_interrupt_rpc_on_notification (void *OBJECT, struct rpc_info *RPC, mach_port_t PORT, mach_msg_id_t WHAT)
     Arrange for hurd_cancel to be called on RPC's thread if OBJECT gets notified that any of the things in WHAT have happened to PORT.  RPC should be an RPC on OBJECT.

 -- Function: error_t ports_interrupt_self_on_notification (void *OBJECT, mach_port_t PORT, mach_msg_id_t WHAT)
     Arrange for hurd_cancel to be called on the current thread, which should be an RPC on OBJECT, if PORT gets notified with the condition WHAT.

 -- Function: error_t ports_interrupt_self_on_port_death (void *OBJECT, mach_port_t PORT)
     Same as calling ports_interrupt_self_on_notification with WHAT set to MACH_NOTIFY_DEAD_NAME.

 -- Function: void ports_interrupt_notified_rpcs (void *OBJECT, mach_port_t PORT, mach_msg_id_t WHAT)
     Interrupt any RPCs on OBJECT that have requested such.

 -- Function: void ports_dead_name (void *OBJECT, mach_port_t PORT)
     Same as calling ports_interrupt_notified_rpcs with WHAT set to MACH_NOTIFY_DEAD_NAME.

Integer Hash Library
====================

libihash provides integer-keyed hash tables, for arbitrary element data types.  Such hash tables are frequently used when implementing sparse arrays or buffer caches.

The following functions are declared in <hurd/ihash.h>:

 -- Function: error_t ihash_create (ihash_t *HT)
     Create an integer hash table and return it in HT.  If a memory allocation error occurs, ENOMEM is returned, otherwise zero.

 -- Function: void ihash_free (ihash_t HT)
     Free HT and all resources it consumes.

 -- Function: void ihash_set_cleanup (ihash_t HT, void (*CLEANUP) (void *VALUE, void *ARG), void *ARG)
     Sets HT's element cleanup function to CLEANUP, and its second argument to ARG.  CLEANUP will be called on every element VALUE to be subsequently overwritten or deleted, with ARG as the second argument.

 -- Function: error_t ihash_add (ihash_t HT, int ID, void *ITEM, void ***LOCP)
     Add ITEM to the hash table HT under the integer key ID.  LOCP is the address of a pointer located in ITEM; If non-null, LOCP should point to a variable of type void **, and will be filled with a pointer that may be used as an argument to ihash_locp_remove.  The variable pointed to by LOCP may be overwritten sometime between this call and when the element is deleted, so you cannot stash its value elsewhere and hope to use the stashed value with ihash_locp_remove.  If a memory allocation error occurs, ENOMEM is returned, otherwise zero.

 -- Function: void * ihash_find (ihash_t HT, int ID)
     Find and return the item in hash table HT with key ID. Returns null if the specified item doesn't exist.

 -- Function: error_t ihash_iterate (ihash_t HT, error_t (*FUN) (void *VALUE))
     Call function FUN on every element of HT.  FUN's only arg, VALUE, is a pointer to the value stored in the hash table.  If FUN ever returns nonzero, then iteration stops and ihash_iterate returns that value, otherwise it (eventually) returns 0.

 -- Function: int ihash_remove (ihash_t HT, int ID)
     Remove the entry with a key of ID from HT.  If there was no such element, then return zero, otherwise nonzero.

 -- Function: void ihash_locp_remove (ihash_t HT, void **HT_LOCP)
     Remove the entry at LOCP from the hashtable HT.  LOCP is as returned from an earlier call to ihash_add.  This call should be faster than ihash_remove.  HT can be null, in which case the call still succeeds, but no cleanup is done.

Misc Library
============

The GNU C library is constantly developing to meet the needs of the Hurd.  However, because the C library needs to be very stable, it is irresponsible to add new functions to it without carefully specifying their interface, and testing them thoroughly.

The Hurd distribution includes a library called libshouldbeinlibc, which serves as a proving ground for additions to the GNU C library.  This library is in flux, as some functions are added to it by the Hurd developers and others are moved to the official C library.

These functions aren't currently documented (other than in their header files), but complete documentation will be added to The GNU C Library Reference Manual (libc), when these functions become part of the GNU C library.

Bug Address Library
===================

libhurdbugaddr exists only to define a single variable:

 -- Variable: char * argp_program_bug_address
     argp_program_bug_address is the default Hurd bug-reporting e-mail address, bug-hurd@gnu.org.  This address is displayed to the user when any of the standard Hurd servers and utilities are invoked using the --help option.

Input and Output
****************

There are no specific programs or servers associated with the I/O subsystem, since it is used to interact with almost all servers in the GNU Hurd.  It provides facilities for reading and writing I/O channels, which are the underlying implementation of file and socket descriptors in the GNU C library.

Iohelp Library
==============

The <hurd/iohelp.h> file declares several functions which are useful for low-level I/O implementations.  Most Hurd servers do not call these functions directly, but they are used by several of the Hurd filesystem and networking helper libraries.  libiohelp requires libthreads.

I/O Users
---------

Most I/O servers need to implement some kind of user authentication checking.  In order to facilitate that process, libiohelp has some functions which encapsulate a set of idvecs (FIXME: xref to C library) in a single struct iouser.

 -- Function: struct iouser * iohelp_create_iouser (struct idvec *UIDS, struct idvec *GIDS)
     Create a new IOUSER for the specified UIDS and GIDS.

 -- Function: struct iouser * iohelp_dup_iouser (struct iouser *IOUSER)
     Return a copy of IOUSER.

 -- Function: void iohelp_free_iouser (struct iouser *IOUSER)
     Release a reference to IOUSER.

I/O reauthentication is a rather complex protocol involving the authserver as a trusted third party (see Auth Protocol).  In order to reduce the risk of flawed implementations, I/O reauthentication is encapsulated in the iohelp_reauth function:

 -- Function: struct iouser * iohelp_reauth (auth_t AUTHSERVER, mach_port_t REND_PORT, mach_port_t NEWRIGHT, int PERMIT_FAILURE)
     Conduct a reauthentication transaction, and return a new IOUSER. AUTHSERVER is the I/O server's auth port.  The rendezvous port provided by the user is REND_PORT.

     If the transaction cannot be completed, return zero, unless PERMIT_FAILURE is nonzero.  If PERMIT_FAILURE is nonzero, then should the transaction fail, return an IOUSER that has no ids.  The new port to be sent to the user is NEWRIGHT.

Conch Management
----------------

The conch is at the heart of the shared memory I/O system. Several Hurd libraries implement shared I/O, and so libiohelp contains functions to facilitate conch management.

Everything about shared I/O is undocumented because it is not needed for adequate performance, and the RPC interface is simpler (see I/O Interface).  It is not useful for new libraries or servers to implement shared I/O.

Pager Library
=============

The external pager (XP) microkernel interface allows applications to provide the backing store for a memory object, by converting hardware page faults into RPC requests.  External pagers are required for memory-mapped I/O (see Mapped Data) and stored filesystems (see Stored Filesystems).

The external pager interface is quite complex, so the Hurd pager library contains functions which aid in creating multithreaded external pagers. libpager is declared in <hurd/pager.h>, and requires only the threads and ports libraries.

Pager Management
----------------

The pager library defines the struct pager data type in order to represent a multi-threaded pager.  The general procedure for creating a pager is to define the functions listed in Pager Callbacks, allocate a libports bucket for the ports which will access the pager, and create at least one new struct pager with pager_create.

 -- Function: struct pager * pager_create (struct user_pager_info *U_PAGER, struct port_bucket *BUCKET, boolean_t MAY_CACHE, memory_object_copy_strategy_t COPY_STRATEGY)
     Create a new pager.  The pager will have a port created for it (using libports, in BUCKET) and will be immediately ready to receive requests.  U_PAGER will be provided to later calls to pager_find_address.  The pager will have one user reference created.  MAY_CACHE and COPY_STRATEGY are the original values of those attributes as for memory_object_ready.  Users may create references to pagers by use of the relevant ports library functions.  On errors, return null and set errno.

Once you are ready to turn over control to the pager library, you should call ports_manage_port_operations_multithread on the BUCKET, using pager_demuxer as the ports DEMUXER. This will handle all external pager RPCs, invoking your pager callbacks when necessary.

 -- Function: int pager_demuxer (mach_msg_header_t *INP, mach_msg_header_t *OUTP)
     Demultiplex incoming libports messages on pager ports.

The following functions are the body of the pager library, and provide a clean interface to pager functionality:

 -- Function: void pager_sync (struct pager *PAGER, int WAIT)
 -- Function: void pager_sync_some (struct pager *PAGER, vm_address_t START, vm_size_t LEN, int WAIT)
     Write data from pager PAGER to its backing store.  Wait for all the writes to complete if and only if WAIT is set.

     pager_sync writes all data; pager_sync_some only writes data starting at START, for LEN bytes.

 -- Function: void pager_flush (struct pager *PAGER, int WAIT)
 -- Function: void pager_flush_some (struct pager *PAGER, vm_address_t START, vm_size_t LEN, int WAIT)
     Flush data from the kernel for pager PAGER and force any pending delayed copies.  Wait for all pages to be flushed if and only if WAIT is set.

     pager_flush flushes all data; pager_flush_some only flushes data starting at START, for LEN bytes.

 -- Function: void pager_return (struct pager *PAGER, int WAIT)
 -- Function: void pager_return_some (struct pager *PAGER, vm_address_t START, vm_size_t LEN, int WAIT)
     Flush data from the kernel for pager PAGER and force any pending delayed copies.  Wait for all pages to be flushed if and only if WAIT is set.  Have the kernel write back modifications.

     pager_return flushes and restores all data; pager_return_some only flushes and restores data starting at START, for LEN bytes.

 -- Function: void pager_offer_page (struct pager *PAGER, int PRECIOUS, int WRITELOCK, vm_offset_t PAGE, vm_address_t BUF)
     Offer a page of data to the kernel.  If PRECIOUS is set, then this page will be paged out at some future point, otherwise it might be dropped by the kernel.  If the page is currently in core, the kernel might ignore this call.

attributes void pager_change_attributes (struct pager *PAGER, boolean_t MAY_CACHE, memory_object_copy_strategy_t COPY_STRATEGY, int WAIT) Change the attributes of the memory object underlying pager PAGER. The MAY_CACHE and COPY_STRATEGY arguments are as for memory_object_change_.  Wait for the kernel to report completion if and only if WAIT is set.

 -- Function: void pager_shutdown (struct pager *PAGER)
     Force termination of a pager.  After this returns, no more paging requests on the pager will be honoured, and the pager will be deallocated.  The actual deallocation might occur asynchronously if there are currently outstanding paging requests that will complete first.

 -- Function: error_t pager_get_error (struct pager *P, vm_address_t ADDR)
     Return the error code of the last page error for pager P at address ADDR. (Note that this function will be deleted when the Mach pager interface is fixed to provide this information.)

 -- Function: error_t pager_memcpy (struct pager *PAGER, memory_object_t MEMOBJ, vm_offset_t OFFSET, void *OTHER, size_t *SIZE, vm_prot_t PROT)
     Try to copy *SIZE bytes between the region OTHER points to and the region at OFFSET in the pager indicated by PAGER and MEMOBJ.  If PROT is VM_PROT_READ, copying is from the pager to OTHER; if PROT contains VM_PROT_WRITE, copying is from OTHER into the pager. *SIZE is always filled in with the actual number of bytes successfully copied.  Returns an error code if the pager-backed memory faults; if there is no fault, returns zero and *SIZE will be unchanged.

These functions allow you to recover the internal struct pager state, in case the libpager interface doesn't provide an operation you need:

 -- Function: struct user_pager_info * pager_get_upi (struct pager *P)
     Return the struct user_pager_info associated with a pager.

 -- Function: mach_port_t pager_get_port (struct pager *PAGER)
     Return the port (receive right) for requests to the pager.  It is absolutely necessary that a new send right be created from this receive right.

Pager Callbacks
---------------

Like several other Hurd libraries, libpager depends on you to implement application-specific callback functions.  You must define the following functions:

 -- Function: error_t pager_read_page (struct user_pager_info *PAGER, vm_offset_t PAGE, vm_address_t *BUF, int *WRITE_LOCK)
     For pager PAGER, read one page from offset PAGE.  Set *BUF to be the address of the page, and set *WRITE_LOCK if the page must be provided read-only.  The only permissible error returns are EIO, EDQUOT, and ENOSPC.

 -- Function: error_t pager_write_page (struct user_pager_info *PAGER, vm_offset_t PAGE, vm_address_t BUF)
     For pager PAGER, synchronously write one page from BUF to offset PAGE.  In addition, vm_deallocate (or equivalent) BUF.  The only permissible error returns are EIO, EDQUOT, and ENOSPC.

 -- Function: error_t pager_unlock_page (struct user_pager_info *PAGER, vm_offset_t ADDRESS)
     A page should be made writable.

 -- Function: error_t pager_report_extent (struct user_pager_info *PAGER, vm_address_t *OFFSET, vm_size_t *SIZE)
     This function should report in *OFFSET and *SIZE the minimum valid address the pager will accept and the size of the object.

 -- Function: void pager_clear_user_data (struct user_pager_info *PAGER)
     This is called when a pager is being deallocated after all extant send rights have been destroyed.

 -- Function: void pager_dropweak (struct user_pager_info *P)
     This will be called when the ports library wants to drop weak references.  The pager library creates no weak references itself, so if the user doesn't either, then it is all right for this function to do nothing.

I/O Interface
=============

The I/O interface facilities are described in <hurd/io.defs>. This section discusses only RPC-based I/O operations. (The latter portion of <hurd/io.defs> and all of <hurd/shared.h> describe how to implement shared-memory I/O operations.  However, shared I/O has been deprecated.  See Conch Management, for more details.)

I/O Object Ports
----------------

The I/O server must associate each I/O port with a particular set of uids and gids, identifying the user who is responsible for operations on the port.  Every port to an I/O server should also support either the file protocol (see File Interface) or the socket protocol (see Socket Interface); naked I/O ports are not allowed.

In addition, the server associates with each port a default file pointer, a set of open mode bits, a pid (called the ``owner''), and some underlying object which can absorb data (for write) or provide data (for read).

The uid and gid sets associated with a port may not be visibly shared with other ports, nor may they ever change.  The server must fix the identification of a set of uids and gids with a particular port at the moment of the port's creation.  The other characteristics of an I/O port may be shared with other users.  The I/O server interface does not generally specify the way in which servers may share these other characteristics (with the exception of the deprecated O_ASYNC interface); however, the file and socket interfaces make further requirements about what sharing is required and what sharing is prohibited.

In general, users get send rights to I/O ports by some mechanism that is external to the I/O protocol.  (For example, fileservers give out I/O ports in response to the dir_lookup and fsys_getroot calls.  Socket servers give out ports in response to the socket_create and socket_accept calls.)  However, the I/O protocol provides methods of obtaining new ports that refer to the same underlying object as another port.  In response to all of these calls, all underlying state (including, but not limited to, the default file pointer, open mode bits, and underlying object) must be shared between the old and new ports.  In the following descriptions of these calls, the term ``identical'' means this kind of sharing.  All these calls must return send rights to a newly-constructed Mach port.

The io_duplicate call simply returns another port which is identical to an existing port and has the same uid and gid set.

The io_restrict_auth call returns another port, identical to the provided port, but which has a smaller associated uid and gid set.  The uid and gid sets of the new port are the intersection of the set on the existing port and the lists of uids and gids provided in the call.

Users use the io_reauthenticate call when they wish to have an entirely new set of uids or gids associated with a port.  In response to the io_reauthenticate call, the server must create a new port, and then make the call auth_server_authenticate to the auth server.  The rendezvous port for the auth_server_authenticate call is the I/O port to which was made the io_reauthenticate call.  The server provides the REND_INT parameter to the auth server as a copy from the corresponding parameter in the io_reauthenticate call.  The I/O server also gives the auth server a new port; this must be a newly created port identical to the old port.  The authserver will return the set of uids and gids associated with the user, and guarantees that the new port will go directly to the user that possessed the associated authentication port. The server then identifies the new port given out with the specified ID's.

Simple Operations
-----------------

Users write to I/O ports by calling the io_write RPC.  They specify an OFFSET parameter; if the object supports writing at arbitrary offsets, the server should honour this parameter.  If -1 is passed as the offset, then the server should use the default file pointer.  The server should return the amount of data which was successfully written.  If the operation was interrupted after some but not all of the data was written, then it is considered to have succeeded and the server should return the amount written.  If the port is not an I/O port at all, the server should reply with the error EOPNOTSUPP.  If the port is an I/O port, but does not happen to support writing, then the correct error is EBADF.

Users read from I/O ports by calling the io_read RPC.  They specify the amount of data they wish to read, and the offset.  The offset has the same meaning as for io_write above.  The server should return the data that was read.  If the call is interrupted after some data has been read (and the operation is not idempotent) then the server should return the amount read, even if it was less than the amount requested. The server should return as much data as possible, but never more than requested by the user.  If there is no data, but there might be later, the call should block until data becomes available.  The server indicates end-of-file by returning zero bytes.  If the call is interrupted after some data has been read, but the call is idempotent, then the server may return EINTR rather than actually filling the buffer (taking care that any modifications of the default file pointer have been reversed).  Preferably, however, servers should return data.

There are two categories of objects: seekable and non-seekable. Seekable objects must accept arbitrary offset parameters in the io_read and io_write calls, and must implement the io_seek call.  Non-seekable objects must ignore the offset parameters to io_read and io_write, and should return ESPIPE to the io_seek call.

On seekable objects, io_seek changes the default file pointer for reads and writes.  (see File Positioning (The GNU C Library Reference Manual), for the interpretation of the WHENCE and OFFSET arguments.) It returns the new offset as modified by io_seek.

The io_readable interface returns the amount of data which can be immediately read.  For the special technical meaning of ``immediately'', see Asynchronous I/O.

Open Modes
----------

The server associates each port with a set of bits that affect its operation.  The io_set_all_openmodes call modifies these bits and the io_get_openmodes call returns them.  In addition, the io_set_some_openmodes and io_clear_some_openmodes do an atomic read/modify/write of the openmodes.

The O_APPEND bit, when set, changes the behaviour of io_write when it uses the default file pointer on seekable objects.  When io_write is done on a port with the O_APPEND bit set, is must set the file pointer to the current file size before doing the write (which would then increment the file pointer as usual).  The current file size is the smallest offset which returns end-of-file when provided to io_read.  The server must atomically bind this update to the actual data write with respect to other users of io_read, io_write, and io_seek.

The O_FSYNC bit, when set, guarantees that io_write will not return until data is fully written to the underlying medium.

The O_NONBLOCK bit, when set, prevents read and write from blocking.  They should copy such data as is immediately available.  If no data is immediately available they should return EWOULDBLOCK.

The definition of ``immediately'' is more or less server-dependent. Some servers, notably stored filesystem servers (see Stored Filesystems), regard all data as immediately available.  The one criterion is that something which must happen immediately may not wait for any user-synchronizable event.

The O_ASYNC bit is deprecated; its use is documented in the following section.  This bit must be shared between all users of the same underlying object.

Asynchronous I/O
----------------

Users may wish to be notified when I/O can be done without blocking; they use the io_async call to indicate this to the server.  In the io_async call the user provides a port on which will the server should send sig_post messages as I/O becomes possible. The server must return a port which will be the reference port in the sig_post messages.  Each io_async call should generate a new reference port.  (FIXME: xref the C library manual for information on how to send sig_post messages.)

The server then sends one SIGIO signal to each registered async user everytime I/O becomes possible.  I/O is possible if at least one byte can be read or written immediately.  The definition of ``immediately'' must be the same as for the implementation of the O_NONBLOCK flag (see Open Modes).  In addition, every time a user calls io_read or io_write on a non-seekable object, or at the default file pointer on a seekable object, another signal should be sent to each user if I/O is still possible.

Some objects may also define ``urgent'' conditions.  Such servers should send the SIGURG signal to each registered async user anytime an urgent condition appears.  After any RPC that has the possibility of clearing the urgent condition, the server should again send the signal to all registered users if the urgent condition is still present.

A more fine-grained mechanism for doing async I/O is the io_select call.  The user specifies the kind of access desired, and a send-once right.  If I/O of the kind the user desires is immediately possible, then the server should return so indicating, and destroy the send-once right.  If I/O is not immediately possible, the server should save the send-once right, and send a select_done message as soon as I/O becomes immediately possible.  Again, the definition of ``immediately'' must be the same for io_select, io_async, and O_NONBLOCK (see Open Modes).

For compatibility with 4.2 and 4.3 BSD, the I/O interface provides a deprecated feature (known as icky async I/O).  The calls io_mod_owner and io_get_owner set the ``owner'' of the object, providing either a pid or a pgrp (if the value is negative). This implies that only one process at a time can do icky I/O on a given object.  Whenever the I/O server is sending sig_post messages to all the io_async users, if the O_ASYNC bit is set, the server should also send a signal to the owning pid/pgrp.  The ID port for this call should be different from all the io_async ID ports given to users.  Users may find out what ID port the server uses for this by calling io_get_icky_async_id.

Information Queries
-------------------

Users may call io_stat to find out information about the I/O object.  Most of the fields of a struct stat are meaningful only for files.  All objects, however, must support the fields ST_FSTYPE, ST_FSID, ST_INO, ST_ATIM, ST_MTIM, ST_CTIM, and ST_BLKSIZE.

ST_FSTYPE, ST_FSID, and ST_INO must be unique for the underlying object across the entire system.

ST_ATIM holds the timestamp of the system clock at the last time the object was read with io_read.

ST_MTIM holds the timestamp of the system clock at the last time the object was written with io_write.

Other appropriate operations may update the ATIM and the MTIM as well; both the file and socket interfaces specify such operations.

ST_CTIM holds the timestamp of the system clock at the last time permanent meta-data associated with the object was changed.  The exact operations which cause such an update are server-dependent, but must include the creation of the object.

The server is permitted to delay the actual update of these times until stat is called; before the server stores the times on permanent media (if it ever does so) it should update them if necessary.

ST_BLKSIZE gives the optimal I/O size in bytes for io_read and io_write; users should endeavor to read and write amounts which are multiples of the optimal size, and to use offsets which are multiples of the optimal size.

In addition, objects which are seekable should set ST_SIZE to the current file size as in the description of the O_APPEND flag (see Open Modes).

The ST_UID and ST_GID fields are unrelated to the ``owner'' as described above for icky async I/O.

Users may find out the version of the server they are talking to by calling io_server_version; this should return strings and integers describing the version number of the server, as well as its name.

Mapped Data
-----------

Servers may optionally implement the io_map call.  The ports returned by io_map must implement the external pager kernel interface (see Pager Library) and be suitable as arguments to vm_map.

Seekable objects must allow access from zero up to (but not including) the current file size as described for O_APPEND (see Open Modes).  Whether they provide access beyond such a point is server-dependent; in addition, the meaning of accessing a non-seekable object is server-dependent.

Files
*****

A file is traditionally thought of as a quantity of disk storage.  In the Hurd, files are an extension of the I/O interface, but they do not necessarily correspond to disk storage.

Every file in the Hurd is represented by a port, which is connected to the server that manages the file.  When a client wants to operate on a file, it makes RPC requests via a file port to its server process, which is commonly called a translator.

Translators
===========

The Hurd filesystem allows you to set translators on any file or directory that you own.  A translator is any Hurd server which provides the basic filesystem interface.  Translated nodes are somewhat like a cross between Unix symbolic links and mount points.

Whenever a program tries to access the contents of a translated node, the filesystem server redirects the request to the appropriate translator (starting it if necessary).  Then, the new translator services the client's request.  The GNU C library makes this behaviour seamless from the client's perspective, so that standard Unix programs behave correctly under the Hurd.

Translators run with the privileges of the translated node's owner, so they cannot be used to compromise the security of the system.  This also means that any user can write their own translators, and provide other users with arbitrary filesystem-structured data, regardless of the data's actual source. Other chapters in this manual describe existing translators, and how you can modify them or write your own.

The standard Hurd filesystem servers are constantly evolving to provide innovative features that users want.  Here are a few examples of existing translators:

  * Disk-based filesystem formats, such as ext2fs, ufs, and iso9660fs (see Stored Filesystems).

  * Network filesystems, such as nfs and ftpfs (see Distributed Filesystems).

  * Single files with dynamic content, such as FIXME: we need a good example.

  * Hurd servers which translate rendezvous filesystem nodes in standard locations, so that other programs can easily find them and use server-specific interfaces.  For example, pflocal implements the filesystem interfaces, but it also provides a special Unix-domain socket RPC interface (FIXME xref).  Programs can fetch a port to this translator simply by calling file_name_lookup (FIXME xref) on /servers/socket/1 (The number 1 corresponds to the PF_LOCAL C library socket domain constant.), then use Unix socket-specific RPCs on that port, rather than adhering to the file protocol.

This section focuses on the generic programs that you need to understand in order to use existing translators.  Many other parts of this manual describe how you can write your own translators.

Invoking settrans
-----------------

The settrans program allows you to set a translator on a file or directory.  By default, the passive translator is set (see the --passive option).

The settrans program has the following synopsis:

     settrans [OPTION]... NODE [TRANSLATOR ARG...]

where TRANSLATOR is the absolute filename of the new translator program.  Each ARG is passed to TRANSLATOR when it starts. If TRANSLATOR is not specified, then settrans clears the existing translator rather than setting a new one.

settrans accepts the following options:

-a
--active
     Set NODE's active translator.  Active translators are started immediately and are not persistent: if the system is rebooted then they are lost.

-c
--create
     Create NODE as a zero-length file if it doesn't already exist.

-L
--dereference
     If NODE is already translated, stack the new translator on top of it (rather than replacing the existing translator).

--help
     Display a brief usage message, then exit.

-p
--passive
     Set NODE's passive translator.  Passive translators are only activated by the underlying filesystem when clients try to use the NODE, and they shut down automatically after they are no longer active in order to conserve system resources.

     Passive translators are stored on the underlying filesystem media, and so they persist between system reboots.  Not all filesystems support passive translators, due to limitations in their underlying media. Consult the filesystem-specific documentation to see if they are supported.

     If you are setting the passive translator, and NODE already has an active translator, then the following options apply:

     -g
     --goaway
          Tell the active translator to go away.  In this case, the following additional options apply:

          -f
          --force
               If the active translator doesn't go away, then force it.

          -S
          --nosync
               Don't flush its contents to disk before terminating.

          -R
          --recursive
               Shut down all of the active translator's children, too.

     -k
     --keep-active
          Leave the existing active translator running.  The new translator will not be started unless the active translator has stopped.

-P
--pause
     When starting an active translator, prompt and wait for a newline on standard input before completing the startup handshake.  This is useful when debugging a translator, as it gives you time to start the debugger.

-t SEC
--timeout=SEC
     If the translator does not start up in SEC seconds (the default is 60), then return an error; if SEC is 0, then never timeout.

--version
     Output program version information and exit.

-x
--exclusive
     Only set the translator if there is none already.

Invoking showtrans
------------------

The showtrans program allows you to show the passive translator setting on a file system node.

The showtrans program has the following synopsis:

     showtrans [OPTION]... FILE...

showtrans accepts the following options:

-p
--prefix
     Always display FILENAME: before translators.

-P
--no-prefix
     Never display FILENAME: before translators.

-s
--silent
     No output; useful when checking error status.

-t
--translated
     Only display files that have translators.

Invoking mount
--------------

Invoking fsysopts
-----------------

The fsysopts program allows you to retrieve or set command line options for running translator FILESYS.

The fsysopts program has the following synopsis:

     fsysopts [OPTION...] FILESYS [FS_OPTION...]

fsysopts accepts the following options:

-L
--dereference
     If FILESYS is a symbolic link, follow it.

-R
--recursive
     Pass these options to any child translators.

The legal values for FS_OPTION depends on FILESYS, but some common ones are:

--readonly

--writable

--remount

--sync[=INTERVAL]

--nosync

If no options are supplied, FILESYS' current options are printed.

The options passed as FS_OPTIONs are meant to augment or change those which are already set, they're not meant to completely replace the existing command line options.  For example, passing --readonly to a file system server will change the server from writable to read-only, but will not touch the used backing store.  Passing --address=new_IP to /servers/socket/2 will change the local IP address to NEW_IP, but will not touch the interface, netmask and gateway settings.

Trivfs Library
==============

Certain translators do not need to be very complex, because they represent a single file rather than an entire directory hierarchy.  The trivfs library, which is declared in <hurd/trivfs.h>, does most of the work of implementing this kind of translator.  This library requires the iohelp and ports libraries.

Trivfs Startup
--------------

In order to use the trivfs library, you will need to define the appropriate callbacks (see Trivfs Callbacks).  As with all Hurd servers, your trivfs-based translator should first parse any command-line options, in case the user is just asking for help.  Trivfs uses argp (see Argp (The GNU C Library Reference Manual)) for parsing command-line arguments.

Your translator should redefine the following functions and variables as necessary, and then call argp_parse with the relevant arguments:

 -- Variable: extern struct argp * trivfs_runtime_argp
     If this is defined or set to an argp structure, it will be used by the default trivfs_set_options to handle runtime options parsing. Redefining this is the normal way to add option parsing to a trivfs program.

 -- Function: error_t trivfs_set_options (struct trivfs_control *FSYS, char *ARGZ, size_t ARGZ_LEN)
     Set runtime options for FSYS to ARGZ and ARGZ_LEN. The default definition for this routine simply uses TRIVFS_RUNTIME_ARGP (supplying FSYS as the argp input field).

 -- Function: error_t trivfs_append_args (struct trivfs_control *FSYS, char **ARGZ, size_t *ARGZ_LEN)
     Append to the malloced string *ARGZ of length *ARGZ_LEN a NUL-separated list of the arguments to this translator.

After your translator parses its command-line arguments, it should fetch its bootstrap port by using task_get_bootstrap_port.  If this port is MACH_PORT_NULL, then your program wasn't started as a translator.  Otherwise, you can use the bootstrap port to create a new control structure (and advertise its port) with trivfs_startup:

 -- Function: error_t trivfs_startup (mach_port_t BOOTSTRAP, int FLAGS, struct port_class *CONTROL_CLASS, struct port_bucket *CONTROL_BUCKET, struct port_class *PROTID_CLASS, struct port_bucket *PROTID_BUCKET, struct trivfs_control **CONTROL)
 -- Function: error_t trivfs_create_control (mach_port_t BOOTSTRAP, struct port_class *CONTROL_CLASS, struct port_bucket *CONTROL_BUCKET, struct port_class *PROTID_CLASS, struct port_bucket *PROTID_BUCKET, struct trivfs_control **CONTROL)
     trivfs_startup creates a new trivfs control port, advertises it to the underlying node BOOTSTRAP with fsys_startup, returning the results of this call, and places its control structure in *CONTROL.  trivfs_create_control does the same thing, except it doesn't advertise the control port to the underlying node.  CONTROL_CLASS and CONTROL_BUCKET are passed to libports to create the control port, and PROTID_CLASS and PROTID_BUCKET are used when creating ports representing opens of this node; any of these may be zero, in which case an appropriate port class/bucket is created.  If CONTROL is non-null, the trivfs control port is returned in it.  FLAGS (a bitmask of the appropriate O_* constants) specifies how to open the underlying node.

If you did not supply zeros as the class and bucket arguments to trivfs_startup, you will probably need to use the trivfs port management functions (see Trivfs Ports).

Once you have successfully called trivfs_startup, and have a pointer to the control structure stored in, say, the FSYS variable, you are ready to call one of the ports_manage_port_operations_* functions using FSYS->pi.bucket and trivfs_demuxer.  This will handle any incoming filesystem requests, invoking your callbacks when necessary.

 -- Function: int trivfs_demuxer (mach_msg_header_t *INP, mach_msg_header_t *OUTP)
     Demultiplex incoming libports messages on trivfs ports.

The following functions are not usually necessary, but they allow you to use the trivfs library even when it is not possible to turn message-handling over to trivfs_demuxer and libports:

 -- Function: struct trivfs_control * trivfs_begin_using_control (mach_port_t PORT)
 -- Function: struct trivfs_protid * trivfs_begin_using_protid (mach_port_t PORT)
     These functions can be used as intran functions for a MiG port type to have the stubs called with either the control or protid pointer.

 -- Function: void trivfs_end_using_control (struct trivfs_control *PORT)
 -- Function: void trivfs_end_using_protid (struct trivfs_protid *PORT)
     These can be used as `destructor' functions for a MiG port type, to have the stubs called with the control or protid pointer.

 -- Function: error_t trivfs_open (struct trivfs_control *FSYS, struct iouser *USER, unsigned FLAGS, mach_port_t REALNODE, struct trivfs_protid **CRED)
     Return a new protid (that is, a port representing an open of this node) pointing to a new peropen in CRED, with REALNODE as the underlying node reference, with the given identity, and open flags in FLAGS.  CNTL is the trivfs control object.

 -- Function: error_t trivfs_protid_dup (struct trivfs_protid *CRED, struct trivfs_protid **DUP)
     Return a duplicate of CRED in DUP, sharing the same peropen and hook.  A non-null protid HOOK indicates that TRIVFS_PEROPEN_CREATE_HOOK created this protid (see Trivfs Options).

 -- Function: error_t trivfs_set_atime (struct trivfs_control *CNTL)
 -- Function: error_t trivfs_set_mtime (struct trivfs_control *CNTL)
     Call these to set atime or mtime for the node to the current time.

Trivfs Callbacks
----------------

Like several other Hurd libraries, libtrivfs requires that you define a number of application-specific callback functions and configuration variables.  You must define the following variables and functions:

 -- Variable: extern int trivfs_fstype
 -- Variable: extern int trivfs_fsid
     These variables are returned in the ST_FSTYPE and ST_FSID fields of struct stat.  TRIVFS_FSTYPE should be chosen from the FSTYPE_* constants found in <hurd/hurd_types.h>.

 -- Variable: extern int trivfs_allow_open
     Set this to some bitwise OR combination of O_READ, O_WRITE, and O_EXEC; trivfs will only allow opens of the specified modes.

 -- Variable: extern int trivfs_support_read
 -- Variable: extern int trivfs_support_write
 -- Variable: extern int trivfs_support_exec
     Set these to nonzero if trivfs should allow read, write, or execute of the file.  These variables are necessary because TRIVFS_ALLOW_OPEN is used only to validate opens, not actual operations.

 -- Function: void trivfs_modify_stat (struct trivfs_protid *CRED, struct stat *STBUF)
     This should modify a struct stat (as returned from the underlying node) for presentation to callers of io_stat.  It is permissible for this function to do nothing, but it must still be defined.

 -- Function: error_t trivfs_goaway (struct trivfs_control *CNTL, int FLAGS)
     This function is called when someone wants the filesystem CNTL to go away.  FLAGS are from the set FSYS_GOAWAY_* found in <hurd/hurd_types.h>.

Trivfs Options
--------------

The functions and variables described in this subsection already have default definitions in libtrivfs, so you are not forced to define them; rather, they may be redefined on a case-by-case basis.

 -- Variable: extern struct port_class * trivfs_protid_portclasses []
 -- Variable: extern int trivfs_protid_nportclasses
 -- Variable: extern struct port_class * trivfs_cntl_portclasses []
 -- Variable: extern int trivfs_cntl_nportclasses
     If you define these, they should be vectors (and the associated sizes) of port classes that will be translated into control and protid pointers for passing to RPCs, in addition to those passed to or created by trivfs_create_control (or trivfs_startup), which will automatically be recognized.

 -- Variable: error_t (*trivfs_check_open_hook) (struct trivfs_control *CNTL, struct iouser *USER, int FLAGS)
     If this variable is non-zero, it will be called every time an open happens. USER and FLAGS are from the open; CNTL identifies the node being opened.  This call need not check permissions on the underlying node.  This call can block as necessary, unless O_NONBLOCK is set in FLAGS.  Any desired error can be returned, which will be reflected to the user and will prevent the open from succeeding.

 -- Variable: error_t (*trivfs_protid_create_hook) (struct trivfs_protid *PROT)
 -- Variable: error_t (*trivfs_peropen_create_hook) (struct trivfs_peropen *PEROP)
     If these variables are non-zero, they will be called every time a new protid or peropen structure is created and initialized.

 -- Variable: void (*trivfs_protid_destroy_hook) (struct trivfs_protid *PROT)
 -- Variable: void (*trivfs_peropen_destroy_hook) (struct trivfs_peropen *PEROP)
     If these variables is non-zero, they will be called every time a protid or peropen structure is about to be destroyed.

 -- Variable: error_t (*trivfs_getroot_hook) (struct trivfs_control *CNTL, mach_port_t REPLY_PORT, mach_msg_type_name_t REPLY_PORT_TYPE, mach_port_t DOTDOT, uid_t *UIDS, u_int NUIDS, uid_t *GIDS, u_int NGIDS, int FLAGS, retry_type *DO_RETRY, char *RETRY_NAME, mach_port_t *NODE, mach_msg_type_name_t *NODE_TYPE)
     If this variable is set, it will be called by trivfs_S_fsys_getroot before any other processing takes place.  If the return value is EAGAIN, normal trivfs getroot processing continues, otherwise the RPC returns with that return value.

Trivfs Ports
------------

If you choose to allocate your own trivfs port classes and buckets, the following functions may come in handy:

 -- Function: error_t trivfs_add_port_bucket (struct port_bucket **BUCKET)
     Add the port bucket *BUCKET to the list of dynamically- allocated port buckets; if *BUCKET is zero, an attempt is made to allocate a new port bucket, which is then stored in *BUCKET.

 -- Function: void trivfs_remove_port_bucket (struct port_bucket *BUCKET)
     Remove the previously added dynamic port bucket BUCKET, freeing it if it was allocated by trivfs_add_port_bucket.

 -- Function: error_t trivfs_add_control_port_class (struct port_class **CLASS)
 -- Function: error_t trivfs_add_protid_port_class (struct port_class **CLASS)
     Add the port class *CLASS to the list of control or protid port classes recognized by trivfs; if *CLASS is zero, an attempt is made to allocate a new port class, which is stored in *CLASS.

 -- Function: void trivfs_remove_control_port_class (struct port_class *CLASS)
 -- Function: void trivfs_remove_protid_port_class (struct port_class *CLASS)
     Remove the previously added dynamic control or protid port class CLASS, freeing it if it was allocated by trivfs_add_control_port_class or trivfs_add_protid_port_class.

Even if you do not use the above allocation functions, you may still be able to use the default trivfs cleanroutines:

 -- Function: void trivfs_clean_cntl (void *PORT)
 -- Function: void trivfs_clean_protid (void *PORT)
     These functions should be installed as libports cleanroutines for control port classes and protid port classes, respectively.

Fshelp Library
==============

The fshelp library implements various things that are useful to most implementors of the file protocol.  It presumes that you are using the iohelp library as well.  libfshelp is divided into separate facilities which may be used independently.  These functions are declared in <hurd/fshelp.h>.

Passive Translator Linkage
--------------------------

These routines are self-contained and start passive translators, returning the control port.  They do not require multithreading or the ports library.

 -- Typedef: typedef error_t (*fshelp_open_fn_t) (int FLAGS, file_t *NODE, mach_msg_type_name_t *NODE_TYPE)
     A callback used by the translator starting functions. Given some open flags, opens the appropriate file, and returns the node port.

 -- Function: error_t fshelp_start_translator_long (fshelp_open_fn_t UNDERLYING_OPEN_FN, char *NAME, char *ARGZ, int ARGZ_LEN, mach_port_t *FDS, mach_msg_type_name_t FDS_TYPE, int FDS_LEN, mach_port_t *PORTS, mach_msg_type_name_t PORTS_TYPE, int PORTS_LEN, int *INTS, int INTS_LEN, int TIMEOUT, fsys_t *CONTROL)
     Start a passive translator NAME with arguments ARGZ (length ARGZ_LEN).  Initialize the initports to PORTS (length PORTS_LEN), the initints to INTS (length INTS_LEN), and the file descriptor table to FDS (length FDS_LEN). Return the control port in *CONTROL.  If the translator doesn't respond or die in TIMEOUT milliseconds (if TIMEOUT is greater than zero), return an appropriate error.  If the translator dies before responding, return EDIED.

 -- Function: error_t fshelp_start_translator (fshelp_open_fn_t UNDERLYING_OPEN_FN, char *NAME, char *ARGZ, int ARGZ_LEN, int TIMEOUT, fsys_t *CONTROL)
     Same as fshelp_start_translator_long, except the initports and ints are copied from our own state, FD[2] is copied from our own stderr, and the other fds are cleared.  For full-service filesystems, it is almost always wrong to use fshelp_start_translator, because the current working directory of the translator will not then be as normally expected.  (Current working directories of passive translators should be the directory they were found in.)  In fact, full-service filesystems should usually start passive translators as a side-effect of calling fshelp_fetch_root (see Active Translator Linkage).

Active Translator Linkage
-------------------------

These routines implement the linkage to active translators needed by any filesystem which supports them.  They require the threads library and use the passive translator routines above, but they don't require the ports library at all.

This interface is complex, because creating the ports and state necessary for start_translator_long is expensive.  The caller to fshelp_fetch_root should not need to create them on every call, since usually there will be an existing active translator.

 -- Function: void fshelp_transbox_init (struct transbox *TRANSBOX, struct mutex *LOCK, void *COOKIE)
     Initialize a transbox, which contains state information for active translators.

 -- Typedef: typedef error_t (*fshelp_fetch_root_callback1_t) (void *COOKIE1, void *COOKIE2, uid_t *UID, gid_t *GID, char **ARGZ, size_t *ARGZ_LEN)
     This routine is called by fshelp_fetch_root to fetch more information.  Return the owner and group of the underlying translated file in *UID and *GID; point *ARGZ at the entire passive translator specification for the file (setting *ARGZ_LEN to the length).  If there is no passive translator, then return ENOENT.  COOKIE1 is the cookie passed in fshelp_transbox_init.  COOKIE2 is the cookie passed in the call to fshelp_fetch_root.

 -- Typedef: typedef error_t (*fshelp_fetch_root_callback2_t) (void *COOKIE1, void *COOKIE2, int FLAGS, mach_port_t *UNDERLYING, mach_msg_type_name_t *UNDERLYING_TYPE)
     This routine is called by fshelp_fetch_root to fetch more information.  Return an unauthenticated node for the file itself in *UNDERLYING and *UNDERLYING_TYPE (opened with FLAGS).  COOKIE1 is the cookie passed in fshelp_transbox_init.  COOKIE2 is the cookie passed in the call to fshelp_fetch_root.

 -- Function: error_t fshelp_fetch_root (struct transbox *TRANSBOX, void *COOKIE, file_t DOTDOT, struct iouser *USER, int FLAGS, fshelp_fetch_root_callback1_t CALLBACK1, fshelp_fetch_root_callback2_t CALLBACK2, retry_type *RETRY, char *RETRYNAME, mach_port_t *ROOT)
     Fetch the root from TRANSBOX.  DOTDOT is an unauthenticated port for the directory in which we are looking; USER specifies the ids of the user responsible for the call.  FLAGS are as for dir_lookup (but O_CREAT and O_EXCL are not meaningful and are ignored).  The transbox lock (as set by fshelp_transbox_init) must be held before the call, and will be held upon return, but may be released during the operation of the call.

 -- Function: int fshelp_translated (struct transbox *BOX)
     Return true if and only if there is an active translator on this box.

 -- Function: error_t fshelp_set_active (struct transbox *BOX, fsys_t NEWACTIVE, int EXCL)
     Atomically replace the existing active translator port for this box with NEWACTIVE.  If EXCL is non-zero then don't modify an existing active transbox; return EBUSY instead.

 -- Function: error_t fshelp_fetch_control (struct transbox *BOX, mach_port_t *CONTROL)
     Fetch the control port to make a request on it.  It's a bad idea to use fsys_getroot with the result; use fshelp_fetch_root instead.

 -- Function: void fshelp_drop_transbox (struct transbox *BOX)
     Clean transbox state so that deallocation or reuse is possible.

Fshelp Locking
--------------

The flock call is in flux, as the current Hurd interface (as of version VERSION) is not suitable for implementing the POSIX record-locking semantics.

Fshelp Permissions
------------------

These functions are designed to aid with user permission checking.  It is a good idea to use these routines rather than to roll your own, so that Hurd users see consistent handling of file and directory permission bits.

 -- Function: error_t fshelp_isowner (struct stat *ST, struct iouser *USER)
     Check to see whether USER should be considered the owner of the file identified by ST.  If so, return zero; otherwise return an appropriate error code.

 -- Function: error_t fshelp_access (struct stat *ST, int OP, struct iouser *USER)
     Check to see whether the user USER can operate on the file identified by ST.  OP is one of S_IREAD, S_IWRITE, and S_IEXEC.  If the access is permitted, return zero; otherwise return an appropriate error code.

 -- Function: error_t fshelp_checkdirmod (struct stat *DIR, struct stat *ST, struct iouser *USER)
     Check to see whether USER is allowed to modify DIR with respect to existing file ST.  If there is no existing file, then ST should be set to zero.  If the access is permissible, return zero; otherwise return an appropriate error code.

Fshelp Misc
-----------

The following functions are completely standalone:

 -- Function: error_t fshelp_delegate_translation (char *SERVER_NAME, mach_port_t REQUESTOR, char **ARGV)
     Try to hand off responsibility from a translator to the server located on the node SERVER_NAME.  REQUESTOR is the translator's bootstrap port, and ARGV is the command line.  If SERVER_NAME is null, then a name is concocted by prepending _servers to argv[0] .

 -- Function: error_t fshelp_exec_reauth (int SUID, uid_t UID, int SGID, gid_t GID, auth_t AUTH, error_t (*GET_FILE_IDS) (struct idvec *UIDS, struct idvec *GIDS), mach_port_t *PORTS, mach_msg_type_number_t NUM_PORTS, mach_port_t *FDS, mach_msg_type_number_t NUM_FDS, int *SECURE)
     If SUID or SGID is true, adds UID and/or GID respectively to the authentication in PORTS[INIT_PORT_AUTH], and replaces it with the result. All the other ports in PORTS and FDS are then reauthenticated, using any privileges available through AUTH.  If the auth port in PORTS[INIT_PORT_AUTH] is bogus, and GET_FILE_IDS is non-null, it is called to get a list of uids and gids from the file to use as a replacement.  If SECURE is non-null and any added ids are new, then the variable it points to is set to nonzero, otherwise zero.  If either the uid or gid case fails, then the other may still apply.

 -- Function: error_t fshelp_get_identity (struct port_bucket *BUCKET, ino_t FILENO, mach_port_t *PT)
     Return an identity port in *PT for the node numbered FILENO, suitable for returning from io_identity; exactly one send right must be created from the returned value.  FILENO should be the same value returned as the FILENO out-parameter in io_identity, and in the enclosing directory (except for mount points), and in the st_ino stat field.  BUCKET should be a libports port bucket; fshelp requires the caller to make sure port operations (for no-senders notifications) are used.

 -- Function: error_t fshelp_return_malloced_buffer (char *BUF, size_t LEN, char **RBUF, mach_msg_type_number_t *RLEN)
     Put data from the malloced buffer BUF, LEN bytes long, into RBUF (which is RLEN bytes long), suitable for returning from an RPC.  If LEN is greater than zero, BUF is freed, regardless of whether an error is returned or not.

 -- Function: error_t fshelp_set_options (struct argp *ARGP, int FLAGS, char *ARGZ, size_t ARGZ_LEN, void *INPUT)
     Invoke argp_parse in the standard way, with data from ARGZ and ARGZ_LEN.

 -- Function: void fshelp_touch (struct stat *ST, unsigned WHAT, volatile struct mapped_time_value *MAPTIME)
     Change the stat times of NODE as indicated by WHAT to the current time.  WHAT is a bitmask of one or more of the TOUCH_ATIME, TOUCH_MTIME, and TOUCH_CTIME constants.

File Interface
==============

This section documents the interface for operating on files.

File Overview
-------------

The file interface is a superset of the I/O interface (see I/O Interface).  Servers which provide the file interface are required to support the I/O interface as well.  All objects reachable in the filesystem are expected to provide the file interface, even if they do not contain data.  (The trivfs library makes it easy to do so for ordinary sorts of cases.  See Trivfs Library.)

The interface definitions for the file interface are found in <hurd/fs.defs>.

Files have various pieces of status information which are returned by io_stat (see Information Queries).  Most of this status information can be directly changed by various calls in the file interface; some of it should vary implicitly as the contents of the file change.

Many of these calls have general rules associated with them describing how security and privilege should operate.  The diskfs library (see Diskfs Library) implements these rules for stored filesystems. These rules have also been implemented in the fshelp library (see Fshelp Library).  Trivfs-based servers generally have no need to implement these rules at all.

In special cases, there may be a reason to implement a different security check from that specified here, or to implement a call to do something slightly different.  But such cases must be carefully considered; make sure that you will not confuse innocent user programs through excessive cleverness.

If some operation cannot be implemented (for example, chauthor over FTP), then the call should return EOPNOTSUPP.  If it is merely difficult to implement a call, it is much better to figure out a way to implement it as a series of operations rather than to return errors to the user.

Changing Status
---------------

There are several RPCs available for users to change much of the status information associated with a file.  (The information is returned by the io_stat RPC; see Information Queries.)

All these operations are restricted to root and the owner of the file. When attempted by another user, they should return EPERM.

The file_chown RPC changes the owner and group of the file.  Only root should be able to change the owner, and changing the group to a group the caller is not in should also be prohibited.  Violating either of these conditions should return EPERM.

The file_chauthor RPC changes the author of the file.  It should be legitimate to change the author to any value without restriction.

The file_chmod RPC changes the file permission mode bits.

The file_chflags RPC changes the flags of the file.  It should be legitimate to change the flags to any value without restriction.  No standard meanings have been assigned to the flags yet, but we intend to do so.  Do not assume that the flags format we choose will map identically to that of some existing filesystem format.

The file_utimes RPC changes the ATIME and MTIME of the file.  Making this call must cause the CTIME to be updated as well, even if no actual change to either the MTIME or the ATIME occurs.

The file_set_size RPC is special; not only does it change the status word specifying the size of the file, but it also changes the actual contents of the file.  If the file size is being reduced it should release secondary storage associated with the previous contents of the file.  If the file is being extended, the new region added to the file must be zero-filled.  Unlike the other RPCs in this section, file_set_size should be permitted to any user who is allowed to write the file.

Program Execution
-----------------

Execution of programs on the Hurd is done through fileservers with the file_exec RPC.  The fileserver is expected to verify that the user is allowed to execute the file, make whatever modifications to the ports are necessary for setuid execution, and then invoke the standard execserver found on /servers/exec.

This section specifically addresses what fileservers are expected to do, with minimal attention to the other parts of the process.  See Running Programs, for more general information.

The file must be opened for execution; if it is not, EBADF should be returned.  In addition, at least one of the execute bits must be on.  A failure of this check should result in EACCES---not ENOEXEC.  It is not proper for the fileserver ever to respond to the file_exec RPC with ENOEXEC.

If either the setuid or setgid bits are set, the server needs to construct a new authentication handle with the additional new ID's. Then all the ports passed to file_exec need to be reauthenticated with the new handle.  If the fileserver is unable to make the new authentication handle (for example, because it is not running as root) it is not acceptable to return an error; in such a case the server should simply silently fail to implement the setuid/setgid semantics.

If the setuid/setgid transformation adds a new uid or gid to the user's authentication handle that was not previously present (as opposed to merely reordering them), then the EXEC_SECURE and EXEC_NEWTASK flags should both be added in the call to exec_exec.

The server then needs to open a new port onto the executed file which will not share any file pointers with the port the user passed in, opened with O_READ.  Finally, all the information (mutated appropriately for setuid/setgid) should be sent to the execserver with exec_exec.  Whatever error code exec_exec returns should returned to the caller of file_exec.

File Locking
------------

The flock call is in flux, as the current Hurd interface (as of version VERSION) is not suitable for implementing the POSIX record-locking semantics.

You should ignore the file_lock and file_lock_stat calls until the new record-locking interface is implemented.

File Frobbing
-------------

FIXME: Other active calls on files

file_sync

file_getfh

file_getlinknode

file_check_access

These manipulate meta-information:

file_reparent

file_statfs

file_syncfs

file_getcontrol

file_get_storage_info

file_get_fs_options

Opening Files
-------------

FIXME: Looking up files in directories

dir_lookup

dir_readdir

Modifying Directories
---------------------

 -- Function: kern_return_t dir_mkfile (file_t DIRECTORY, int FLAGS, mode_t MODE, mach_port_t *NEWNODE)
     Create a new file in DIRECTORY without linking it into the filesystem.  You still must have write permission on the specified directory, even though it will not actually be written.

     The function returns a port to the new file in *NEWNODE.  Flags are the same as for dir_lookup, but O_CREAT and O_TRUNC are assumed even if not specified.

 -- Function: kern_return_t dir_mkdir (file_t DIRECTORY, char *NAME, mode_t MODE)
     Create a new directory named NAME in DIRECTORY with permission specified by MODE.

 -- Function: kern_return_t dir_rmdir (file_t DIRECTORY, char *NAME)
     Remove the directory named NAME from DIRECTORY.

 -- Function: kern_return_t dir_unlink (file_t DIRECTORY, char *NAME)
     Remove the non-directory node NAME from DIRECTORY.

 -- Function: kern_return_t dir_link (file_t DIRECTORY, file_t FILE, char *NAME, int EXCL)
     Create a hard link in DIRECTORY.  If EXCL is set and NAME already exists in DIRECTORY, then this function will fail.  If EXCL is not set and NAME already exists the old file named NAME will be unlinked.  If DIRECTORY and FILE are not on the same filesystem, then dir_link might fail with EXDEV.

 -- Function: kern_return_t dir_rename (file_t OLDDIRECTORY, char *OLDNAME, file_t NEWDIRECTORY, char *NEWNAME, int EXCL)
     Move the node OLDNAME in OLDDIRECTORY to the node NEWNAME in NEWDIRECTORY.  If EXCL is set and NEWNAME already exists in NEWDIRECTORY, then this function will fail.  If EXCL is not set and NEWNAME already exists, the old file named NEWNAME will be unlinked.  If OLDDIRECTORY and NEWDIRECTORY are not on the same filesystem, then dir_rename might fail with EXDEV.

Notifications
-------------

FIXME: File and directory change callbacks

File change notifications are not yet implemented, but directory notifications are.

file_notice_changes

dir_notice_changes

File Translators
----------------

FIXME: How to set and get translators

file_set_translator

file_get_translator

file_get_translator_cntl

Filesystem Interface
====================

The filesystem interface (described in <hurd/fsys.defs>) is supported by translator control ports.

FIXME: finish

Special Files
*************

In Unix, any file that does not act as a general-purpose unit of storage is called a special file.  These are FIFOs, Unix-domain sockets, and device nodes.  In the Hurd, there is no need for the ``special file'' distinction, since they are implemented by translators, just as regular files are.

Nevertheless, the Hurd maintains this distinction, in order to provide backward compatibility for Unix programs (which do not know about translators).  Studying the implementation of Hurd special files is a good way to introduce the idea of translators to people who are familiar with Unix.

This chapter does not discuss /dev/zero or any of the microkernel-based devices, since these are translated by the generalized storeio server (FIXME xref).

FIXME: finish

fifo
====

ifsock
======

magic
=====

null
====

FIXME: a chapter on libtreefs and libdirmgt will probably go here

Stores
******

A store is a fixed-size block of storage, which can be read and perhaps written to.  A store is more general than a file: it refers to any type of storage such as devices, files, memory, tasks, etc.  Stores can also be representations of other stores, which may be combined and filtered in various ways.

storeinfo, storecat, storeread
==============================

storeio
=======

FIXME: finish

Store Library
=============

The store library (which is declared in <hurd/store.h>) implements many different backends which support the store abstraction. Hurd programs use libstore so that new storage types can be implemented with minimum impact.

Store Arguments
---------------

FIXME: describe startup sequence

 -- Structure: struct store_parsed
     The result of parsing a store, which should be enough information to open it, or return the arguments.

 -- Structure: struct store_argp_params { struct store_parsed *RESULT; const char *DEFAULT_TYPE; const struct store_class *const *CLASSES; }
     This is the structure used to pass args back and forth from STORE_ARGP.  RESULT is the resulting parsed result.  If --store-type isn't specified, then DEFAULT_TYPE should be used as the store type; zero is equivalent to "query". CLASSES is set of classes used to validate store types and argument syntax.

 -- Variable: extern struct argp store_argp
     This is an argument parser that may be used for parsing a simple command line specification for stores.  The accompanying input parameter must be a pointer to a struct store_argp_params.

 -- Function: void store_parsed_free (struct store_parsed *PARSED)
     Free all resources used by PARSED.

 -- Function: error_t store_parsed_open (const struct store_parsed *PARSED, int FLAGS, struct store **STORE)
     Open the store specified by PARSED, and return it in STORE.

 -- Function: error_t store_parsed_append_args (const struct store_parsed *PARSED, char **ARGZ, size_t *ARGZ_LEN)
     Add the arguments used to create PARSED to ARGZ and ARGZ_LEN.

 -- Function: error_t store_parsed_name (const struct store_parsed *PARSED, char **NAME)
     Make an option string describing PARSED, and return it in malloced storage in NAME.

Store Management
----------------

The following functions provide basic management of stores:

 -- Function: error_t store_create (file_t SOURCE, int FLAGS, const struct store_class *const *CLASSES, struct store **STORE)
     Return a new store in STORE, which refers to the storage underlying SOURCE.  CLASSES is used to select classes specified by the provider; if zero, STORE_STD_CLASSES is used. FLAGS is set with store_set_flags, with the exception of STORE_INACTIVE, which merely indicates that no attempt should be made to activate an inactive store; if STORE_INACTIVE is not specified, and the store returned for SOURCE is inactive, an attempt is made to activate it (failure of which causes an error to be returned). A reference to SOURCE is created (but may be destroyed with store_close_source).

     It is usually better to use a specific store open or create function such as store_open (see Store Classes), since they are tailored to the needs of a specific store.  Generally, you should only use store_create if you are defining your own store class, or you need options that are not provided by a more specific store creation function.

 -- Function: void store_close_source (struct store *STORE)
     If STORE was created using store_create, remove the reference to the source from which it was created.

 -- Function: void store_free (struct store *STORE)
     Clean up and deallocate STORE's underlying stores.

 -- Structure: struct store_run { store_offset_t START, LENGTH; }
     A struct store_run represents a contiguous region in a store's address range.  These are used to designate active portions of a store. If START is -1, then the region is a hole (it is zero-filled and doesn't correspond to any real addresses).

 -- Function: error_t store_set_runs (struct store *STORE, const struct store_run *RUNS, size_t NUM_RUNS)
     Set STORE's current runs list to (a copy of) RUNS and NUM_RUNS.

 -- Function: error_t store_set_children (struct store *STORE, struct store *const *CHILDREN, size_t NUM_CHILDREN)
     Set STORE's current children to (a copy of) CHILDREN and NUM_CHILDREN (note that just the vector CHILDREN is copied, not the actual children).

 -- Function: error_t store_children_name (const struct store *STORE, char **NAME)
     Try to come up with a name for the children in STORE, combining the names of each child in a way that could be used to parse them with store_open_children.  This is done heuristically, and so may not succeed.  If a child doesn't have a name, EINVAL is returned.

 -- Function: error_t store_set_name (struct store *STORE, const char *NAME)
     Sets the name associated with STORE to a copy of NAME.

 -- Function: error_t store_set_flags (struct store *STORE, int FLAGS)
     Add FLAGS to STORE's currently set flags.

 -- Function: error_t store_clear_flags (struct store *STORE, int FLAGS)
     Remove FLAGS from STORE's currently set flags.

 -- Function: error_t store_set_child_flags (struct store *STORE, int FLAGS)
     Set FLAGS in all children of STORE, and if successful, add FLAGS to STORE's flags.

 -- Function: error_t store_clear_child_flags (struct store *STORE, int FLAGS)
     Clear FLAGS in all children of STORE, and if successful, remove FLAGS from STORE's flags.

 -- Function: int store_is_securely_returnable (struct store *STORE, int OPEN_FLAGS)
     Returns true if STORE can safely be returned to a user who has accessed it via a node using OPEN_FLAGS, without compromising security.

 -- Function: error_t store_clone (struct store *FROM, struct store **TO)
     Return a copy of FROM in TO.

 -- Function: error_t store_remap (struct store *SOURCE, const struct store_run *RUNS, size_t NUM_RUNS, struct store **STORE)
     Return a store in STORE that reflects the blocks in RUNS and RUNS_LEN from source; SOURCE is consumed, but not RUNS.  Unlike the store_remap_create function, this may simply modify SOURCE and return it.

Store I/O
---------

The following functions allow you to read and modify the contents of a store:

 -- Function: error_t store_map (const struct store *STORE, vm_prot_t PROT, mach_port_t *MEMOBJ)
     Return a memory object paging on STORE.

 -- Function: error_t store_read (struct store *STORE, store_offset_t ADDR, size_t AMOUNT, void **BUF, size_t *LEN)
     Read AMOUNT bytes from STORE at ADDR into BUF and LEN (which follows the usual Mach buffer-return semantics) to STORE at ADDR.  ADDR is in BLOCKS (as defined by STORE->block_size).  Note that LEN is in bytes.

 -- Function: error_t store_write (struct store *STORE, store_offset_t ADDR, void *BUF, size_t LEN, size_t *AMOUNT)
     Write LEN bytes from BUF to STORE at ADDR. Returns the amount written in AMOUNT (in bytes).  ADDR is in BLOCKS (as defined by STORE->block_size).

 -- Function: error_t store_set_size (struct store *STORE, store_offset_t NEWSIZE)
     Set STORE's size to NEWSIZE (in bytes).

Store Classes
-------------

The store library comes with a number of standard store class implementations:

 -- Variable: extern const struct store_class *const store_std_classes []
     This is a null-terminated vector of the standard store classes implemented by libstore.

If you are building your own class vectors, the following function may be useful:

 -- Variable: error_t store_concat_class_vectors (struct store_class **CV1, struct store_class **CV2, struct store_class ***CONCAT)
     Concatenate the store class vectors in CV1 and CV2, and return a new (malloced) vector in CONCAT.

query store
...........

 -- Variable: extern const struct store_class store_query_class
     This store is a virtual store which queries a filesystem node, and delegates control to an appropriate store class.

 -- Function: error_t store_open (const char *NAME, int FLAGS, const struct store_class *const *CLASSES, struct store **STORE)
     Open the file NAME, and return a new store in STORE, which refers to the storage underlying it.  CLASSES is used to select classes specified by the provider; if it is zero, then STORE_STD_CLASSES is used.  FLAGS is set with store_set_flags.  A reference to the open file is created (but may be destroyed with store_close_source).

typed_open store
................

 -- Variable: extern const struct store_class store_typed_open_class
     This store is special in that it doesn't correspond to any specific store functions, rather it provides a way to interpret character strings as specifications for other stores.

 -- Function: error_t store_typed_open (const char *NAME, int FLAGS, const struct store_class *const *CLASSES, struct store **STORE)
     Open the store indicated by NAME, which should consist of a store type name followed by a : and any type-specific name, returning the new store in STORE.  CLASSES is used to select classes specified by the type name; if it is zero, STORE_STD_CLASSES is used.

 -- Function: error_t store_open_children (const char *NAME, int FLAGS, const struct store_class *const *CLASSES, struct store ***STORES, size_t *NUM_STORES)
     Parse multiple store names in NAME, and open each individually, returning all in the vector STORES, and the number in NUM_STORES.  The syntax of NAME is a single non-alphanumeric separator character, followed by each child store name separated by the same separator; each child name is TYPE:NAME notation as parsed by store_typed_open.  If every child uses the same TYPE: prefix, then it may be factored out and put before the child list instead (the two notations are differentiated by whether or not the first character of NAME is alphanumeric).

device store
............

 -- Variable: extern const struct store_class store_device_class
     This store is a simple wrapper for a microkernel device driver. (It is important to note that device drivers are not provided by the Hurd, but by the underlying microkernel.  Hurd `devices' are just storeio-translated nodes which make the microkernel device drivers obey Hurd semantics.  If you wish to implement a new device driver, you will need to consult the appropriate microkernel documentation.)

 -- Function: error_t store_device_open (const char *NAME, int FLAGS, struct store **STORE)
     Open the device named NAME, and return the corresponding store in STORE.

 -- Function: error_t store_device_create (device_t DEVICE, int FLAGS, struct store **STORE)
     Return a new store in STORE referring to the microkernel device DEVICE.  Consumes the DEVICE send right.

file store
..........

 -- Variable: extern const struct store_class store_file_class
     This store reads and writes the contents of a Hurd file.

 -- Function: error_t store_file_open (const char *NAME, int FLAGS, struct store **STORE)
     Open the file NAME, and return the corresponding store in STORE.

 -- Function: error_t store_file_create (file_t FILE, int FLAGS, struct store **STORE)
     Return a new store in STORE referring to the file FILE. Unlike store_create, this will always use file I/O, even it would be possible to be more direct.  This may work in more cases, for instance if the file has holes.  Consumes the FILE send right.

task store
..........

 -- Variable: extern const struct store_class store_task_class
     This store provides access to the contents of a microkernel task.

 -- Variable: error_t store_task_open (const char *NAME, int FLAGS, struct store **STORE)
     Open the task NAME (NAME should be the task's pid), and return the corresponding store in STORE.

 -- Variable: error_t store_task_create (task_t TASK, int FLAGS, struct store **STORE)
     Return a new store in STORE referring to the task TASK, consuming the TASK send right.

zero store
..........

 -- Variable: extern const struct store_class store_zero_class
     Reads to this store always return zero-filled buffers, no matter what has been written into it.  This store corresponds to the Unix /dev/zero device node.

 -- Function: error_t store_zero_create (store_offset_t SIZE, int FLAGS, struct store **STORE)
     Return a new zero store SIZE bytes long in STORE.

copy store
..........

 -- Variable: extern const struct store_class store_copy_class
     This store provides a temporary copy of another store.  This is useful if you want to provide writable data, but do not wish to modify the underlying store.  All changes to a copy store are lost when it is closed.

 -- Function: error_t store_copy_open (const char *NAME, int FLAGS, const struct store_class *const *CLASSES, struct store **STORE)
     Open the copy store NAME (which consists of another store class name, a :, and a name for the store class to open) and return the corresponding store in STORE.  CLASSES is used to select classes specified by the type name; if it is zero, STORE_STD_CLASSES is used.

 -- Function: error_t store_copy_create (struct store *FROM, int FLAGS, struct store **STORE)
     Return a new store in STORE which contains a snapshot of the contents of the store FROM; FROM is consumed.

 -- Function: error_t store_buffer_create (void *BUF, size_t BUF_LEN, int FLAGS, struct store **STORE)
     Return a new store in STORE which contains the memory buffer BUF, of length BUF_LEN.  BUF must be allocated with vm_allocate, and will be consumed.

gunzip store
............

 -- Variable: extern const struct store_class store_gunzip_class
     This store provides transparent GNU zip decompression of a substore. Unfortunately, this store is currently read-only.

 -- Variable: error_t store_gunzip_open (const char *NAME, int FLAGS, const struct store_class *const *CLASSES, struct store **STORE)
     Open the gunzip store NAME (which consists of another store class name, a :, and a name for that store class to open), and return the corresponding store in STORE.  CLASSES is used to select classes specified by the type name; if it is zero, STORE_STD_CLASSES is used.

 -- Variable: error_t store_gunzip_create (struct store *FROM, int FLAGS, struct store **STORE)
     Return a new store in STORE which contains a snapshot of the uncompressed contents of the store FROM; FROM is consumed. BLOCK_SIZE is the desired block size of the result.

concat store
............

 -- Variable: extern const struct store_class store_concat_class
     This class provides a linear concatenation storage mode.  It creates a new virtual store which consists of several different substores appended to one another.

     This mode is designed to increase storage capacity, so that when one substore is filled, new data is transparently written to the next substore.  Concatenation requires robust hardware, since a failure in any single substore will wipe out a large section of the data.

 -- Function: error_t store_concat_open (const char *NAME, int FLAGS, const struct store_class *const *CLASSES, struct store **STORE)
     Return a new store that concatenates the stores created by opening all the individual stores described in NAME; for the syntax of NAME, see store_open_children.

 -- Function: error_t store_concat_create (struct store * const *STORES, size_t NUM_STORES, int FLAGS, struct store **STORE)
     Return a new store in STORE that concatenates all the stores in STORES (NUM_STORES of them).  The stores in STORES are consumed; that is, they will be freed when this store is freed.  The STORES array, however, is copied, and so should be freed by the caller.

ileave store
............

 -- Variable: extern const struct store_class store_ileave_class
     This class provides a RAID-0 (``RAID'' stands for Redundant Array of Independent Disks: several disks used in parallel to achieve increased capacity, redundancy and/or performance.) storage mode (also called disk striping).  It creates a new virtual store by interleaving the contents of several different substores.

     This RAID mode is designed to increase storage performance, since I/O will probably occur in parallel if the substores reside on different physical devices.  Interleaving works best with evenly-yoked substores... if the stores are different sizes, some space will be not be used at the end of the larger stores; if the stores are different speeds, then I/O will have to wait for the slowest store; if some stores are not as reliable as others, failures will wipe out every Nth storage block, where N is the number of substores.

 -- Function: error_t store_ileave_create (struct store * const *STRIPES, size_t num_stripes, store_offset_t INTERLEAVE, int FLAGS, struct store **STORE)
     Return a new store in STORE that interleaves all the stores in STRIPES (NUM_STRIPES of them) every INTERLEAVE bytes; INTERLEAVE must be an integer multiple of each stripe's block size.  The stores in STRIPES are consumed; that is, they will be freed when this store is freed.  The STRIPES array, however, is copied, and so should be freed by the caller.

mvol store
..........

 -- Variable: extern const struct store_class store_mvol_class
     This store provides access to multiple volumes using a single-volume device.  One use of this store would be to provide a store which consists of multiple floppy disks when there is only a single disk drive.  It works by remapping a single linear address range to multiple address ranges, and keeping track of the currently active range. Whenever a request maps to a range that is not active, a callback is made in order to switch to the new range.

     This class is not included in STORE_STD_CLASSES, because it requires an application-specific callback.

 -- Function: error_t store_mvol_create (struct store *PHYS, error_t (*SWAP_VOLS) (struct store *STORE, size_t NEW_VOL, ssize_t OLD_VOL), int FLAGS, struct store **STORE)
     Return a new store in STORE that multiplexes multiple physical volumes from PHYS as one larger virtual volume.  SWAP_VOLS is a function that will be called whenever reads or writes refer to a block which is not addressable on the currently active volume. PHYS is consumed.

remap store
...........

 -- Variable: extern const struct store_class store_remap_class
     This store translates I/O requests into different addresses on a different store.

 -- Function: error_t store_remap_create (struct store *SOURCE, const struct store_run *RUNS, size_t NUM_RUNS, int FLAGS, struct store **STORE)
     Return a new store in STORE that reflects the blocks in RUNS and RUNS_LEN from SOURCE; SOURCE is consumed, but RUNS is not.  Unlike the store_remap function, this function always operates by creating a new store of type remap which has SOURCE as a child, and so may be less efficient than store_remap for some types of stores.

Store RPC Encoding
------------------

The store library also provides some functions which help transfer stores between tasks via RPC:

 -- Structure: struct store_enc
     This structure is used to hold the various bits that make up the representation of a store for transmission via RPC.  See <hurd/hurd_types.h> for an explanation of the encodings for the various storage types.

 -- Function: void store_enc_init (struct store_enc *ENC, mach_port_t *PORTS, mach_msg_type_number_t NUM_PORTS, int *INTS, mach_msg_type_number_t NUM_INTS, off_t *OFFSETS, mach_msg_type_number_t NUM_OFFSETS, char *DATA, mach_msg_type_number_t DATA_LEN)
     Initialize ENC.  The given vector and sizes will be used for the encoding if they are big enough (otherwise new ones will be automatically allocated).

 -- Function: void store_enc_dealloc (struct store_enc *ENC)
     Deallocate storage used by the fields in ENC (but nothing is done with ENC itself).

 -- Function: void store_enc_return (struct store_enc *ENC, mach_port_t **PORTS, mach_msg_type_number_t *NUM_PORTS, int **INTS, mach_msg_type_number_t *NUM_INTS, off_t **OFFSETS, mach_msg_type_number_t *NUM_OFFSETS, char **DATA, mach_msg_type_number_t *DATA_LEN)
     Copy out the parameters from ENC into the given variables suitably for returning from a file_get_storage_info RPC, and deallocate ENC.

 -- Function: error_t store_return (const struct store *STORE, mach_port_t **PORTS, mach_msg_type_number_t *NUM_PORTS, int **INTS, mach_msg_type_number_t *NUM_INTS, off_t **OFFSETS, mach_msg_type_number_t *NUM_OFFSETS, char **DATA, mach_msg_type_number_t *DATA_LEN)
     Encode STORE into the given return variables, suitably for returning from a file_get_storage_info RPC.

 -- Function: error_t store_encode (const struct store *STORE, struct store_enc *ENC)
     Encode STORE into ENC, which should have been prepared with store_enc_init, or return an error.  The contents of ENC may then be returned as the value of file_get_storage_info; if for some reason this can't be done, store_enc_dealloc may be used to deallocate the memory used by the unsent vectors.

 -- Function: error_t store_decode (struct store_enc *ENC, const struct store_class *const *CLASSES, struct store **STORE)
     Decode ENC, either returning a new store in STORE, or an error.  CLASSES is the mapping from Hurd storage class ids to store classes; if it is zero, STORE_STD_CLASSES is used.  If nothing else is to be done with ENC, its contents may then be freed using store_enc_dealloc.

 -- Function: error_t store_allocate_child_encodings (const struct store *STORE, struct store_enc *ENC)
     Calls the allocate_encoding method in each child store of STORE, propagating any errors.  If any child does not have such a method, EOPNOTSUPP is returned.

 -- Function: error_t store_encode_children (const struct store *STORE, struct store_enc *ENC)
     Calls the encode method in each child store of STORE, propagating any errors.  If any child does not have such a method, EOPNOTSUPP is returned.

 -- Function: error_t store_decode_children (struct store_enc *ENC, int NUM_CHILDREN, const struct store_class *const *CLASSES, struct store **CHILDREN)
     Decodes NUM_CHILDREN from ENC, storing the results into successive positions in CHILDREN.

 -- Function: error_t store_with_decoded_runs (struct store_enc *ENC, size_t NUM_RUNS, error_t (*FUN) (const struct store_run *RUNS, size_t NUM_RUNS))
     Call FUN with the vector RUNS of length NUM_RUNS extracted from ENC.

 -- Function: error_t store_std_leaf_allocate_encoding (const struct store *STORE, struct store_enc *ENC)
 -- Function: error_t store_std_leaf_encode (const struct store *STORE, struct store_enc *ENC)
     Standard encoding used for most data-providing (as opposed to filtering) store classes.

 -- Typedef: typedef error_t (*store_std_leaf_create_t) (mach_port_t PORT, int FLAGS, size_t BLOCK_SIZE, const struct store_run *RUNS, size_t NUM_RUNS, struct store **STORE)
     Creation function used by store_std_leaf_decode.

 -- Function: error_t store_std_leaf_decode (struct store_enc *ENC, store_std_leaf_create_t CREATE, struct store **STORE)
     Decodes the standard leaf encoding which is common to various builtin formats, and calls CREATE to actually create the store.

Stored Filesystems
******************

Stored filesystems allow users to save and load persistent data from any random-access storage media, such as hard disks, floppy diskettes, and CD-ROMs.  Stored filesystems are required for bootstrapping standalone workstations, as well.

Repairing Filesystems
=====================

FIXME: finish

Linux Extended 2 FS
===================

FIXME: finish

BSD Unix FS
===========

FIXME: finish

ISO-9660 CD-ROM FS
==================

FIXME: finish

Diskfs Library
==============

The diskfs library is declared in <hurd/diskfs.h>, and does a lot of the work of implementing stored filesystems.  libdiskfs requires the threads, ports, iohelp, fshelp, and store libraries.  You should understand all these libraries before you attempt to use diskfs, and you should also be familiar with the pager library (see Pager Library).

For historical reasons, the library for implementing stored filesystems is called libdiskfs instead of libstorefs.  Keep in mind, however, that diskfs is useful for filesystems which are implemented on any block-addressed storage device, since it uses the store library to do I/O.

Note that stored filesystems can be tricky to implement, since the diskfs callback interfaces are not trivial.  It really is best if you examine the source code of a similar existing filesystem server, and follow its example rather than trying to write your own from scratch.

Diskfs Startup
--------------

This subsection gives an outline of the general steps involved in implementing a filesystem server, to help refresh your memory and to offer explanations rather than to serve as a tutorial.

The first thing a filesystem server should do is parse its command-line arguments (see Diskfs Arguments).  Then, the standard output and error streams should be redirected to the console, so that error messages are not lost if this is the bootstrap filesystem:

 -- Function: void diskfs_console_stdio (void)
     Redirect error messages to the console, so that they can be seen by users.

The following is a list of the relevant functions which would be called during the rest of the server initialization.  Again, you should refer to the implementation of an already-working filesystem if you have any questions about how these functions should be used:

 -- Function: error_t diskfs_init_diskfs (void)
     Call this function after arguments have been parsed to initialize the library.  You must call this before calling any other diskfs functions, and after parsing diskfs options.

 -- Function: void diskfs_spawn_first_thread (void)
     Call this after all format-specific initialization is done (except for setting diskfs_root_node); at this point the pagers should be ready to go.

 -- Function: mach_port_t diskfs_startup_diskfs (mach_port_t BOOTSTRAP, int FLAGS)
     Call this once the filesystem is fully initialized, to advertise the new filesystem control port to our parent filesystem.  If BOOTSTRAP is set, diskfs will call fsys_startup on that port as appropriate and return the REALNODE from that call; otherwise we call diskfs_start_bootstrap and return MACH_PORT_NULL. FLAGS specifies how to open REALNODE (from the O_* set).

You should not need to call the following function directly, since diskfs_startup_diskfs will do it for you, when appropriate:

 -- Function: void diskfs_start_bootstrap (void)
     Start the Hurd bootstrap sequence as if we were the bootstrap filesystem (that is, diskfs_boot_flags is nonzero).  All filesystem initialization must be complete before you call this function.

Diskfs Arguments
----------------

The following functions implement standard diskfs command-line and runtime argument parsing, using argp (see Argp (The GNU C Library Reference Manual)):

 -- Function: error_t diskfs_set_options (char *ARGZ, size_t ARGZ_LEN)
     Parse and execute the runtime options specified by ARGZ and ARGZ_LEN.  EINVAL is returned if some option is unrecognized.  The default definition of this routine will parse them using diskfs_runtime_argp.

 -- Function: error_t diskfs_append_args (char **ARGZ, unsigned *ARGZ_LEN)
     Append to the malloced string *ARGZ of length *ARGZ_LEN a NUL-separated list of the arguments to this translator.  The default definition of this routine simply calls diskfs_append_std_options.

 -- Function: error_t diskfs_append_std_options (char **ARGZ, unsigned *ARGZ_LEN)
     Appends NUL-separated options describing the standard diskfs option state to ARGZ and increments ARGZ_LEN appropriately. Note that unlike diskfs_get_options, ARGZ and ARGZ_LEN must already have sane values.

 -- Variable: struct argp * diskfs_runtime_argp
     If this is defined or set to an argp structure, it will be used by the default diskfs_set_options to handle runtime option parsing.  The default definition is initialized to a pointer to diskfs_std_runtime_argp.

 -- Variable: const struct argp diskfs_std_runtime_argp
     An argp for the standard diskfs runtime options.  The default definition of diskfs_runtime_argp points to this, although the user can redefine that to chain this onto his own argp.

 -- Variable: const struct argp diskfs_startup_argp
     An argp structure for the standard diskfs command line arguments.  The user may call argp_parse on this to parse the command line, chain it onto the end of his own argp structure, or ignore it completely.

 -- Variable: const struct argp diskfs_store_startup_argp
     An argp structure for the standard diskfs command line arguments plus a store specification.  The address of a location in which to return the resulting struct store_parsed structure should be passed as the input argument to argp_parse; FIXME xref the declaration for STORE_ARGP.

Diskfs Globals
--------------

The following functions and variables control the overall behaviour of the library.  Your callback functions may need to refer to these, but you should not need to modify or redefine them.

 -- Variable: mach_port_t diskfs_default_pager
 -- Variable: mach_port_t diskfs_exec_ctl
 -- Variable: mach_port_t diskfs_exec
 -- Variable: auth_t diskfs_auth_server_port
     These are the respective send rights to the default pager, execserver control port, execserver itself, and authserver.

 -- Variable: mach_port_t diskfs_fsys_identity
     The io_identity identity port for the filesystem.

 -- Variable: char ** diskfs_argv
     The command line with which diskfs was started, set by the default argument parser. If you don't use it, set this yourself.  This is only used for bootstrap file systems, to give the procserver.

 -- Variable: char * diskfs_boot_flags
     When this is a bootstrap filesystem, the command line options passed from the kernel.  If not a bootstrap filesystem, it is zero, so it can be used to distinguish between the two cases.

 -- Variable: struct rwlock diskfs_fsys_lock
     Hold this lock while doing filesystem-level operations.  Innocuous users can just hold a reader lock, but operations that might corrupt other threads should hold a writer lock.

 -- Variable: volatile struct mapped_time_value * diskfs_mtime
     The current system time, as used by the diskfs routines.  This is converted into a struct timeval by the maptime_read C library function (FIXME xref).

 -- Variable: int diskfs_synchronous
     True if and only if we should do every operation synchronously.  It is the format-specific code's responsibility to keep allocation information permanently in sync if this is set; the rest will be done by format-independent code.

 -- Function: error_t diskfs_set_sync_interval (int INTERVAL)
     Establish a thread to sync the filesystem every INTERVAL seconds, or never, if INTERVAL is zero.  If an error occurs creating the thread, it is returned, otherwise zero.  Subsequent calls will create a new thread and (eventually) get rid of the old one; the old thread won't do any more syncs, regardless.

 -- Variable: spin_lock_t diskfs_node_refcnt_lock
     Pager reference count lock.

 -- Variable: int diskfs_readonly
     Set to zero if the filesystem is currently writable.

 -- Function: error_t diskfs_set_readonly (int READONLY)
     Change an active filesystem between read-only and writable modes, setting the global variable DISKFS_READONLY to reflect the current mode.  If an error is returned, nothing will have changed. DISKFS_FSYS_LOCK should be held while calling this routine.

 -- Function: int diskfs_check_readonly (void)
     Check if the filesystem is readonly before an operation that writes it. Return nonzero if readonly, otherwise zero.

 -- Function: error_t diskfs_remount (void)
     Reread all in-core data structures from disk.  This function can only be successful if DISKFS_READONLY is true.  DISKFS_FSYS_LOCK should be held while calling this routine.

 -- Function: error_t diskfs_shutdown (int FLAGS)
     Shutdown the filesystem; FLAGS are as for fsys_shutdown.

Diskfs Node Management
----------------------

Every file or directory is a diskfs node.  The following functions help your diskfs callbacks manage nodes and their references:

 -- Function: void diskfs_drop_node (struct node *NP)
     Node NP now has no more references; clean all state.  The DISKFS_NODE_REFCNT_LOCK must be held, and will be released upon return.  NP must be locked.

 -- Function: void diskfs_node_update (struct node *NP, int WAIT)
     Set disk fields from NP->dn_stat; update ctime, atime, and mtime if necessary.  If WAIT is true, then return only after the physical media has been completely updated.

 -- Function: void diskfs_nref (struct node *NP)
     Add a hard reference to node NP.  If there were no hard references previously, then the node cannot be locked (because you must hold a hard reference to hold the lock).

 -- Function: void diskfs_nput (struct node *NP)
     Unlock node NP and release a hard reference; if this is the last hard reference and there are no links to the file then request light references to be dropped.

 -- Function: void diskfs_nrele (struct node *NP)
     Release a hard reference on NP.  If NP is locked by anyone, then this cannot be the last hard reference (because you must hold a hard reference in order to hold the lock).  If this is the last hard reference and there are no links, then request light references to be dropped.

 -- Function: void diskfs_nref_light (struct node *NP)
     Add a light reference to a node.

 -- Function: void diskfs_nput_light (struct node *NP)
     Unlock node NP and release a light reference.

 -- Function: void diskfs_nrele_light (struct node *NP)
     Release a light reference on NP.  If NP is locked by anyone, then this cannot be the last reference (because you must hold a hard reference in order to hold the lock).

 -- Function: error_t diskfs_node_rdwr (struct node *NP, char *DATA, off_t OFF, size_t AMT, int DIRECTION, struct protid *CRED, size_t *AMTREAD)
     This is called by other filesystem routines to read or write files, and extends them automatically, if necessary.  NP is the node to be read or written, and must be locked.  DATA will be written or filled.  OFF identifies where in the file the I/O is to take place (negative values are not allowed).  AMT is the size of DATA and tells how much to copy.  DIR is zero for reading or nonzero for writing.  CRED is the user doing the access (only used to validate attempted file extension).  For reads, *AMTREAD is filled with the amount actually read.

 -- Function: void diskfs_notice_dirchange (struct node *DP, enum dir_changed_type TYPE, char *NAME)
     Send notifications to users who have requested them for directory DP with dir_notice_changes.  The type of modification and affected name are TYPE and NAME respectively.  This should be called by diskfs_direnter, diskfs_dirremove, diskfs_dirrewrite, and anything else that changes the directory, after the change is fully completed.

 -- Function: struct node * diskfs_make_node (struct disknode *DN)
     Create a new node structure with DS as its physical disknode.  The new node will have one hard reference and no light references.

These next node manipulation functions are not generally useful, but may come in handy if you need to redefine any diskfs functions.

 -- Function: error_t diskfs_create_node (struct node *DIR, char *NAME, mode_t MODE, struct node **NEWNODE, struct protid *CRED, struct dirstat *DS)
     Create a new node.  Give it MODE: if MODE includes IFDIR, also initialize . and .. in the new directory.  Return the node in NPP.  CRED identifies the user responsible for the call.  If NAME is nonzero, then link the new node into DIR with name NAME; DS is the result of a prior diskfs_lookup for creation (and DIR has been held locked since).  DIR must always be provided as at least a hint for disk allocation strategies.

 -- Function: void diskfs_set_node_atime (struct node *NP)
     If disk is not readonly and the noatime option is not enabled, set NP->dn_set_atime.

 -- Function: void diskfs_set_node_times (struct node *NP)
     If NP->dn_set_ctime is set, then modify NP->dn_stat.st_ctim appropriately; do the analogous operations for st_atim and st_mtim as well.

 -- Function: struct node * diskfs_check_lookup_cache (struct node *DIR, char *NAME)
     Scan the cache looking for NAME inside DIR.  If we don't know any entries at all, then return zero.  If the entry is confirmed to not exist, then return -1.  Otherwise, return NP for the entry, with a newly-allocated reference.

 -- Function: error_t diskfs_cached_lookup (int CACHE_ID, struct node **NPP)
     Return the node corresponding to CACHE_ID in *NPP.

 -- Function: void diskfs_enter_lookup_cache (struct node *DIR, struct node *NP, char *NAME)
     Node NP has just been found in DIR with NAME.  If NP is null, that means that this name has been confirmed as absent in the directory.

 -- Function: void diskfs_purge_lookup_cache (struct node *DP, struct node *NP)
     Purge all references in the cache to NP as a node inside directory DP.

Diskfs Callbacks
----------------

Like several other Hurd libraries, libdiskfs depends on you to implement application-specific callback functions.  You must define the following functions and variables, but you should also look at Diskfs Options, as there are several defaults which should be modified to provide good filesystem support:

 -- Structure: struct dirstat
     You must define this type, which will hold information between a call to diskfs_lookup and a call to one of diskfs_direnter, diskfs_dirremove, or diskfs_dirrewrite.  It must contain enough information so that those calls work as described below.

 -- Variable: const size_t diskfs_dirstat_size
     This must be the size in bytes of a struct dirstat.

 -- Variable: int diskfs_link_max
     This is the maximum number of links to any one file, which must be a positive integer.  The implementation of dir_rename does not know how to succeed if this is only one allowed link; on such formats you need to reimplement dir_rename yourself.

 -- Variable: int diskfs_maxsymlinks
     This variable is a positive integer which is the maximum number of symbolic links which can be traversed within a single call to dir_lookup.  If this is exceeded, dir_lookup will return ELOOP.

 -- Variable: struct node * diskfs_root_node
     Set this to be the node of the root of the filesystem.

 -- Variable: char * diskfs_server_name
     Set this to the name of the filesystem server.

 -- Variable: char * diskfs_server_version
     Set this to be the server version string.

 -- Variable: char * diskfs_disk_name
     This should be a string that somehow identifies the particular disk this filesystem is interpreting.  It is generally only used to print messages or to distinguish instances of the same filesystem type from one another.  If this filesystem accesses no external media, then define this to be zero.

 -- Function: error_t diskfs_set_statfs (fsys_statfsbuf_t *STATFSBUF)
     Set *STATFSBUF with appropriate values to reflect the current state of the filesystem.

 -- Function: error_t diskfs_lookup (struct node *DP, const char *NAME, enum lookup_type TYPE, struct node **NP, struct dirstat *DS, struct protid *CRED)
 -- Function: error_t diskfs_lookup_hard (struct node *DP, const char *NAME, enum lookup_type TYPE, struct node **NP, struct dirstat *DS, struct protid *CRED)
     You should not define diskfs_lookup, because it is simply a wrapper for diskfs_lookup_hard, and is already defined in libdiskfs.

     Lookup in directory DP (which is locked) the name NAME. TYPE will either be LOOKUP, CREATE, RENAME, or REMOVE.  CRED identifies the user making the call.

     If the name is found, return zero, and (if NP is nonzero) set *NP to point to the node for it, which should be locked. If the name is not found, return ENOENT, and (if NP is nonzero) set *NP to zero.  If NP is zero, then the node found must not be locked, not even transitorily.  Lookups for REMOVE and RENAME (which must often check permissions on the node being found) will always set NP.

     If DS is nonzero then the behaviour varies depending on the requested lookup TYPE:

     LOOKUP
          Set *DS to be ignored by diskfs_drop_dirstat

     CREATE
          On success, set *DS to be ignored by diskfs_drop_dirstat.

          On failure, set *DS for a future call to diskfs_direnter.

     RENAME
          On success, set *DS for a future call to diskfs_dirrewrite.

          On failure, set *DS for a future call to diskfs_direnter.

     REMOVE
          On success, set *DS for a future call to diskfs_dirremove.

          On failure, set *DS to be ignored by diskfs_drop_dirstat.

     The caller of this function guarantees that if DS is nonzero, then either the appropriate call listed above or diskfs_drop_dirstat will be called with DS before the directory DP is unlocked, and guarantees that no lookup calls will be made on this directory between this lookup and the use (or destruction) of *DS.

     If you use the library's versions of diskfs_rename_dir, diskfs_clear_directory, and diskfs_init_dir, then lookups for .. might have the flag SPEC_DOTDOT ORed in.  This has a special meaning depending on the requested lookup TYPE:

     LOOKUP
          DP should be unlocked and its reference dropped before returning.

     CREATE
          Ignore this case, because SPEC_DOTDOT is guaranteed not to be given.

     RENAME
     REMOVE
          In both of these cases, the node being found (*NP) is already held locked, so don't lock it or add a reference to it.

     Return ENOENT if NAME isn't in the directory.  Return EAGAIN if NAME refers to the .. of this filesystem's root.  Return EIO if appropriate.

 -- Function: error_t diskfs_direnter (struct node *DP, char *NAME, struct node *NP, struct dirstat *DS, struct protid *CRED)
 -- Function: error_t diskfs_direnter_hard (struct node *DP, char *NAME, struct node *NP, struct dirstat *DS, struct protid *CRED)
     You should not define diskfs_direnter, because it is simply a wrapper for diskfs_direnter_hard, and is already defined in libdiskfs.

     Add NP to directory DP under the name NAME.  This will only be called after an unsuccessful call to diskfs_lookup of type CREATE or RENAME; DP has been locked continuously since that call and DS is as that call set it, NP is locked. CRED identifies the user responsible for the call (to be used only to validate directory growth).

 -- Function: error_t diskfs_dirrewrite (struct node *DP, struct node *OLDNP, struct node *NP, char *NAME, struct dirstat *DS)
 -- Function: error_t diskfs_dirrewrite_hard (struct node *DP, struct node *NP, struct dirstat *DS)
     You should not define diskfs_dirrewrite, because it is simply a wrapper for diskfs_dirrewrite_hard, and is already defined in libdiskfs.

     This will only be called after a successful call to diskfs_lookup of type RENAME; this call should change the name found in directory DP to point to node NP instead of its previous referent.  DP has been locked continuously since the call to diskfs_lookup and DS is as that call set it; NP is locked.

     diskfs_dirrewrite has some additional specifications: NAME is the name within DP which used to correspond to the previous referent, OLDNP; it is this reference which is being rewritten. diskfs_dirrewrite also calls diskfs_notice_dirchange if DP->dirmod_reqs is nonzero.

 -- Function: error_t diskfs_dirremove (struct node *DP, struct node *NP, char *NAME, struct dirstat *DS)
 -- Function: error_t diskfs_dirremove_hard (struct node *DP, struct dirstat *DS)
     You should not define diskfs_dirremove, because it is simply a wrapper for diskfs_dirremove_hard, and is already defined in libdiskfs.

     This will only be called after a successful call to diskfs_lookup of type REMOVE; this call should remove the name found from the directory DS.  DP has been locked continuously since the call to diskfs_lookup and DS is as that call set it.

     diskfs_dirremove has some additional specifications: this routine should call diskfs_notice_dirchange if DP->dirmod_reqs is nonzero.  The entry being removed has name NAME and refers to NP.

 -- Function: error_t diskfs_drop_dirstat (struct node *DP, struct dirstat *DS)
     DS has been set by a previous call to diskfs_lookup on directory DP; this function is guaranteed to be called if diskfs_direnter, diskfs_dirrewrite, and diskfs_dirremove have not been called, and should free any state retained by a struct dirstat.  DP has been locked continuously since the call to diskfs_lookup.

 -- Function: void diskfs_null_dirstat (struct dirstat *DS)
     Initialize DS such that diskfs_drop_dirstat will ignore it.

 -- Function: error_t diskfs_get_directs (struct node *DP, int ENTRY, int N, char **DATA, u_int *DATACNT, vm_size_t BUFSIZ, int *AMT)
     Return N directory entries starting at ENTRY from locked directory node DP.  Fill *DATA with the entries; which currently points to *DATACNT bytes.  If it isn't big enough, vm_allocate into *DATA.  Set *DATACNT with the total size used.  Fill AMT with the number of entries copied.  Regardless, never copy more than BUFSIZ bytes.  If BUFSIZ is zero, then there is no limit on *DATACNT; if N is -1, then there is no limit on AMT.

 -- Function: int diskfs_dirempty (struct node *DP, struct protid *CRED)
     Return nonzero if locked directory DP is empty.  If the user has not redefined diskfs_clear_directory and diskfs_init_directory, then `empty' means `only possesses entries labelled . and ...  CRED identifies the user making the call... if this user cannot search the directory, then this routine should fail.

 -- Function: error_t diskfs_get_translator (struct node *NP, char **NAMEP, u_int *NAMELEN)
     For locked node NP (for which diskfs_node_translated is true) look up the name of its translator.  Store the name into newly malloced storage and set *NAMELEN to the total length.

 -- Function: error_t diskfs_set_translator (struct node *NP, char *NAME, u_int NAMELEN, struct protid *CRED)
     For locked node NP, set the name of the translating program to be NAME, which is NAMELEN bytes long.  CRED identifies the user responsible for the call.

 -- Function: error_t diskfs_truncate (struct node *NP, off_t SIZE)
     Truncate locked node NP to be SIZE bytes long.  If NP is already less than or equal to SIZE bytes long, do nothing.  If this is a symlink (and diskfs_shortcut_symlink is set) then this should clear the symlink, even if diskfs_create_symlink_hook stores the link target elsewhere.

 -- Function: error_t diskfs_grow (struct node *NP, off_t SIZE, struct protid *CRED)
     Grow the disk allocated to locked node NP to be at least SIZE bytes, and set NP->allocsize to the actual allocated size.  If the allocated size is already SIZE bytes, do nothing.  CRED identifies the user responsible for the call.

 -- Function: error_t diskfs_node_reload (struct node *NODE)
     This function must reread all data specific to NODE from disk, without writing anything.  It is always called with DISKFS_READONLY set to true.

 -- Function: error_t diskfs_reload_global_state (void)
     This function must invalidate all cached global state, and reread it as necessary from disk, without writing anything.  It is always called with DISKFS_READONLY set to true.  diskfs_node_reload is subsequently called on all active nodes, so this call doesn't need to reread any node-specific data.

 -- Function: error_t diskfs_node_iterate (error_t (*FUN) (struct node *NP))
     For each active node NP, call FUN.  The node is to be locked around the call to FUN.  If FUN returns nonzero for any node, then stop immediately, and return that value.

 -- Function: error_t diskfs_alloc_node (struct node *DP, mode_t MODE, struct node **NP)
     Allocate a new node to be of mode MODE in locked directory DP, but don't actually set the mode or modify the directory, since that will be done by the caller.  The user responsible for the request can be identified with CRED.  Set *NP to be the newly allocated node.

 -- Function: void diskfs_free_node (struct node *NP, mode_t MODE)
     Free node NP; the on-disk copy has already been synchronized with diskfs_node_update (where NP->dn_stat.st_mode was zero).  NP's mode used to be MODE.

 -- Function: void diskfs_lost_hardrefs (struct node *NP)
     Locked node NP has some light references but has just lost its last hard reference.

 -- Function: void diskfs_new_hardrefs (struct node *NP)
     Locked node NP has just acquired a hard reference where it had none previously.  Therefore, it is okay again to have light references without real users.

 -- Function: void diskfs_try_dropping_softrefs (struct node *NP)
     Node NP has some light references, but has just lost its last hard references.  Take steps so that if any light references can be freed, they are.  Both DISKFS_NODE_REFCNT_LOCK and NP are locked. This function will be called after diskfs_lost_hardrefs.

 -- Function: void diskfs_node_norefs (struct node *NP)
     Node NP has no more references; free local state, including *NP if it shouldn't be retained. DISKFS_NODE_REFCNT_LOCK is held.

 -- Function: error_t diskfs_set_hypermetadata (int WAIT, int CLEAN)
     Write any non-paged metadata from format-specific buffers to disk, asynchronously unless WAIT is nonzero.  If CLEAN is nonzero, then after this is written the filesystem will be absolutely clean, and it must be possible for the non-paged metadata to indicate that fact.

 -- Function: void diskfs_write_disknode (struct node *NP, int WAIT)
     Write the information in NP->dn_stat and any associated format-specific information to the disk.  If WAIT is true, then return only after the physical media has been completely updated.

 -- Function: void diskfs_file_update (struct node *NP, int WAIT)
     Write the contents and all associated metadata of file NP to disk. Generally, this will involve calling diskfs_node_update for much of the metadata.  If WAIT is true, then return only after the physical media has been completely updated.

 -- Function: mach_port_t diskfs_get_filemap (struct node *NP, vm_prot_t PROT)
     Return a memory object port (send right) for the file contents of NP.  PROT is the maximum allowable access.  On errors, return MACH_PORT_NULL and set errno.

 -- Function: struct pager * diskfs_get_filemap_pager_struct (struct node *NP)
     Return a struct pager * that refers to the pager returned by diskfs_get_filemap for locked node NP, suitable for use as an argument to pager_memcpy.

 -- Function: vm_prot_t diskfs_max_user_pager_prot (void)
     Return the bitwise OR of the maximum prot parameter (the second argument to diskfs_get_filemap) for all active user pagers.

 -- Function: int diskfs_pager_users (void)
     Return nonzero if there are pager ports exported that might be in use by users.  Further pager creation should be blocked before this function returns zero.

 -- Function: void diskfs_sync_everything (int WAIT)
     Sync all the pagers and write any data belonging on disk except for the hypermetadata.  If WAIT is true, then return only after the physical media has been completely updated.

 -- Function: void diskfs_shutdown_pager (void)
     Shut down all pagers.  This is irreversible, and is done when the filesystem is exiting.

Diskfs Options
--------------

The functions and variables described in this subsection already have default definitions in libdiskfs, so you are not forced to define them; rather, they may be redefined on a case-by-case basis.

You should set the values of any option variables as soon as your program starts (before you make any calls to diskfs, such as argument parsing).

 -- Variable: int diskfs_hard_readonly
     You should set this variable to nonzero if the filesystem media can never be made writable.

 -- Variable: char * diskfs_extra_version
     Set this to be any additional version specification that should be printed for --version.

 -- Variable: int diskfs_shortcut_symlink
     This should be nonzero if and only if the filesystem format supports shortcutting symbolic link translation.  The library guarantees that users will not be able to read or write the contents of the node directly, and the library will only do so if the symlink hook functions (diskfs_create_symlink_hook and diskfs_read_symlink_hook) return EINVAL or are not defined.  The library knows that the dn_stat.st_size field is the length of the symlink, even if the hook functions are used.

 -- Variable: int diskfs_shortcut_chrdev
 -- Variable: int diskfs_shortcut_blkdev
 -- Variable: int diskfs_shortcut_fifo
 -- Variable: int diskfs_shortcut_ifsock
     These variables should be nonzero if and only if the filesystem format supports shortcutting character device node, block device node, FIFO, or Unix-domain socket translation, respectively.

 -- Variable: int diskfs_default_sync_interval
     diskfs_set_sync_interval is called with this value when the first diskfs thread is started up (in diskfs_spawn_first_thread).  This variable has a default default value of 30, which causes disk buffers to be flushed at least every 30 seconds.

 -- Function: error_t diskfs_validate_mode_change (struct node *NP, mode_t MODE)
 -- Function: error_t diskfs_validate_owner_change (struct node *NP, uid_t UID)
 -- Function: error_t diskfs_validate_group_change (struct node *NP, gid_t GID)
 -- Function: error_t diskfs_validate_author_change (struct node *NP, uid_t AUTHOR)
 -- Function: error_t diskfs_validate_flags_change (struct node *NP, int FLAGS)
 -- Function: error_t diskfs_validate_rdev_change (struct node *NP, dev_t RDEV)
     Return zero if for the node NP can be changed as requested.  That is, if NP's mode can be changed to MODE, owner to UID, group to GID, author to AUTHOR, flags to FLAGS, or raw device number to RDEV, respectively.  Otherwise, return an error code.

     It must always be possible to clear the mode or the flags; diskfs will not ask for permission before doing so.

 -- Function: void diskfs_readonly_changed (int READONLY)
     This is called when the disk has been changed from read-only to read-write mode or vice-versa.  READONLY is the new state (which is also reflected in DISKFS_READONLY).  This function is also called during initial startup if the filesystem is to be writable.

 -- Variable: error_t (*diskfs_create_symlink_hook) (struct node *NP, char *TARGET)
     If this function pointer is nonzero (and diskfs_shortcut_symlink is set) it is called to set a symlink.  If it returns EINVAL or isn't set, then the normal method (writing the contents into the file data) is used.  If it returns any other error, it is returned to the user.

 -- Variable: error_t (*diskfs_read_symlink_hook) (struct node *NP, char *TARGET)
     If this function pointer is nonzero (and diskfs_shortcut_symlink is set) it is called to read the contents of a symlink.  If it returns EINVAL or isn't set, then the normal method (reading from the file data) is used.  If it returns any other error, it is returned to the user.

 -- Function: error_t diskfs_rename_dir (struct node *FDP, struct node *FNP, char *FROMNAME, struct node *TDP, char *TONAME, struct protid *FROMCRED, struct protid *TOCRED)
     Rename directory node FNP (whose parent is FDP, and which has name FROMNAME in that directory) to have name TONAME inside directory TDP.  None of these nodes are locked, and none should be locked upon return.  This routine is serialized, so it doesn't have to be reentrant.  Directories will never be renamed except by this routine.  FROMCRED is the user responsible for FDP and FNP.  TOCRED is the user responsible for TDP.  This routine assumes the usual convention where .  and .. are represented by ordinary links; if that is not true for your format, you have to redefine this function.

 -- Function: error_t diskfs_clear_directory (struct node *DP, struct node *PDP, struct protid *CRED)
     Clear the . and .. entries from directory DP.  Its parent is PDP, and the user responsible for this is identified by CRED.  Both directories must be locked.  This routine assumes the usual convention where . and .. are represented by ordinary links; if that is not true for your format, you have to redefine this function.

 -- Function: error_t diskfs_init_dir (struct node *DP, struct node *PDP, struct protid *CRED)
     Locked node DP is a new directory; add whatever links are necessary to give it structure; its parent is the (locked) node PDP.  This routine may not call diskfs_lookup on PDP. The new directory must be clear within the meaning of diskfs_dirempty.  This routine assumes the usual convention where . and .. are represented by ordinary links; if that is not true for your format, you have to redefine this function.  CRED identifies the user making the call.

Diskfs Internals
----------------

The library also exports the following functions, but they are not generally useful unless you are redefining other functions the library provides.

 -- Function: error_t diskfs_create_protid (struct peropen *PO, struct iouser *USER, struct protid **CRED)
     Create and return a protid for an existing peropen PO in CRED, referring to user USER.  The node PO->np must be locked.

 -- Function: error_t diskfs_start_protid (struct peropen *PO, struct protid **CRED)
     Build and return in CRED a protid which has no user identification, for peropen PO.  The node PO->np must be locked.

 -- Function: void diskfs_finish_protid (struct protid *CRED, struct iouser *USER)
     Finish building protid CRED started with diskfs_start_protid; the user to install is USER.

 -- Function: void diskfs_protid_rele (void *ARG)
     Called when a protid CRED has no more references.  Because references to protids are maintained by the port management library, this is installed in the clean routines list.  The ports library will free the structure.

 -- Function: struct peropen * diskfs_make_peropen (struct node *NP, int FLAGS, struct peropen *CONTEXT)
     Create and return a new peropen structure on node NP with open flags FLAGS.  The initial values for the root_parent, shadow_root, and shadow_root_parent fields are copied from CONTEXT if it is nonzero, otherwise each of these values are set to zero.

 -- Function: void diskfs_release_peropen (struct peropen *PO)
     Decrement the reference count on PO.

 -- Function: error_t diskfs_execboot_fsys_startup (mach_port_t PORT, int FLAGS, mach_port_t CTL, mach_port_t *REAL, mach_msg_type_name_t *REALPOLY)
     This function is called by S_fsys_startup for execserver bootstrap.  The execserver is able to function without a real node, hence this fraud.  Arguments are as for fsys_startup in <hurd/fsys.defs>.

 -- Function: int diskfs_demuxer (mach_msg_header_t *INP, mach_msg_header_t *OUTP)
     Demultiplex incoming libports messages on diskfs ports.

The diskfs library also provides functions to demultiplex the fs, io, fsys, interrupt, and notify interfaces.  All the server routines have the prefix diskfs_S_.  For those routines, in arguments of type file_t or io_t appear as struct protid * to the stub.

Twisted Filesystems
*******************

In the Hurd, translators are capable of redirecting filesystem requests to other translators, which makes it possible to implement alternative views of the same underlying data.  The translators described in this chapter do not provide direct access to any data; rather, they are organizational tools to help you simplify an existing physical filesystem layout.

Be prudent with these translators: you may accidentally injure people who want their filesystems to be rigidly tree-structured. (You are lost in a maze of twisty little filesystems, all alike....)

FIXME: finish

symlink, firmlink
=================

hostmux, usermux
================

shadowfs
========

Distributed Filesystems
***********************

Distributed filesystems are designed to share files between separate machines via a network connection of some sort.  Their design is significantly different than stored filesystems (see Stored Filesystems): they need to deal with the problems of network delays and failures, and may require complex authentication and replication protocols involving multiple file servers.

File Transfer Protocol
======================

FIXME: finish

ftpcp, ftpdir
-------------

ftpfs
-----

FTP Connection Library
----------------------

FIXME: finish

Network File System
===================

FIXME: finish

nfsd
----

nfs
---

Networking
**********

FIXME: this subsystem is in flux

pfinet
======

pflocal
=======

libpipe
=======

Socket Interface
================

FIXME: net frobbing stuff may be added to socket.defs

Terminal Handling
*****************

FIXME: finish

term
====

term.defs
=========

Running Programs
****************

FIXME: finish

ps, w
=====

libps
=====

exec
====

proc
====

crash
=====

Authentication
**************

FIXME: finish

addauth, rmauth, setauth
========================

su, sush, unsu
==============

login, loginpr
==============

auth
====

Auth Interface
==============

FIXME: finish

Auth Protocol
-------------

FIXME: finish