Common Lisp programs need to use names to designate files. The main difficulty in dealing with names of files is that different file systems have different naming formats for files. For example, here is a table of several file systems (actually, operating systems that provide file systems) and what equivalent file names might look like for each one:
| System | File Name |
| TOPS-20 | <LISPIO>FORMAT.FASL.13 |
| TOPS-10 | FORMAT.FAS[1,4] |
| ITS | LISPIO;FORMAT FASL |
| MULTICS | >udd>LispIO>format.fasl |
| TENEX | <LISPIO>FORMAT.FASL;13 |
| VAX/VMS | [LISPIO]FORMAT.FAS;13 |
| UNIX | /usr/lispio/format.fasl |
It would be impossible for each program that deals with file names to know about each different file name format that exists; a new Common Lisp implementation might use a format different from any of its predecessors. Therefore, Common Lisp provides two ways to represent file names: namestrings, which are strings in the implementation-dependent form customary for the file system, and pathnames, which are special abstract data objects that represent file names in an implementation-independent way. Functions are provided to convert between these two representations, and all manipulations of files can be expressed in machine-independent terms by using pathnames.
In order to allow Common Lisp programs to operate in a network environment that may have more than one kind of file system, the pathname facility allows a file name to specify which file system is to be used. In this context, each file system is called a host, in keeping with the usual networking terminology.
Different hosts may use different notations for file names. Common Lisp allows customary notation to be used for each host, but also supports a system of logical pathnames that provides a standard framework for naming files in a portable manner (see section 23.1.5).
All file systems dealt with by Common Lisp are forced into a common framework, in which files are named by a Lisp data object of type pathname.
A pathname always has six components, described below. These components are the common interface that allows programs to work the same way with different file systems; the mapping of the pathname components into the concepts peculiar to each file system is taken care of by the Common Lisp implementation.
Note that a pathname is not necessarily the name of a specific file. Rather, it is a specification (possibly only a partial specification) of how to access a file. A pathname need not correspond to any file that actually exists, and more than one pathname can refer to the same file. For example, the pathname with a version of “newest” may refer to the same file as a pathname with the same components except a certain number as the version. Indeed, a pathname with version “newest” may refer to different files as time passes, because the meaning of such a pathname depends on the state of the file system. In file systems with such facilities as “links,” multiple file names, logical devices, and so on, two pathnames that look quite different may turn out to address the same file. To access a file given a pathname, one must do a file system operation such as open.
Two important operations involving pathnames are parsing and merging. Parsing is the conversion of a namestring (which might be something supplied interactively by the user when asked to supply the name of a file) into a pathname object. This operation is implementation-dependent, because the format of namestrings is implementation-dependent. Merging takes a pathname with missing components and supplies values for those components from a source of defaults.
Not all of the components of a pathname need to be specified. If a component of a pathname is missing, its value is nil. Before the file system interface can do anything interesting with a file, such as opening the file, all the missing components of a pathname must be filled in (typically from a set of defaults). Pathnames with missing components may be used internally for various purposes; in particular, parsing a namestring that does not specify certain components will result in a pathname with missing components.
X3J13 voted in January 1989 to permit any component of a pathname to have the value :unspecific, meaning that the component simply does not exist, for file systems in which such a value makes sense. (For example, a UNIX file system usually does not support version numbers, so the version component of a pathname for a UNIX host might be :unspecific. Similarly, the file type is usually regarded in a UNIX file system as the part of a name after a period, but some file names contain no periods and therefore have no file types.)
When a pathname is converted to a namestring, the values nil and :unspecific have the same effect: they are treated as if the component were empty (that is, they each cause the component not to appear in the namestring). When merging, however, only a nil value for a component will be replaced with the default for that component; the value :unspecific will be left alone as if the field were filled.
The results are undefined if :unspecific is supplied to a file system in a component for which :unspecific does not make sense for that file system.
Programming hint: portable programs should be prepared to handle the value :unspecific in the device, directory, type, or version field in some implementations. Portable programs should not explicitly place :unspecific in any field because it might not be permitted in some situations, but portable programs may sometimes do so implicitly (by copying such a value from another pathname, for example).
What values are allowed for components of a pathname depends, in general, on the pathname’s host. However, in order for pathnames to be usable in a system-independent way, certain global conventions are adhered to. These conventions are stronger for the type and version than for the other components, since the type and version are explicitly manipulated by many programs, while the other components are usually treated as something supplied by the user that just needs to be remembered and copied from place to place.
The type is always a string or nil or :wild. It is expected that most programs that deal with files will supply a default type for each file.
The version is either a positive integer or a special symbol. The meanings of nil and :wild have been explained above. The keyword :newest refers to the largest version number that already exists in the file system when reading a file, or to a version number greater than any already existing in the file system when writing a new file. Some Common Lisp implementors may choose to define other special version symbols. Some semi-standard names, suggested but not required to be supported by every Common Lisp implementation, are :oldest, to refer to the smallest version number that exists in the file system; :previous, to refer to the version previous to the newest version; and :installed, to refer to a version that is officially installed for users (as opposed to a working or development version). Some Common Lisp implementors may also choose to attach a meaning to non-positive version numbers (a typical convention is that 0 is synonymous with :newest and -1 with :previous), but such interpretations are implementation-dependent.
The host may be a string, indicating a file system, or a list of strings, of which the first names the file system and the rest may be used for such a purpose as inter-network routing.
X3J13 voted in June 1989 to approve the following clarifications and specifications of precisely what are valid values for the various components of a pathname.
Pathname component value strings never contain the punctuation characters that are used to separate fields in a namestring (for example, slashes and periods as used in UNIX file systems). Punctuation characters appear only in namestrings. Characters used as punctuation can appear in pathname component values with a non-punctuation meaning if the file system allows it (for example, UNIX file systems allow a file name to begin with a period).
When examining pathname components, conforming programs must be prepared to encounter any of the following siutations:
When examining wildcard components of a wildcard pathname, conforming programs must be prepared to encounter any of the following additional values in any component or any element of a list that is the directory component:
When constructing a pathname from components, conforming programs must follow these rules:
The best way to compare two pathnames for equality is with equal, not eql. (On pathnames, eql is simply the same as eq.) Two pathname objects are equal if and only if all the corresponding components (host, device, and so on) are equivalent. (Whether or not uppercase and lowercase letters are considered equivalent in strings appearing in components depends on the file name conventions of the file system.) Pathnames that are equal should be functionally equivalent.
Issues of alphabetic case in pathnames are a major source of problems. In some file systems, the customary case is lowercase, in some uppercase, in some mixed. Some file systems are case-sensitive (that is, they treat FOO and foo as different file names) and others are not.
There are two kinds of pathname case portability problems: moving programs from one Common Lisp to another, and moving pathname component values from one file system to another. The solution to the first problem is the requirement that all Common Lisp implementations that support a particular file system must use compatible representations for pathname component values. The solution to the second problem is the use of a common representation for the least-common-denominator pathname component values that exist on all interesting file systems.
Requiring a common representation directly conflicts with the desire among programmers that use only one file system to work with the local conventions and to ignore issues of porting to other file systems. The common representation cannot be the same as local (varying) conventions.
X3J13 voted in June 1989 to add a keyword argument :case to each of the functions make-pathname, pathname-host, pathname-device, pathname-directory, pathname-name, and pathname-type. The possible values for the argument are :common and :local. The default is :local.
The value :local means that strings given to make-pathname or returned by any of the pathname component accessors follow the local file system’s conventions for alphabetic case. Strings given to make-pathname will be used exactly as written if the file system supports both cases. If the file system supports only one case, the strings will be translated to that case.
The value :common means that strings given to make-pathname or returned by any of the pathname component accessors follow this common convention:
Uppercase is used as the common case for no better reason than consistency with Lisp symbols. The second and third points allow translation from local representation to common and back to be information-preserving. (Note that translation from common to local representation and back may or may not be information-preserving, depending on the nature of the local representation.)
Namestrings always use :local file system case conventions.
Finally, merge-pathnames and translate-pathname map customary case in the input pathnames into customary case in the output pathname.
Examples of possible use of this convention:
Here are some examples of this behavior. Assume that the host T runs TOPS-20, U runs UNIX, V runs VAX/VMS, and M runs the Macintosh operating system.
The following example illustrates the creation of new pathnames. The name is converted from common representation to local because namestrings always use local conventions.
A big advantage of this set of conventions is that one can, for example, call make-pathname with :type "LISP" and :case :common, and the result will appear in a namestring as .LISP or .lisp, whichever is appropriate.
X3J13 voted in June 1989 to define a specific pathname component format for structured directories.
The value of a pathname’s directory component may be a list. The car of the list should be a keyword, either :absolute or :relative. Each remaining element of the list should be a string or a symbol (see below). Each string names a single level of directory structure and should consist of only the directory name without any punctuation characters.
A list whose car is the symbol :absolute represents a directory path starting from the root directory. For example, the list (:absolute) represents the root directory itself; the list (:absolute "foo" "bar" "baz") represents the directory that in a UNIX file system would be called /foo/bar/baz.
A list whose car is the symbol :relative represents a directory path starting from a default directory. The list (:relative) has the same meaning as nil and hence normally is not used. The list (:relative "foo" "bar") represents the directory named bar in the directory named foo in the default directory.
In place of a string, at any point in the list, a symbol may occur to indicate a special file notation. The following symbols have standard meanings.
(See section 23.1.4 for a discussion of wildcard pathnames.)
Implementations are permitted to add additional objects of any non-string type if necessary to represent features of their file systems that cannot be represented with the standard strings and symbols. Supplying any non-string, including any of the symbols listed below, to a file system for which it does not make sense signals an error of type file-error. For example, most implementations of the UNIX file system do not support :wild-inferiors. Any directory list in which :absolute or :wild-inferiors is immediately followed by :up or :back is illegal and when processed causes an error to be signaled.
The keyword :back has a “syntactic” meaning that depends only on the pathname and not on the contents of the file system. The keyword :up has a “semantic” meaning that depends on the contents of the file system; to resolve a pathname containing :up to a pathname whose directory component contains only :absolute and strings requires a search of the file system. Note that use of :up instead of :back can result in designating a different actual directory only in file systems that support multiple names for directories, perhaps via symbolic links. For example, suppose that there is a directory link such that
and there also exist directories
Then
but
If a string is used as the value of the :directory argument to make-pathname, it should be the name of a top-level directory and should not contain any punctuation characters. Specifying a string s is equivalent to specifying the list (:absolute s). Specifying the symbol :wild is equivalent to specifying the list (:absolute :wild-inferiors) (or (:absolute :wild) in a file system that does not support :wild-inferiors).
The function pathname-directory always returns nil, :unspecific, or a list—never a string, never :wild. If a list is returned, it is not guaranteed to be freshly consed; the consequences of modifying this list are undefined.
In non-hierarchical file systems, the only valid list values for the directory component of a pathname are (:absolute s) (where s is a string) and (:absolute :wild). The keywords :relative, :wild-inferiors, :up, and :back are not used in non-hierarchical file systems.
Pathname merging treats a relative directory specially. Let pathname and defaults be the first two arguments to merge-pathnames. If (pathname-directory pathname) is a list whose car is :relative, and (pathname-directory defaults) is a list, then the merged directory is the value of
except that if the resulting list contains a string or :wild immediately followed by :back, both of them are removed. This removal of redundant occurrences of :back is repeated as many times as possible. If (pathname-directory defaults) is not a list or (pathname-directory pathname) is not a list whose car is :relative, the merged directory is the value of
A relative directory in the pathname argument to a function such as open is merged with the value of *default-pathname-defaults* before the file system is accessed.
Here are some examples of the use of structured directories. Suppose that host L supports a Symbolics Lisp Machine file system, host U supports a UNIX file system, and host V supports a VAX/VMS file system.
Some file systems provide more complex conventions for wildcards than simple component-wise wildcards representable by :wild. For example, the namestring "F*O" might mean a normal three-character name; a three-character name with the middle character wild; a name with at least two characters, beginning with F and ending with O; or perhaps a wild match spanning multiple directories. Similarly, the namestring ">foo>**>bar>" might imply that the middle directory is named "**"; the middle directory is :wild; there are zero or more middle directories that are :wild; or perhaps that the middle directory name matches any two-letter name. Some file systems support even more complex wildcards, such as regular expressions.
X3J13 voted in June 1989 to provide some facilities for dealing with more general wildcard pathnames in a fairly portable manner.
Tests a pathname for the presence of wildcard components. If the first argument is not a pathname, string, or file stream, an error of type type-error is signaled.
If no field-key is provided, or the field-key is nil, the result is true if and only if pathname has any wildcard components.
If a non-null field-key is provided, it must be one of :host, :device, :directory, :name, :type, or :version. In this case, the result is true if and only if the indicated component of pathname is a wildcard.
Note that X3J13 voted in June 1989 to specify that an implementation need not support wildcards in all fields; the only requirement is that the name, type, or version may be :wild. However, portable programs should be prepared to encounter either :wild or implementation-dependent wildcards in any pathname component. The function wild-pathname-p provides a portable way for testing the presence of wildcards.
This predicate is true if and only if the pathname matches the wildname. The matching rules are implementation-defined but should be consistent with the behavior of the directory function. Missing components of wildname default to :wild.
If either argument is not a pathname, string, or file stream, an error of type type-error is signaled. It is valid for pathname to be a wild pathname; a wildcard field in pathname will match only a wildcard field in wildname; that is, pathname-match-p is not commutative. It is valid for wildname to be a non-wild pathname; I believe that in this case pathname-match-p will have the same behavior as equal, though the X3J13 specification did not say so.
Translates the pathname source, which must match from-wildname, into a corresponding pathname (call it result), which is constructed so as to match to-wildname, and returns result.
The pathname result is a copy of to-wildname with each missing or wildcard field replaced by a portion of source; for this purpose a wildcard field is a pathname component with a value of :wild, a :wild element of a list-valued directory component, or an implementation-defined portion of a component, such as the * in the complex wildcard string "foo*bar" that some implementations support. An implementation that adds other wildcard features, such as regular expressions, must define how translate-pathname extends to those features. A missing field is a pathname component that is nil.
The portion of source that is copied into result is implementation-defined. Typically it is determined by the user interface conventions of the file systems involved. Usually it is the portion of source that matches a wildcard field of from-wildname that is in the same position as the missing or wildcard field of to-wildname. If there is no wildcard field in from-wildname at that position, then usually it is the entire corresponding pathname component of source or, in the case of a list-valued directory component, the entire corresponding list element. For example, if the name components of source, from-wildname, and to-wildname are "gazonk", "gaz*", and "h*" respectively, then in most file systems the wildcard fields of the name component of from-wildname and to-wildname are each "*", the matching portion of source is "onk", and the name component of result is "honk"; however, the exact behavior of translate-pathname is not dictated by the Common Lisp language and may vary according to the user interface conventions of the file systems involved.
During the copying of a portion of source into result, additional implementation-defined translations of alphabetic case or file naming conventions may occur, especially when from-wildname and to-wildname are for different hosts.
If any of the first three arguments is not a pathname, string, or file stream, an error of type type-error is signaled. It is valid for source to be a wild pathname; in general this will produce a wild result pathname. It is valid for from-wildname or to-wildname or both to be non-wild. An error is signaled if the source pathname does not match the from-wildname, that is, if (pathname-match-p source from-wildname) would not be true.
There are no specified keyword arguments for translate-pathname, but implementations are permitted to extend it by adding keyword arguments. There is one specified return value from translate-pathname; implementations are permitted to extend it by returning additional values.
Here is an implementation suggestion. One file system performs this operation by examining corresponding pieces of the three pathnames in turn, where a piece is a pathname component or a list element of a structured component such as a hierarchical directory. Hierarchical directory elements in from-wildname and to-wildname are matched by whether they are wildcards, not by depth in the directory hierarchy. If the piece in to-wildname is present and not wild, it is copied into the result. If the piece in to-wildname is :wild or nil, the corresponding piece in source is copied into the result. Otherwise, the piece in to-wildname might be a complex wildcard such as "foo*bar"; the portion of the piece in source that matches the wildcard portion of the corresponding piece in from-wildname (or the entire source piece, if the from-wildname piece is not wild and therefore equals the source piece) replaces the wildcard portion of the piece in to-wildname and the value produced is used in the result.
X3J13 voted in June 1989 to require translate-pathname to map customary case in argument pathnames to the customary case in returned pathnames (see section 23.1.2).
Here are some examples of the use of the new wildcard pathname facilities. These examples are not portable. They are written to run with particular file systems and particular wildcard conventions and are intended to be illustrative, not prescriptive. Other implementations may behave differently.
One cannot rely on rename-file to handle wild pathnames in a predictable manner. However, one can use translate-pathname explicitly to control the process.
Assuming one particular set of popular wildcard conventions, this function might exhibit the following behavior. Not all file systems will run this example exactly as written.
The following examples use UNIX syntax and the wildcard conventions of one particular version of UNIX.
The following examples are similar to the preceding examples but use two different hosts; host U supports a UNIX file system and host V supports a VAX/VMS file system. Note the translation of file type (from l to LSP) and the change of alphabetic case conventions.
The next example is a version of the function translate-logical-pathname (simplified a bit) for a logical host named FOO. The points of interest are the use of pathname-match-p as a :test argument for assoc and the use of translate-pathname as a substrate for translate-logical-pathname.
Pathname values are not portable, but sometimes they must be mentioned in a program (for example, the names of files containing the program and the data used by the program).
X3J13 voted in June 1989 to provide some facilities for portable pathname values. The idea is to provide a portable framework for pathname values; these logical pathnames are then mapped to physical (that is, actual) pathnames by a set of implementation-dependent or site-dependent rules. The logical pathname facility therefore separates the concerns of program writing and user software architecture from the details of how a software system is embedded in a particular file system or operating environment.
Pathname values are not portable because not all Common Lisp implementations use the same operating system and file name syntax varies widely among operating systems. In addition, corresponding files at two different sites may have different names even when the operating system is the same; for example, they may be on different directories or different devices. The Common Lisp logical pathname system defines a particular pathname structure and namestring syntax that must be supported by all implementations.
The syntax of a logical pathname namestring is as follows:
A word consists of one or more uppercase letters, digits, and hyphens.
A wildcard word consists of one or more asterisks, uppercase letters, digits, and hyphens, including at least one asterisk, with no two asterisks adjacent. Each asterisk matches a sequence of zero or more characters. The wildcard word * parses as :wild; all others parse as strings.
Lowercase letters may also appear in a word or wildcard word occurring in a namestring. Such letters are converted to uppercase when the namestring is converted to a pathname. The consequences of using other characters are unspecified.
The host is a word that has been defined as a logical pathname host by using setf with the function logical-pathname-translations.
There is no device, so the device component of a logical pathname is always :unspecific. No other component of a logical pathname can be :unspecific.
Each directory is a word, a wildcard word, or ** (which is parsed as :wild-inferiors). If a semicolon precedes the directories, the directory component is relative; otherwise it is absolute.
The name is a word or a wildcard word.
The type is a word or a wildcard word.
The version is a positive decimal integer or the word NEWEST (which is parsed as :newest) or * (which is parsed as :wild). The letters in NEWEST can be in either alphabetic case.
The consequences of using any value not specified here as a logical pathname component are unspecified. The null string "" is not a valid value for any component of a logical pathname, since the null string is not a word or a wildcard word.
Logical pathname namestrings are recognized by the functions logical-pathname and translate-logical-pathname. The host portion of the logical pathname namestring and its following colon must appear in the namestring arguments to these functions.
The function parse-namestring recognizes a logical pathname namestring when the host argument is logical or the defaults argument is a logical pathname. In this case the host portion of the logical pathname namestring and its following colon are optional. If the host portion of the namestring and the host argument are both present and do not match, an error is signaled. The host argument is logical if it is supplied and came from pathname-host of a logical pathname. Whether a host argument is logical if it is a string equal to a logical pathname host name is implementation-defined.
The function merge-pathnames recognizes a logical pathname namestring when the defaults argument is a logical pathname. In this case the host portion of the logical pathname namestring and its following colon are optional.
Whether the other functions that coerce strings to pathnames recognize logical pathname namestrings is implementation-defined. These functions include parse-namestring in circumstances other than those described above, merge-pathnames in circumstances other than those described above, the :defaults argument to make-pathname, and the following functions:
Note that many of these functions must accept logical pathnames even though they do not accept logical pathname namestrings.
Some real file systems do not have versions. Logical pathname translation to such a file system ignores the version. This implies that a portable program cannot rely on being able to store in a file system more than one version of a file named by a logical pathname.
The type of a logical pathname for a Common Lisp source file is LISP. This should be translated into whatever implementation-defined type is appropriate in a physical pathname.
The logical pathname host name SYS is reserved for the implementation. The existence and meaning of logical pathnames for logical host SYS is implementation-defined.
File manipulation functions must operate with logical pathnames according to the following requirements:
Converts the argument to a logical pathname and returns it. The argument can be a logical pathname, a logical pathname namestring containing a host component, or a stream for which the pathname function returns a logical pathname. For any other argument, logical-pathname signals an error of type type-error.
Translates a logical pathname to the corresponding physical pathname. The pathname argument is first coerced to a pathname. If it is not a pathname, string, or file stream, an error of type type-error is signaled.
If the coerced argument is a physical pathname, it is returned.
If the coerced argument is a logical pathname, the first matching translation (according to pathname-match-p) of the logical pathname host is applied, as if by calling translate-pathname. If the result is a logical pathname, this process is repeated. When the result is finally a physical pathname, it is returned.
If no translation matches a logical pathname, an error of type file-error is signaled.
translate-logical-pathname may perform additional translations, typically to provide translation of file types to local naming conventions, to accommodate physical file systems with names of limited length, or to deal with special character requirements such as translating hyphens to underscores or uppercase letters to lowercase. Any such additional translations are implementation-defined. Some implementations do no additional translations.
There are no specified keyword arguments for translate-logical-pathname but implementations are permitted to extend it by adding keyword arguments. There is one specified return value from translate-logical-pathname; implementations are permitted to extend it by returning additional values.
If the specified host is not the host component of a logical pathname and is not a string that has been defined as a logical pathname host name by setf of logical-pathname-translations, this function signals an error of type type-error; otherwise, it returns the list of translations for the specified host. Each translation is a list of at least two elements, from-wildname and to-wildname. Any additional elements are implementation-defined. A from-wildname is a logical pathname whose host is the specified host. A to-wildname is any pathname. Translations are searched in the order listed, so more specific from-wildnames must precede more general ones.
(setf (logical-pathname-translations host) translations) sets the list of translations for the logical pathname host to translations. If host is a string that has not previously been used as logical pathname host, a new logical pathname host is defined; otherwise an existing host’s translations are replaced. Logical pathname host names are compared with string-equal.
When setting the translations list, each from-wildname can be a logical pathname whose host is host or a logical pathname namestring s parseable by (parse-namestring s host-object), where host-object is an appropriate object for representing the specified host to parse-namestring. (This circuitous specification dodges the fact that parse-namestring does not necessarily accept as its second argument any old string that names a logical host.) Each to-wildname can be anything coercible to a pathname by application of the function pathname. If to-wildname coerces to a logical pathname, translate-logical-pathname will retranslate the result, repeatedly if necessary.
Implementations may define additional functions that operate on logical pathname hosts (for example, to specify additional translation rules or options).
If a logical pathname host named host (a string) is already defined, this function returns nil. Otherwise, it searches for a logical pathname host definition in an implementation-defined manner. If none is found, it signals an error. If a definition is found, it installs the definition and returns t.
The search used by load-logical-pathname-translations should be documented, as logical pathname definitions will be created by users as well as by Lisp implementors. A typical search technique is to look in an implementation-defined directory for a file whose name is derived from the host name in an implementation-defined fashion.
Returns the pathname that compile-file would write into, if given the same arguments. If the pathname argument is a logical pathname and the :output-file argument is unspecified, the result is a logical pathname. If an implementation supports additional keyword arguments to compile-file, compile-file-pathname must accept the same arguments.
Here is a very simple example of setting up a logical pathname host named FOO. Suppose that no translations are necessary to get around file system restrictions, so all that is necessary is to specify the root of the physical directory tree that contains the logical file system. The namestring syntax in the to-wildname is implementation-specific.
The following is a sample use of that logical pathname. All return values are of course implementation-specific; all of the examples in this section are of course meant to be illustrative and not prescriptive.
Next we have a more complex example, dividing the files among two file servers (U, supporting a UNIX file system, and V, supporting a VAX/VMS file system) and several different directories. This UNIX file system doesn’t support :wild-inferiors in the directory, so each directory level must be translated individually. No file name or type translations are required except for .MAIL to .MBX. The namestring syntax used for the to-wildnames is implementation-specific.
Here are sample uses of logical host PROG. All return values are of course implementation-specific.
Suppose now that we have a program that uses three files logically named MAIN.LISP, AUXILIARY.LISP, and DOCUMENTATION.LISP. The following translations might be provided by a software supplier as examples.
For a UNIX file system with long file names:
For a UNIX file system with 14-character file names, using .lisp as the type:
For a UNIX file system with 14-character file names, using .l as the type (the second translation shortens the compiled file type to .b):
Large programs can be moved between sites without changing any pathnames, provided all pathnames used are logical. A portable system construction tool can be created that operates on programs defined as sets of files named by logical pathnames.
Logical pathname syntax was chosen to be easily translated into the formats of most popular file systems, while still being powerful enough for storing large programs. Although they have hierarchical directories, extended wildcard matching, versions, and no limit on the length of names, logical pathnames can be mapped onto a less capable real file system by translating each directory that is used into a flat directory name, processing wildcards in the Lisp implementation rather than in the file system, treating all versions as :newest, and using translations to shorten long names.
Logical pathname words are restricted to non-case-sensitive letters, digits, and hyphens to avoid creating problems with real file systems that support limited character sets for file naming. (If logical pathnames were case-sensitive, it would be very difficult to map them into a file system that is not sensitive to case in its file names.)
It is not a goal of logical pathnames to be able to represent all possible file names. Their goal is rather to represent just enough file names to be useful for storing software. Real pathnames, in contrast, need to provide a uniform interface to all possible file names, including names and naming conventions that are not under the control of Common Lisp.
The choice of logical pathname syntax, using colon, semicolon, and period, was guided by the goals of being visually distinct from real file systems and minimizing the use of special characters.
The logical-pathname function is separate from the pathname function so that the syntax of logical pathname namestrings does not constrain the syntax of physical pathname namestrings in any way. Logical pathname syntax must be defined by Common Lisp so that logical pathnames can be conveniently exchanged between implementations, but physical pathname syntax is dictated by the operating environments.
The compile-file-pathname function and the specification of LISP as the type of a logical pathname for a Common Lisp source file together provide enough information about compilation to make possible a portable system construction tool. Suppose that it is desirable to call compile-file only if the source file is newer than the compiled file. For this to succeed, it must be possible to know the name of the compiled file without actually calling compile-file. In some implementations the compiler produces one of several file types, depending on a variety of implementation-dependent circumstances, so it is not sufficient simply to prescribe a standard logical file type for compiled files; compile-file-pathname provides access to the defaulting that is performed by compile-file “in a manner appropriate to the implementation’s file system conventions.”
The use of the logical pathname host name SYS for the implementation is current practice. Standardizing on this name helps users choose logical pathname host names that avoid conflicting with implementation-defined names.
Loading of logical pathname translations from a site-dependent file allows software to be distributed using logical pathnames. The assumed model of software distribution is a division of labor between the supplier of the software and the user installing it. The supplier chooses logical pathnames to name all the files used or created by the software, and supplies examples of logical pathname translations for a few popular file systems. Each example uses an assumed directory and/or device name, assumes local file naming conventions, and provides translations that will translate all the logical pathnames used or generated by the particular software into valid physical pathnames. For a powerful file system these translations can be quite simple. For a more restricted file system, it may be necessary to list an explicit translation for every logical pathname used (for example, when dealing with restrictions on the maximum length of a file name).
The user installing the software decides on which device and directory to store the files and edits the example logical pathname translations accordingly. If necessary, the user also adjusts the translations for local file naming conventions and any other special aspects of the user’s local file system policy and local Common Lisp implementation. For example, the files might be divided among several file server hosts to share the load. The process of defining site-customized logical pathname translations is quite easy for a user of a popular file system for which the software supplier has provided an example. A user of a more unusual file system might have to take more time; the supplier can help by providing a list of all the logical pathnames used or generated by the software.
Once the user has created and executed a suitable setf form for setting the logical-pathname-translations of the relevant logical host, the software can be loaded and run. It may be necessary to use the translations again, or on another workstation at the same site, so it is best to save the setf form in the standard place where it can be found later by load-logical-pathname-translations. Often a software supplier will include a program for restoring software from the distribution medium to the file system and a program for loading the software from the file system into a Common Lisp; these programs will start by calling load-logical-pathname-translations to make sure that the logical pathname host is defined.
Note that the setf of logical-pathname-translations form isn’t part of the program; it is separate and is written by the user, not by the software supplier. That separation and a uniform convention for doing the separation are the key aspects of logical pathnames. For small programs involving only a handful of files, it doesn’t matter much. The real benefits come with large programs with hundreds or thousands of files and more complicated situations such as program-generated file names or porting a program developed on a system with long file names onto a system with a very restrictive limit on the length of file names.
These functions are what programs use to parse and default file names that have been typed in or otherwise supplied by the user.
Any argument called pathname in this book may actually be a pathname, a string or symbol, or a stream. Any argument called defaults may likewise be a pathname, a string or symbol, or a stream.
X3J13 voted in March 1988 to change the language so that a symbol is never allowed as a pathname argument. More specifically, the following functions are changed to disallow a symbol as a pathname argument:
(The function require was also changed by this vote but was deleted from the language by a vote in January 1989 .) Furthermore, the vote reaffirmed that the following functions do not accept symbols as file, filename, or pathname arguments:
In older implementations of Lisp that did not have strings, for example MacLisp, symbols were the only means for specifying pathnames. This was convenient only because the file systems of the time allowed only uppercase letters in file names. Typing (load ’foo) caused the function load to receive the symbol FOO (with uppercase letters because of the way symbols are parsed) and therefore to load the file named FOO. Now that many file systems, most notably UNIX, support case-sensitive file names, the use of symbols is less convenient and more error-prone.
X3J13 voted in March 1988 to specify that a stream may be used as a pathname, file, or filename argument only if it was created by use of open or with-open-file, or if it is a synonym stream whose symbol is bound to a stream that may be used as a pathname.
If such a stream is used as a pathname, it is as if the pathname function were applied to the stream and the resulting pathname used in place of the stream. This represents the name used to open the file. This may be, but is not required to be, the actual name of the file.
It is an error to attempt to obtain a pathname from a stream created by any of the following:
In the examples, it is assumed that the host named CMUC runs the TOPS-20 operating system, and therefore uses TOPS-20 file system syntax; furthermore, an explicit host name is indicated by following the host name with a double colon. Remember, however, that namestring syntax is implementation-dependent, and this syntax is used here purely for the sake of examples.
The pathname function converts its argument to be a pathname. The argument may be a pathname, a string or symbol, or a stream; the result is always a pathname.
X3J13 voted in March 1988 not to permit symbols as pathnames and to specify exactly which streams may be used as pathnames .
X3J13 voted in January 1989 to specify that pathname is unaffected by whether its argument, if a stream, is open or closed. X3J13 further commented that because some implementations cannot provide the “true name” of a file until the file is closed, in such an implementation pathname might, in principle, return a different (perhaps more specific) file name after the stream is closed. However, such behavior is prohibited; pathname must return the same pathname after a stream is closed as it would have while the stream was open. See truename.
The truename function endeavors to discover the “true name” of the file associated with the pathname within the file system. If the pathname is an open stream already associated with a file in the file system, that file is used. The “true name” is returned as a pathname. An error is signaled if an appropriate file cannot be located within the file system for the given pathname.
The truename function may be used to account for any file name translations performed by the file system, for example.
For example, suppose that DOC: is a TOPS-20 logical device name that is translated by the TOPS-20 file system to be PS:<DOCUMENTATION>.
X3J13 voted in March 1988 not to permit symbols as pathnames and to specify exactly which streams may be used as pathnames .
X3J13 voted in January 1989 to specify that truename may be applied to a stream whether the stream is open or closed. X3J13 further commented that because some implementations cannot provide the “true name” of a file until the file is closed, in principle it would be possible in such an implementation for truename to return a different file name after the stream is closed. Such behavior is permitted; in this respect truename differs from pathname.
X3J13 voted in June 1989 to clarify that truename accepts only non-wild pathnames; an error is signaled if wild-pathname-p would be true of the pathname argument.
X3J13 voted in June 1989 to require truename to accept logical pathnames (see section 23.1.5). However, truename never returns a logical pathname.
This turns thing into a pathname. The thing is usually a string (that is, a namestring), but it may be a symbol (in which case the print name is used) or a pathname or stream (in which case no parsing is needed, but an error check may be made for matching hosts).
X3J13 voted in March 1988 not to permit symbols as pathnames and to specify exactly which streams may be used as pathnames . The thing argument may not be a symbol.
X3J13 voted in June 1989 to require parse-namestring to accept logical pathname namestrings (see section 23.1.5).
This function does not, in general, do defaulting of pathname components, even though it has an argument named defaults; it only does parsing. The host and defaults arguments are present because in some implementations it may be that a namestring can only be parsed with reference to a particular file name syntax of several available in the implementation. If host is non-nil, it must be a host name that could appear in the host component of a pathname, or nil; if host is nil then the host name is extracted from the default pathname in defaults and used to determine the syntax convention. The defaults argument defaults to the value of *default-pathname-defaults*.
For a string (or symbol) argument, parse-namestring parses a file name within it in the range delimited by the :start and :end arguments (which are integer indices into string, defaulting to the beginning and end of the string).
See chapter 14 for a discussion of :start and :end arguments.
If :junk-allowed is not nil, then the first value returned is the pathname parsed, or nil if no syntactically correct pathname was seen.
If :junk-allowed is nil (the default), then the entire substring is scanned. The returned value is the pathname parsed. An error is signaled if the substring does not consist entirely of the representation of a pathname, possibly surrounded on either side by whitespace characters if that is appropriate to the cultural conventions of the implementation.
In either case, the second value is the index into the string of the delimiter that terminated the parse, or the index beyond the substring if the parse terminated at the end of the substring (as will always be the case if :junk-allowed is false).
If thing is not a string or symbol, then start (which defaults to zero in any case) is always returned as the second value.
Parsing an empty string always succeeds, producing a pathname with all components (except the host) equal to nil.
Note that if host is specified and not nil, and thing contains a manifest host name, an error is signaled if the hosts do not match.
If thing contains an explicit host name and no explicit device name, then it might be appropriate, depending on the implementation environment, for parse-namestring to supply the standard default device for that host as the device component of the resulting pathname.
X3J13 voted in March 1988 not to permit symbols as pathnames and to specify exactly which streams may be used as pathnames .
X3J13 voted in June 1989 to require merge-namestrings to recognize a logical pathname namestring as its first argument if its second argument is a logical pathname (see section 23.1.5).
X3J13 voted in January 1989 to specify that merge-pathname is unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, merge-pathname behaves as if the function pathname were applied to the stream and the resulting pathname used instead.
X3J13 voted in June 1989 to require merge-pathnames to map customary case in argument pathnames to the customary case in returned pathnames (see section 23.1.2).
defaults
defaults to the value of *default-pathname-defaults*.default-version
defaults to :newest.Here is an example of the use of merge-pathnames:
Defaulting of pathname components is done by filling in components taken from another pathname. This is especially useful for cases such as a program that has an input file and an output file, and asks the user for the name of both, letting the unsupplied components of one name default from the other. Unspecified components of the output pathname will come from the input pathname, except that the type should default not to the type of the input but to the appropriate default type for output from this program.
The pathname merging operation takes as input a given pathname, a defaults pathname, and a default version, and returns a new pathname. Basically, the missing components in the given pathname are filled in from the defaults pathname, except that if no version is specified the default version is used. The default version is usually :newest; if no version is specified the newest version in existence should be used. The default version can be nil, to preserve the information that it was missing in the input pathname.
If the given pathname explicitly specifies a host and does not supply a device, then if the host component of the defaults matches the host component of the given pathname, then the device is taken from the defaults; otherwise the device will be the default file device for that host. Next, if the given pathname does not specify a host, device, directory, name, or type, each such component is copied from the defaults. The merging rules for the version are more complicated and depend on whether the pathname specifies a name. If the pathname doesn’t specify a name, then the version, if not provided, will come from the defaults, just like the other components. However, if the pathname does specify a name, then the version is not affected by the defaults. The reason is that the version “belongs to” some other file name and is unlikely to have anything to do with the new one. Finally, if this process leaves the version missing, the default version is used.
The net effect is that if the user supplies just a name, then the host, device, directory, and type will come from the defaults, but the version will come from the default version argument to the merging operation. If the user supplies nothing, or just a directory, the name, type, and version will come over from the defaults together. If the host’s file name syntax provides a way to input a version without a name or type, the user can let the name and type default but supply a version different from the one in the defaults.
X3J13 voted in June 1989 to agree to disagree: merge-pathname might or might not perform plausibility checking on its arguments to ensure that the resulting pathname can be converted a valid namestring. User beware: this could cause portability problems.
For example, suppose that host LOSER constrains file types to be three characters or fewer but host CMUC does not. Then "LOSER::FORMAT" is a valid namestring and "CMUC::PS:<LISPIO>.FASL" is a valid namestring, but
might signal an error in some implementations because the hypothetical result would be a pathname equivalent to the namestring "LOSER::FORMAT.FASL" which is illegal because the file type FASL has more than three characters. In other implementations merge-pathname might return a pathname but that pathname might cause namestring to signal an error.
This is the default pathname-defaults pathname; if any pathname primitive that needs a set of defaults is not given one, it uses this one. As a general rule, however, each program should have its own pathname defaults rather than using this one.
The following example assumes the use of UNIX syntax and conventions.
X3J13 voted in June 1989 to add a new keyword argument :case to make-pathname. The new argument description is therefore as follows:
See section 23.1.2 for a description of the :case argument.
X3J13 voted in June 1989 to agree to disagree: make-pathname might or might not check on its arguments to ensure that the resulting pathname can be converted to a valid namestring. If make-pathname does not check its arguments and signal an error in problematical cases, namestring yet might or might not signal an error when given the resulting pathname. User beware: this could cause portability problems.
This predicate is true if object is a pathname, and otherwise is false.
X3J13 voted in March 1988 not to permit symbols as pathnames and to specify exactly which streams may be used as pathnames .
X3J13 voted in January 1989 to specify that these operations are unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, each operation behaves as if the function pathname were applied to the stream and the resulting pathname used instead.
X3J13 voted in June 1989 to add a keyword argument :case to all of the pathname accessor functions except pathname-version. The new argument descriptions are therefore as follows:
[Function]
pathname-host pathname &key :caseSee section 23.1.2 for a description of the :case argument.
X3J13 voted in June 1989 to specify that pathname-directory always returns nil, :unspecific, or a list—never a string, never :wild (see section 23.1.3). If a list is returned, it is not guaranteed to be freshly consed; the consequences of modifying this list are undefined.
[Function]
namestring pathnameThe pathname argument may be a pathname, a string or symbol, or a stream that is or was open to a file. The name represented by pathname is returned as a namelist in canonical form.
If pathname is a stream, the name returned represents the name used to open the file, which may not be the actual name of the file (see truename).
X3J13 voted in March 1988 not to permit symbols as pathnames and to specify exactly which streams may be used as pathnames .
X3J13 voted in January 1989 to specify that these operations are unaffected by whether the first argument, if a stream, is open or closed. If the first argument is a stream, each operation behaves as if the function pathname were applied to the stream and the resulting pathname used instead.
namestring returns the full form of the pathname as a string. file-namestring returns a string representing just the name, type, and version components of the pathname; the result of directory-namestring represents just the directory-name portion; and host-namestring returns a string for just the host-name portion. Note that a valid namestring cannot necessarily be constructed simply by concatenating some of the three shorter strings in some order.
enough-namestring takes another argument, defaults. It returns an abbreviated namestring that is just sufficient to identify the file named by pathname when considered relative to the defaults (which defaults to the value of *default-pathname-defaults*). That is, it is required that
in all cases; and the result of enough-namestring is, roughly speaking, the shortest reasonable string that will still satisfy this criterion.
Returns a pathname for the user’s “home directory” on host. The host argument defaults in some appropriate implementation-dependent manner. The concept of “home directory” is itself somewhat implementation-dependent, but from the point of view of Common Lisp it is the directory where the user keeps personal files such as initialization files and mail. If it is impossible to determine this information, then nil is returned instead of a pathname; however, user-homedir-pathname never returns nil if the host argument is not specified. This function returns a pathname without any name, type, or version component (those components are all nil).