Pup File Transfer Protocol Package
This package is a collection of modules implementing the Pup File and
Mail Transfer Protocols. The package is used by the FTP subsystem and
the Interim File System.
1. Overview
This document is organized as a general overview followed by
descriptions of each of the modules in the package. A history of
revisions to the package is included at the end.
Before beginning the main documentation, some general comments are in
order.
a. The File Transfer Protocol is (alas) complex; this package
requires the Pup package and all of its supporting packages plus
some other packages not specific to Pup. This documentation is
less tutorial than normal Alto package descriptions so you should
be prepared to consult its author.
b. This document describes the external program interfaces for a
particular implementation of the File Transfer Protocol, and does
not deal with the internal implementation nor the reasons for
design choices in the protocol or the implementation. Before
considering the details of this package, you should read
[Maxc]<Pup>FtpSpec.press to get the flavor of how the File
Transfer Protocol works. The <Pup> directory also contains
descriptions of the lower level protocols on which FTP is based.
Detailed knowledge of these protocols is not necessary to use this
package, but you must be familiar with the operation of the Pup
package.
c. This package and the protocol are under active development so
users should expect modifications and extensions.
d. This package is designed to run under several operating
systems and with several file systems. Functions are carefuly
split into protocol-specific and environment-specific modules.
This package provides the protocol modules; you must write the
matching environment-specific modules.
1.1. Organization
The FTP package comes in four modules: Server, User, Utilities, and
Property lists. The utility and property list modules are shared by
the User and Server.
The User and Server modules implement their respective halves of the
protocol exchanges.
------------
Copyright Xerox Corporation 1982
Pup FTP Package December 12, 1981 2
The Property List module generates and parses property lists,
filesystem-independent descriptions of files. When passed between User
and Server FTPs through the network byte stream, their form is defined
by protocol as a parenthesized list. When passed between these
protocol modules and the user-supplied modules in a program, they take
the form of a data structure defined by this package.
The Utility module contains protocol routines shared by the User and
Server modules and some efficient routines for transferring data
between a network stream and a disk stream.
1.2. File Conventions
The FTP package is distributed as file FTPPackage.dm, and contains the
following files:
User
FtpUserProt.br User protocol common to file and mail
FtpUserProtFile.br User file commands
FtpUserProtMail.br User mail commands
Server
FtpServProt.br Server protocol common to file and mail
FtpServProtFile.br Server file commands
FtpServProtMail.br Server mail commands
Property lists
FtpPListProt.br Property list protocol
FtpPListImpl.br Implements a 'standard' property list
FtpPListInit.br Initialization
Utility
FtpUtilB.br Common protocol
FtpUtilXfer.br Unformatted data transfer
FtpUtilDmpLd.br Dump/Load data transfer
FtpUtilCompB.br Binary compare data transfer
FtpUtilCompA.br Binary compare data transfer
FtpUtilA.br Assembly-language utility code
FtpUtilInit.br Initialization
Definitions
FtpProt.decl Protocol parameters and structures
Command files
CompileFtpPackage.cm Compiles all files
DumpFtpPackage.cm A list of all binary files
FtpPackage.cm A list of all source files
All of these modules are swappable, and are broken up into pieces no
larger than 1024 words. Modules whose names end in "init" are
initialization code which should be executed once and thrown away.
The source files are kept with the subsystem sources in FTP.dm and are
formatted for printing in a small fixed-pitch font such as Gacha8 (use
the command 'Empress @FtpPackage.cm@').
Pup FTP Package December 12, 1981 3
1.3. Other Packages
FTP is a level 3 Pup protocol and this package uses a number of other
Alto software packages. As always, files whose names end in "init" may
be discarded after initialization (except ContextInit.br).
Pup Package
PupBSPOpenClose PupBSPStreams.br PupBSPProt.br
PupBSPa.br PupBSPBlock.br
PupRTP.br PupRTPOpenClose PupDummyGate.brPupRoute.br
Pup1b.br Pup1OpenClose PupAl1a.br Pup1Init.br
PupAlEthb.br PupAlEtha.br PupAlEthInit.br
Context Package
Context.br ContextInit.br
Interrupt Package
Interrupt.br InterruptInit.br
Queue, Timer, and ByteBLT Packages
AltoQueue.br AltoTimer.br AltoByteBLT.br
Time Package
TimeConvB.br TimeConvA.br TimeIO.br
CmdScan Package
Keyword.br KeywordInit.br
Strings and Template Packages
StringUtil.br Template.br
1.4. Principal Data Structures
The following data structures are of interest to users, and together
with the procedures described later, constitute the package interface.
PL Property List, is this implementation's internal
representation of the protocol-specified property list.
FTPI File Transfer Package Interface, contains pointers to the
network byte stream, user disk stream, log stream, the file
buffer, and various flags.
FTPSI FTP Server Interface, is a vector of user-supplied procedures
constituting the interface between the protocol and
environment-specific Server modules.
FtpCtx FTP Context, is the process-private storage for an instance of
a User or Server FTP. It consists of an FTPI, and if the
process is a Server, an FTPSI. This is a convenient place for
the user-supplied modules to keep process-private data. You
can do this by adding items to the FtpCtx definition and then
recompiling everything.
The entire FtpCtx need not be filled in all of the time. For each
group of procedures, the items they require will be specified. A
general description of the contents of the FTPI part of an FtpCtx is in
order here.
bspSoc a pointer to a BSP socket open to a remote FTP
process.
bspStream a pointer to the stream in the above BSP socket.
Pup package experts will recognize that this is
Pup FTP Package December 12, 1981 4
redundant, but it is often convenient and makes the
code clearer.
dspStream a pointer to a stream to which this package will
output generally useful information, including
copious amounts of debugging information if
debugFlag is true. The only operation that need be
defined is 'Puts'.
debugFlag a boolean. If true, the protocol exchanges for this
context are output to dspStream as text, along with
some other useful information. Use this! It will
save you much head-scratching.
connFlag a boolean. This should be true if bspSoc is open.
The package will cooperate in maintaining this flag,
which is valuble when finishing.
serverFlag a boolean. This flag is tested by procedures in the
shared modules to determine whether the caller is a
User or Server.
getCmdString a pointer to the last string read by the GetCommand
procedure in the Utility module. Commands with
string arguments are No, Yes, Version,
MailBoxException, and Comment.
The following items are used by the data transfer routines in the
Utility module. The routines are Alto-specific and in some cases Ftp
subsystem-specific, so these items need not be filled in if you do not
use the routines.
diskStream a pointer to a disk stream. It should always be
opened in byte mode.
buffer a pointer to a block of memory which can be used for
block transfer I/O operations. The bigger this is
the faster things will go.
bufferLength the length in words of the above buffer.
1.5. Programming Conventions
This package can be used with the Bcpl Overlay package. File
FtpOEPInit.br contains a procedure which will help do this, but you
should consult with the author.
This package does a lot of string manipulation, and uses the following
conventions:
a. All strings are allocated from 'sysZone'.
b. Strings are represented in data structures (such as property
lists) as addresses. Zero means no string is present.
All of the procedures in this package expect to execute in contexts (in
the sense of the Context package), and expect CtxRunning (defined by
the Context package) to point to an appropriately filled in FtpCtx.
Pup FTP Package December 12, 1981 5
1.6. Property Lists
In most contexts, there are two property lists: one generated by the
client of the package, and one generated by the package. A client-
generated property list is referred to as a 'localPL', and it is the
client's responsibility to free it when it is no longer needed.
Property lists created by this package are referred to as 'remotePLs'
since they are copies of property lists generated remotely; they should
never be freed by the client.
2. Server
The FTP Server module consists of three files: FtpServProt.br, routines
common to the file and mail servers, FtpServProtFile.br, file commands,
and FtpServProtMail.br, mail commands. The server module has one
public procedure:
FtpServProt(timeout)
which carrys out protocol commands received over bspStream by
calling the user-supplied procedures in FTPSI. When the BSP
connection is closed by the remote User process, this procedure
returns. FtpServProt passes 'timeout' to GetCommand (in the
utility module) when waiting for top-level commands (retrieve,
store, delete, etc.). This permits the server to break
connections that don't seem to be doing anything.
This module uses the following fields in FtpCtx: dspStream, bspStream,
bspSoc, and FTPSI. The manifest constant MTP in FtpProt.decl
conditionally compiles calls on the MTP commands. The package is
released with this switch false, since I expect only IFS will need it.
All of the FTP commands (Version, Store, Retrieve, etc.) must contain
procedures (except the MTP ones if the MTP switch is false). If you do
not wish to implement a command, it is sufficient to point the command
at:
and NYI(nil, nil) = valof
[
FTPM(markNo, 1, "Unimplemented Command")
resultis false
]
in which case any subsidiary procedures for that command (such as
StoreFile and StoreCleanup for the Store command) need not be filled
in. FTPM is described in more detail below.
2.1. Version Command
By convention, Version is the first command exchanged over a newly
opened FTP connection. The User sends its protocol version number and
a string such as "Maxc Pup Ftp User 1.04 19-Mar-77". When FtpServProt
receives this command, it replys with its protocol version number and
then calls
(CtxRunning>>FtpCtx.Version)(bspStream, nil)
Pup FTP Package December 12, 1981 6
which should generate some herald text:
Wss(CtxRunning>>FtpCtx.bspStream, "Alto Pup FTP Server 1.13 14-
May-77")
2.2. Retrieve Command
When the remote FTP User process sends the command 'Retrieve' and a
property list describing the files it wants to retrieve, FtpServProt
parses the property list and calls
(CtxRunning>>FtpCtx.Retrieve)(remotePL, localPL)
which should decide whether to accept the command. Retrieve's decision
may involve checking passwords, looking up files, and other actions
using the information in remotePL plus other environment-specific
information, such as whether the requester has the correct
capabilities, etc. To refuse the request, Retrieve should call
FTPM(markNo, code, string)
and return false. To accept the command, it should return a new
property list, localPL, describing a file matching remotePL which
Retrieve is willing to send. FtpServProt returns this PL to you as
'localPL' in the next call to Retrieve, so that you can free it. On
the first call, localPL will be zero. Some FTP implementations require
a minimum set of properties here, but the whole subject of who should
specify what properties is rather involved and beyond the scope of this
description. For more information, consult the FTP specification.
This package provides a fast procedure (in the Utility module) for
deciding the 'type' of a file (text or binary) which you may find
useful.
Property lists in retrieve requests may specify multiple files, so
FtpServProt will continue to call Retrieve until it returns false (no
more files). On each call, remotePL will be the same original PL sent
from the remote User, and localPL will be the last PL returned by
Retrieve. If Retrieve supports multiple file requests then it must
save some information so that the next time FtpServProt calls it, it
can generate the next file. If Retrieve does not support multiple file
requests then it should do its thing during the first call and remember
that it is finished. The next time it is called it should return false
having only deallocated localPL (it should not call FTPM).
If Retrieve returns a PL, FtpServProt sends it back to the User to more
fully describe the file. At this point the User may back out of the
transfer, in which case the next procedure will be skipped, and
RetrieveCleanup will be called immediately. If the User indicates a
willingness to proceed, FtpServProt then calls
(CtxRunning>>FtpCtx.RetrieveFile)(localPL, remotePL)
to transfer the file data. This package provides a procedure (in the
Utility module) for transferring data from a disk Stream to a BSP
Stream, but you are free write your own. When RetrieveFile has
finished the transfer, it should return true if everything went OK. If
something bad happened, it should call
FTPM(markNo, code, string)
Pup FTP Package December 12, 1981 7
and return false. In any case, FtpServProt calls
(CtxRunning>>FtpCtx.RetrieveCleanup)(localPL, ok, remotePL)
where 'ok' is false if RetrieveFile returned false or the User backed
out of the command. Note that if Retrieve returned true,
RetrieveCleanup will always be called, but RetrieveFile may not. If
Retrieve allocates any resources (such as opening a file) they should
be deallocated here.
Finally, FtpServProt calls Retrieve again, and the process repeats
until Retrieve returns false.
2.3. Store Command
When the remote FTP User process sends the command 'newStore' followed
by a property list describing the file, FtpServProt parses the property
list and calls
(CtxRunning>>FtpCtx.Store)(remotePL)
which should decide whether to accept the command. To accept, Store
should return a property list (referred to as localPL below) specifying
the destination file (localPL will be passed to StoreCleanup so you can
free it). To refuse the command Store should call FTPM(markNo, code,
string) and return false, in which case the next procedure (StoreFile)
is not called.
If Store returns true, FtpServProt sends the PL to the User and then
calls
(CtxRunning>>FtpCtx.StoreFile)(remotePL, localPL)
to transfer the file data. This package provides a procedure (in the
Utility module) for transferring data from a BSP Stream to a disk
Stream, but you may write your own. When StoreFile has finished the
transfer, it should return true if everything went OK. If something
bad happened, it should call
FTPM(markNo, code, string)
and return false. Finally, FtpServProt calls
(CtxRunning>>FtpCtx.StoreCleanup)(remotePL, ok, localPL)
where 'ok' is true if StoreFile returned true and the User indicated
that everything went ok. If 'ok' is false, StoreCleanup should delete
the file, since it is almost certainly damaged. Note that if Store
returned true, StoreCleanup will always be called, but StoreFile may
not. If Store allocates any resources (such as opening a file) they
should be deallocated here.
2.4. Delete Command
When the remote FTP User process sends the command 'Delete' followed by
a property list describing the files which it wants to delete,
FtpServProt parses the property list and calls
Pup FTP Package December 12, 1981 8
(CtxRunning>>FtpCtx.Delete)(remotePL, localPL)
which should decide whether to accept the command. Don't delete
anything yet! The User may still back out. To refuse the delete
request, Delete should call FTPM(markNo, code, string) and return
false. To accept the command, it should return a new PL with every
property it can find, so that the User can be sure of the identity of
file to be deleted. FtpServProt will return this PL as 'localPL' in
the next call to Delete, so that it can be deallocted.
Property lists in delete requests may specify multiple files, so
FtpServProt will continue to call Delete until it returns false. On
each call, remotePL will be the same original PL sent from the remote
User, and localPL will be the last PL returned by Delete. If Delete
supports multiple file requests then it must save some information so
that the next time FtpServProt calls it, it can generate the PL for the
next file. If Delete does not support multiple file requests then it
should do its thing during the first call and remember that it is
finished. The next time it is called it should return false having
only deallocated localPL (it should not call FTPM).
If Delete returns a PL, FtpServProt sends it back to the User and waits
for confirmation. If the User still wants to delete the file,
FtpServProt calls
(CtxRunning>>FtpCtx.DeleteFile)(localPL, remotePL)
which should delete the file and return true. If something goes wrong,
it should call
FTPM(markNo, code, string)
and return false. Finally, FtpServProtFile calls Delete again, and the
process repeats until Delete returns false.
2.5. Directory Command
When the remote FTP User process sends the command 'Directory' followed
by a property list naming the files about which it wants information,
FtpServProt parses the property lists and calls
(CtxRunning>>FtpCtx.Directory)(remotePL, localPL)
which should decide whether to accept the command. To refuse the
request (because for example the requestor does not have the correct
access capabilities) Directory should call FTPM(markNo, code, string)
and return false. To accept the command it should return a PL
describing a file.
Property lists in directory requests may specify multiple files, so
FtpServProt will continue to call Directory until it returns false. If
Directory supports multiple file requests then it must save some
information so that the next time FtpServProt calls it, it can generate
the PL for the next file. If Directory does not support multiple file
requests then it should do its thing during the first call and remember
that it is finished. The next time it is called it should return false
having only deallocated localPL (it should not call FTPM).
Pup FTP Package December 12, 1981 9
2.6. Rename Command
When the remote FTP User process sends the command 'Rename' followed by
two property lists describing the old and new files, FtpServProt parses
the property lists and calls
(CtxRunning>>FtpCtx.Rename)(oldPL, newPL)
which should decide whether to accept the command. The FTP protocol
does not require that user access information be present in newPL, so
access checking should be done on oldPl only. To refuse the rename
request, Rename should call FTPM(markNo, code, string) and return
false. Otherwise it should rename the file returning true if
successful. If the rename operation fails, Rename should call
FTPM(markNo, code, string)
and return false.
2.7. Mail Protocol
File FtpServProtMail.br implements the server part of the Mail Transfer
Protocol. This description ignores various critical sections and other
vital considerations which must be handled by the user-supplied
routines in order to provide a reliable mail service. For the
semantics of the protocol see [Maxc]<Pup>MailTransfer.press.
2.8. StoreMail Command
When the remote FTP User process sends the command 'StoreMail',
FtpServProt parses the property lists which follow and for each one
calls
(CtxRunning>>FtpCtx.StoreMail)(remotePL)
which should return true or false. Returning true has nothing to do
with whether the mailbox is valid, it just indicates that the command
exchange may continue. If the mailbox is invalid, StoreMail should
call FTPM(markMailboxException, code, string) and return true.
Returning false terminates the exchange: StoreMailFile is skipped and
StoreMailCleanup is called. StoreMail is called with a zero PL the
last time so that it may reply No and return false if none of the
mailboxes are valid.
If StoreMail always returns true, FtpServProt tells the User process to
go ahead and send the mail, and then calls
(CtxRunning>>FtpCtx.StoreMailMessage)()
to transfer the file data. When StoreMaiMessage has finished the
transfer, it should return true if everything went OK. If something
went wrong, it should call
FTPM(markNo, code, string)
and return false. Finally, FtpServProt calls
(CtxRunning>>FtpCtx.StoreMailCleanup)(ok)
Pup FTP Package December 12, 1981 10
where 'ok' is true if StoreMailMessage returned true and the remote
User indicated that everything went ok. If 'ok' is false,
StoreMailCleanup should not deliver the mail. Note that if StoreMail
is ever called, StoreMailCleanup is always called, but StoreMailMessage
may not be. If StoreMail allocates any resources (such as opening a
file) they should be deallocated here.
2.9. RetrieveMail Command
When the remote FTP User process sends the command RetrieveMail
followed by a property list describing the mailbox, FtpServProt parses
the property list into 'remotePList' and then enters a loop:
First FtpServProt calls
(CtxRunning>>FtpCtx.RetrieveMail)(remotePL, localPL)
which should return a PL describing the next message in the
mailbox. If there are no more unread messages in the mailbox,
RetrieveMail should return zero. On each call, remotePL is
the same original PL sent from the remote User, and localPL is
the last PL returned by RetrieveMail, which should be freed by
the client. On the first call localPL is zero. If
RetrieveMail returns a PL, FtpServProt calls.
(CtxRunning>>FtpCtx.RetrieveMailMessage)(remotePL,
localPL)
which should transfer the file and return true. If something
goes wrong, it should call FTPM(markNo, code string) and
return false.
Finally, FtpServProt calls
(CtxRunning>>FtpCtx.RetrieveMailCleanup)(remotePL, ok).
If 'ok' is true, then RetrieveMailCleanup should flush the mailbox. If
this operation fails, RetrieveMailCleanup should call FTPM(markNo,
code, string) and return false, otherwise it should return true. If
any resouces were allocated during the command, they should be
deallocated here.
3. User
The FTP User module (files FtpUserProt.br, FtpUserProtFile.br, and
FtpUserProtMail.br) implements the User protocol exchanges.
Many of the procedures in this module report results by returning a
word containing an FTP mark code in the right byte and a subcode in the
left byte (referred to below as 'subcode,,mark'). Marks and subcodes
are the first two arguments to the FTPM procedure which is described in
more detail in the Utility section. If the mark type is 'markNo', the
subcode describes the reason why the Server refused; your modules may
be able to fix the problem and retry the command. The package will
output to dspStream text accompanying No, Version, and Comment marks.
Pup FTP Package December 12, 1981 11
3.1. Common User Protocol
File FtpUserProt.bcpl contains routines shared by FtpUserProtFile.br
and FtpUserProtMail.br. It uses the bspStream, bspSoc, and dspStream
fields in its FtpCtx and contains the following external procedures:
UserOpen(Version) = true|false
UserOpen should be called after the BSP Connection is open. It
sends a version command and aborts the connection returning false
if the Server's protocol is incompatible. Otherwise it calls
Version(stream, nil)
which should generate some herald text:
Wss(stream, "Alto Pup FTP User 1.13, 4 June 78").
The herald string received from the Server is output to dspStream.
UserClose(abortIt)
UserClose closes the FTP connection, aborting it if 'abortIt' is
true.
UserFlushEOC() = true|false
flushes bspStream up to the next command, and returns true if it is
EndOfCommand. If the stream closes or times out, it returns false.
It calls UserProtocolError if it encounters anything except an EOC.
UserGetYesNo(flushEOC) = subcode,,mark
flushes bspStream up to the next command, which must be 'Yes' or
'No'. If flushEOC is true, it then calls UserFlushEOC and returns
the Yes or No mark and accompanying subCode. If the stream closes
or times out, it returns false. UserGetYesNo calls
UserProtocolError if it encounters anything except Yes or No
followed by EOC.
UserProtocolError()
Writes an error message to dspStream and then calls UserClose to
abort the connection.
3.2. User File Operations
File FtpUserProtFile.br implements the User protocol for standard file
operations. It uses the bspStream, bspSoc, and dspStream fields in its
FtpCtx and contains the following external procedures:
UserStore(localL, StoreFile) = subcode,,mark
Attempts to send the file described by 'localL' to the remote
Server, calling the user-supplied procedure 'StoreFile' to transfer
the data. It returns zero if something catastrophic happens (such
as the Server aborts the connection), in which case retrying is
probably futile.
UserStore sends PL to the Server for approval. The Server can
refuse the command at this point, in which case UserStore returns
subcode,,markNo. If the Server accepts the command, it returns a
PL (remotePL) specifying the destination file, and UserStore calls
StoreFile(localL, remotePL)
Pup FTP Package December 12, 1981 12
which should transfer the file data. This package provides
procedures for transferring data from a disk stream to a network
stream, but you are free to write your own. StoreFile should
return true if the transfer went successfully. If some
environment-specific thing goes wrong (such as an unrecoverable
disk error), StoreFile should call FTPM(markNo, code, string, true)
and return false. UserStore then asks the Server if the transfer
went successfully and returns subcode,,mark. If mark is 'markYes',
the file arrived at the Server safely.
UserRetrieve(localPL, Retrieve) = subcode,,mark
Attempts to retrieve the file described by localPL from the remote
Server, calling the user-supplied procedure 'RetrieveFile' to
transfer the data. UserRetrieve returns zero if some catastrophic
error occurs, markNo if the Server refuses the command, and
markEndOfCommand if the everything goes OK.
UserRetrieve sends localPL to the Server and waits for approval.
The Server can refuse the command at this point, in which case
UserRetieve returns subcode,,markNo. If the Server can handle
property lists that specify multiple files, then the following
steps are taken for each file:
If the Server has no more files matching localPL, UserRetrieve
returns subcode,,markEndOfCommand (subcode is undefined in
this case). Otherwise the Server sends a fully-specified
property list describing a file which it is willing to send.
UserRetrieve parses this into remotePL and calls
Retrieve(remotePL, localPL)
which should decide whether to accept the file. To skip the
file, Retrieve should return false. UserRetrieve so informs
the Server and then loops. To accept the file, Retrieve
should return a procedure which UserRetrieve can call to
transfer the data. Don't open the file yet, because the
Server can still back out, in which case UserRetrieve skips
the next step and just loops. If Retrieve returns true,
UserRetrieve tells the Server to send the file and then calls
RetrieveFile(remotePL, localPL)
which should open the file, transfer the data, and close the
file. This package contains procedures for transferring data
from a network stream to a disk stream, but you are free to
write your own. When RetrieveFile is done, it should return
true if everything went OK, or false after calling
FTPM(markNo, code, string) if something went wrong.
UserRetrieve then loops.
UserDelete(localPL, Delete) = subcode,,mark
Requests the remote Server to delete the files described by
localPL, calling the user-supplied procedure DeleteFile before
allowing the server to actually delete anything. UserDelete
returns zero if some catastrophic error occurs, markNo if the
Server refuses the command, and markEndOfCommand if the everything
goes OK.
UserDelete sends localPL to the Server and waits for approval. The
Pup FTP Package December 12, 1981 13
Server can refuse the command at this point, in which case
UserDelete returns subcode,,markNo. If the Server can handle
property lists that specify multiple files, then the following
steps are taken for each file:
If the Server has no more files matching the original pList,
UserDelete returns subcode,,markEndOfCommand. Otherwise the
Server sends a fully-specified property list describing a file
which it is willing to delete. UserDelete parses this into
remotePL and calls
Delete(remotePL, localPL)
which should return true to confirm deleting the file
described by remotePL. UserDelete passes this answer on to
the Server and then loops.
UserDirectory(localPL, Directory) = subcode,,mark
Requests the remote Server to describe in as much detail as it can
files matching localPL, and then calls the user-supplied procedure
Directory when the answers come back.
UserDirectory sends localPL to the Server and waits for an answer.
The Server can refuse the command at this point, in which case
UserDirectory returns subcode,,markNo. If the Server can handle
property lists that specify multiple files, then the following
steps are taken for each file:
If the Server has no more files matching localPL,
UserDirectory returns subcode,,markEndOfCommand. Otherwise
the Server sends a property list which UserDirectory parses
into remotePL and calls
Directory(remotePL, localPL)
and then loops.
3.3. User Mail Operations
File FtpUserProtMail.br implements the user part of the Mail Transfer
Protocol. This description ignores various critical sections and other
vital considerations which must be handled by the user-supplied
routines in order to provide a reliable mail service. For the
semantics of the protocol see <Pup>MailTransfer.ears.
UserStoreMail(pListGen, ExcpHandler, Xfer)
Attempts to send mail to the mailboxes described by the pLists
generated by pListGen. It returns zero if something catastrophic
happens (such as the Server aborts the connection), in which case
retrying is probably futile.
UserStoreMail repeatedly calls the client-supplied procedure
pListGen which should supply a pList describing a recipient of the
message. When the last recipient has been generated, pListGen
should return zero. The Server can refuse the command at this
point, in which case UserStoreMail returns subcode,,markNo. If the
Server accepts the command, it may still object to some of the
mailboxes, in which case UserStoreMail calls the client-supplied
procedure
Pup FTP Package December 12, 1981 14
ExcpHandler(subcode, index)
which should record the recipient as rejected. Recipients are
numbered from one, in the order in which they were generated by
pListGen. Index is the number of the rejected recipient. A string
describing why the recipient was rejected is in
FtpCtx.getCmdString.
If after rejecting any recipients, there are still some valid ones,
UserStoreMail calls the client-supplied procedure
Xfer()
which should transfer the message text. Xfer should return true if
the transfer went successfully. If some environment-specific thing
goes wrong (such as an unrecoverable disk error), Xfer should call
FTPM(markNo, code, string, true) before returning false.
UserStoreMail then asks the Server if the transfer went
successfully. The server can reject some more recipients at this
point, in which case UserStoreMail calls the client-supplied
procedure ExcpHandler again. Finally UserStoreMail returns
subcode,,mark. If mark is 'markYes', the mail arrived at the
Server safely.
UserRetrieveMail(pList, RetrieveMail) = subCode,,mark
Attempts to retrieve the contents of the mailbox described by
'pList' from the remote Server, calling the user-supplied procedure
'RetrieveMail' to transfer the data. UserRetrieveMail returns zero
if some catastrophic error occurs, markNo if the Server refuses the
command, and markEndOfCommand if the everything goes OK.
UserRetrieveMail sends pList to the Server and waits for approval.
The Server can refuse the command at this point, in which case
UserRetieveMail returns subcode,,markNo. Otherwise
UserRetrieveMail calls
RetrieveMail(pList)
which should transfer the file data. When RetrieveMail is done, it
should return true if everything went OK.
4. Utility Routines
The utility module (files FtpUtilB.br, FtpUtilA.br, FtpUtilXfer,
FtpUtilDmpLd, and FtpUtilInit.br) contains protocol routines shared by
the User and Server modules, and some routines for efficiently
manipulating disk streams.
InitFtpUtil()
builds some internal tables and streams, getting space from
sysZone. You must call this procedure before starting a Server or
issuing any User commands.
FTPM(mark, subCode [0], string [], eoc [false], par0, par1, par2, par3,
par4)
sends the FTP command 'mark' to the remote FTP process, including
Pup FTP Package December 12, 1981 15
'subCode' if the command requires one, and 'string' if one is
present. Then, if 'eoc' is true, an EOC command is sent. 'String'
is written to bspStream using the Template package, and may contain
imbedded format information. 'Par0' through 'par4' are passed as
arguments to the PutTemplate call. The subcode and string
arguments further explain certain commands. For markNo, subCode is
a machine-readable explanation of why a request was refused, and
'String' is human-readable text such as "UserName and Password
required". Codes are tabulated in an appendix to
<Pup>FtpSpec.ears. New codes may be registered on request.
GetCommand(timeout [-1]) = subCode,,mark
flushes bspStream up to the next command and returns the mark and
subcode (if any). Returns false if the stream closes or it hangs
for 'timeout' miliseconds while waiting for a byte (the default,
-1, waits forever). Comment commands are ignored. GetCommand
writes the strings accompanying Version, No, and Comment commands
to dspStream and stores a pointer to them in FtpCtx.getCmdString.
The utility module makes three 'process-relative streams' for use by
the rest of the package. The only operation defined is 'Puts'.
lst writes to dspStream
dls writes to dspStream if debugFlag is true
dbls writes to bspStream and if debugFlag to dspStream
For example, Wss(dls,string) writes 'string' to the running process'
dspStream if the process' debugFlag is set.
4.1. Unformatted Data Transfer
File FtpUtilXfer.br contains procedures for performing efficient
operations on Alto OS disk Streams. They use the following fields in
FtpCtx: bspSoc, bspStream, dspStream, diskStream, buffer, and
bufferLength.
DiskToNet(remotePL, localPL) = true|false
Transfers bytes from diskStream to bspStream up to end-of-file, and
returns true if everything went OK.
NetToDisk(remotePL, localPL) = true|false
Transfers bytes from bspStream to diskStream until it encounters
another FTP command returning true if everything went OK.
FileType() = Text|Binary
Resets diskStream, scans it looking for high order bits on, and
then Resets it again. As soon as it encounters a byte with the
high order bit on, it returns 'Binary', otherwise (having read the
entire file) it returns 'Text'.
PrintBegin(localPL)
Outputs the server filename in remotePL and the type and byte size
from localPL to dspStream.
Pup FTP Package December 12, 1981 16
4.2. Dump Format Data Transfer
File FtpUtilDmpLd.br contains procedures for transferring data between
a disk and an FTP connection in dump format. They may be used as the
inner loops of user-supplied data transfer procedures passed to
UserStore and UserRetrieve and will create and unbundle dump-format
files on the fly. If you don't want to handle dump format, you don't
need this file. Dump-file format is described in an appendix to the
Alto Executive documentation.
These procedures use the same fields in FtpCtx and the same Alto OS
routines as the unformatted transfer routines. Buffer must be at least
130 words long. Making it longer does not speed up the transfer.
DumpToNet(remotePL, localPL) = true|false
Moves a file from diskStream to bspStream converting it to dump
format, returning true if things go OK. The filename is taken from
the name-body field of localPL, and the creation date from the
creation date field. To terminate a dump file, call DumpToNet with
remotePL = 0.
LoadFromNet(remotePL, localPL) = true|false
Moves a file from bspStream to diskStream converting it from dump
format, returning false when it encounters an 'end block'. When it
encounters a file, it returns true with the filename and creation
date in remotePL. If the client wants the file, he should call
LoadFromNet again with FtpCtx.diskStream non zero; to skip a file
set diskStream to zero.
4.3. Binary Compare Data Transfer
Files FtpUtilCompB.br and FtpUtilCompA.br implement a binary compare of
a network stream and a disk stream. If you don't want to do this (not
many people will, I suspect), then you don't need these files.
FtpUtilCompA contains two Block comparison procedures: one uses a fast
machine code loop and the other uses special microcode which you must
load into the Alto's ram.
CompareNetWithDisk(remotePL, localPL) = true|false
Compares diskStream with bspStream byte-by-byte and reports the
results to dspStream. If the two streams are identical (and the
same length), then a string of the form "xxx identical bytes" is
output, otherwise a string of the form "First difference at byte
pos xxx" is output. Returns true if everything went OK, false if
something catastrophic happened to the network connection (note in
particular that a result of true implies nothing about whether the
two streams were identical).
The following fields in the FtpCtx must be set up: bspStream,
dspStream, diskStream, buffer, and bufferLength.
5. Property Lists
The property list module (files FtpPListProt.br, FtpPList1.br, and
FtpPListInit.br) translates between this package's internal
Pup FTP Package December 12, 1981 17
representation of a property list and the protocol-specified network
representation.
The FTP protocol specifies the syntax of a property list and the syntax
of a set of properties sufficient for standard file operations, but
states that property lists are extensible. Therefore the property list
module comes in two parts: a part that knows the syntax of property
lists, and a part which knows the syntax of individual properties. To
add new properties you need only modify the latter.
The principal data structure in this module is the Property List
Keyword Table, or pListKT. This table, built by InitFtpPlist, contains
(propertyName, propertyObject) pairs. PropertyNames are strings such
as "Byte-size". PropertyObjects know how to Scan (parse) properties
into pLists, Generate properties from pLists, Initialize properties
from a pList full of default values, and Free properties stored in
pLists.
5.1. Property List Protocol
File FtpPlistProt.br implements four operations on property lists.
This is the module that knows the syntax of a property list, but not
the syntax of individual properties. Procedures in this file use the
bspStream, bspSoc, and dspStream fields of the FtpCtx and contain the
following external procedures:
InitPList(defaultPL []) = PL
Creates an empty pList, and initializes it to be a copy of
'defaultPL' if one was supplied.
FreePList(PL)
Destroys PL and returns 0 to facilite writing PL = FreePList(PL).
If PL is zero, FreePList returns zero without doing anything.
ScanPList() = PL|false
Expects to find a property list in bspStream. ScanPList parses
this property list and returns a PL if it had proper syntax. If
the property list is malformed, ScanPList calls FTPM(markNo, code,
string) and returns false. If ScanPList encounters a mark before
starting a PL or the connection closes or Gets times out, it
returns false.
GenPList(PL)
Generates a property list in network format from PL and sends it to
bspStream.
5.2. The 'Standard' Properties
Files FtpPlist1.br and FtpPlistInit.br implement the standard
properties. These files know the syntax of individual properties; they
contain the operation procedures for the standard property objects.
These files are used by the FTP subsystem and IFS and are sufficient
for performing 'standard' file operations. If you wish to add
properties, these are the modules which you must change. In addition
to the property operations which are rather specialized to their task,
there are a few generally useful procedures which are made external:
InitFtpPList()
Pup FTP Package December 12, 1981 18
which makes the standard property objects and builds fplKT, getting
space from sysZone. This procedure must be called before calling
any of the procedures in FtpPlist.br (which typically means before
starting a server or calling any procedures in the User module).
Nin(string, lvDest) = true|false
Interprets 'string' as a decimal number and leaves the result in
'lvDest', ignoring leading blanks and terminating on end of string.
A null string results in lvDest getting 0. Returns false if the
string contains any characters other than 0-9 and <space>.
ParseDate(string, lvRes) = true|false
Parses the string format date into an Alto format date which it
puts into the two word vector at 'lvRes'. Returns true if it could
parse the date. ParseDate expects the format of the string to bear
some similarity to "day-month-year hour:minute:second".
WriteDT(stream, dt)
converts 'dt' from 32 bit Alto date format to a string of the form
"dd-mmm-yy hh:mm:ss" and writes it to 'stream'.
6. Example
The following example program makes use of most of the facilities in
the User part of the Ftp Package. I have run it and it works. It is a
rock-bottom minimal User Ftp with no redeeming features whatsoever.
More extensive and realistic examples can be found by looking at the
sources for the Ftp subsystem.
The main procedure FtpUserExample performs initialization, which
consists of augmenting SysZone, initializing the Ftp and Pup packages,
and creating and starting a context running the procedure 'User'.
User opens a BSP connection to Maxc, sets up its FtpCtx, gets and fills
a blank pList, and calls UserRetrieve. When UserRetrieve returns, User
closes the connection, releases its resources and commits suicide.
//FtpUserExample.bcpl - Example Ftp User
//last modified April 9, 1978 4:24 PM
// The load command file is:
// Bldr/l/v 600/W FtpUserExample ↑
// ↑
// FtpUserProt FtpUserProtFile ↑
// FtpPListProt FtpPList1 ↑
// FtpUtilb FtpUtila FtpUtilXfer ↑
// ↑
// PupBspOpenClose PupBspStreams PupBspProt PupBspBlock PupBspA ↑
// PupRtpOpenClose PupRtp PupNameLookup ↑
// Pup1OpenClose Pup1B PupAl1A PupRoute PupDummyGate ↑
// PupAlEthB PupAlEthA ↑
// ↑
// Context ContextInit Interrupt ↑
// AltoQueue AltoTimer AltoByteBlt ↑
// Template CTime StringUtil Keyword ↑
// ↑
// FtpPlistInit FtpUtilInit KeywordInit ↑
Pup FTP Package December 12, 1981 19
// Pup1Init PupAlEthInit InterruptInit
get "FtpProt.decl"
get "Pup.decl"
external
[
//incoming procedures
InitFtpUtil; InitFtpPList; InitPupLevel1
GetFixed; CallSwat; AddToZone; Allocate; Free
InitializeContext; CallContextList; Enqueue
GetPartner; OpenLevel1Socket; OpenRTPSocket; CreateBSPStream
InitPList; FreePList; NetToDisk
UserRetrieve; UserOpen; UserClose; NetToDisk
ExtractSubstring; OpenFile; Closes; Wss
//incoming statics
sysZone; dsp; CtxRunning; UserName; UserPassword
]
let FtpUserExample() be
[
let v = GetFixed(10000)
if v eq 0 then CallSwat("GetFixed failed")
AddToZone(sysZone, v, 10000)
let ctxQ = vec 1; ctxQ!0 = 0
InitFtpUtil()
InitFtpPList()
InitPupLevel1(sysZone, ctxQ, 10)
Enqueue(ctxQ, InitializeContext(Allocate(sysZone, 1000), 1000,
User, lenExtraCtx))
CallContextList(ctxQ!0) repeat
]
and User(ctx) be //a context
[
let soc = Allocate(sysZone, lenBSPSoc)
let maxcPort = vec lenPort
unless GetPartner("Maxc", dsp, maxcPort, 0, socketFTP) do
CallSwat("GetPartner failed")
OpenLevel1Socket(soc, 0, maxcPort)
unless OpenRTPSocket(soc) do
CallSwat("OpenRTPSocket failed")
CtxRunning>>FtpCtx.bspStream = CreateBSPStream(soc)
CtxRunning>>FtpCtx.bspSoc = soc
CtxRunning>>FtpCtx.dspStream = dsp
CtxRunning>>FtpCtx.buffer = Allocate(sysZone, 256)
CtxRunning>>FtpCtx.bufferLength = 256
CtxRunning>>FtpCtx.debugFlag = true
unless UserOpen(Version) do
CallSwat("UserOpen failed")
let localPL = InitPList()
localPL>>PL.UNAM = ExtractSubstring(UserName)
localPL>>PL.UPSW = ExtractSubstring(UserPassword)
localPL>>PL.SFIL = ExtractSubstring("<system>Pup-Network.txt")
let mark = UserRetrieve(localPL, Retrieve)
if mark ne markEndOfCommand then
Pup FTP Package December 12, 1981 20
CallSwat("UserRetrieve failed")
FreePList(localPL)
UserClose()
Free(sysZone, soc)
Free(sysZone, CtxRunning>>FtpCtx.buffer)
finish
]
and Version(stream, nil) be Wss(stream, "Example FTP User")
and Retrieve(remotePL, localPL) = RetrieveFile
and RetrieveFile(remotePL, localPL) = valof
[
let s = OpenFile(remotePL>>PL.NAMB, ksTypeWriteOnly, charItem)
CtxRunning>>FtpCtx.diskStream = s
unless NetToDisk(remotePL, localPL) do CallSwat("NetToDisk failed")
Closes(s)
resultis true
]
7. Revision History
March 30, 1977
First release.
May 15, 1977
Added Directory and Rename commands. Server now handles property lists
which specify multiple files. Added User and Server mail operations.
June 8, 1977
Overlay machinery was changed and some bugs were fixed. Some structure
definitions changed, so recompilation of user programs is necessary.
July 17, 1977
DiskToNet and NetToDisk moved out of FtpUtilb into a new file
FtpUtilXfer. Property lists reorganized, causing changes to the
calling interface in FTPSI. Plist module now uses the Keyword routines
in the CmdScan package. Recompilation of user programs is necessary.
FtpUserDmpLd renamed FtpUtilDmpLd. Timeouts cleaned up.
October 24, 1977
Example program added.
February 14, 1978
Files FtpUtilCompB and FtpUtilCompA, implementing a byte-by-byte
compare of the net stream with a disk stream added.
April 9, 1978
Pup FTP Package December 12, 1981 21
Implemented the new form of Store in which the Server returns a
property list specifying the destination file. The old form is still
supported, but no longer documented.
June 1, 1978
FtpServProt.bcpl split out of FtpServProtFile.bcpl.
FtpServProtMail.bcpl updated to the current MTP. Many data structures
changed so recompiliation of user programs is necessary.
September 20, 1980
Parameters passed to client routines changed. Both property lists are
passed now. Recompilation is necessary.
December 16, 1980
Timeouts reworked. Statics 'getCmdTimeout' and 'getPutTimeout', and
their default values, manifests 'defGetCmdTimeout' and
'defGetPutTimeout' were removed, since Pup byte stream activity
timeouts, added about a year ago, do the same job. FtpServProt now
takes a timeout which it uses while waiting for top level commands.
December 12, 1981
Internal data structure rearranged. Recompilation is necessary.