.alone←declaration(expurgate)=0;
.if alone then start
.require "<altosource>ttydefs.pub" source!file
.end
.every heading (|Pup FTP Package|,|December 12, 1981|,{page})
.once center
Pup File Transfer Protocol Package
.skip 3
This package is a collection of modules implementing the Pup File and Mail Transfer Protocols. The package is used by the FTP subsystem and the Interim File System.
.sec |Overview|
This document is organized as a general overview followed by descriptions of each of the modules in the package. A history of revisions to the package is included at the end.
Before beginning the main documentation, some general comments are in order.
.begin indent 5,5
a. The File Transfer Protocol is (alas) complex; this package requires the Pup package and all of its supporting packages plus some other packages not specific to Pup. This documentation is less tutorial than normal Alto package descriptions so you should be prepared to consult its author.
b. This document describes the external program interfaces for a particular implementation of the File Transfer Protocol, and does not deal with the internal implementation nor the reasons for design choices in the protocol or the implementation. Before considering the details of this package, you should read [Maxc]<Pup>FtpSpec.press to get the flavor of how the File Transfer Protocol works. The <Pup> directory also contains descriptions of the lower level protocols on which FTP is based. Detailed knowledge of these protocols is not necessary to use this package, but you must be familiar with the operation of the Pup package.
c. This package and the protocol are under active development so users should expect modifications and extensions.
d. This package is designed to run under several operating systems and with several file systems. Functions are carefuly split into protocol-specific and environment-specific modules. This package provides the protocol modules; you must write the matching environment-specific modules.
.end
.ssec |Organization|
The FTP package comes in four modules: Server, User, Utilities, and Property lists. The utility and property list modules are shared by the User and Server.
The User and Server modules implement their respective halves of the protocol exchanges.
The Property List module generates and parses property lists, filesystem-independent descriptions of files. When passed between User and Server FTPs through the network byte stream, their form is defined by protocol as a parenthesized list. When passed between these protocol modules and the user-supplied modules in a program, they take the form of a data structure defined by this package.
The Utility module contains protocol routines shared by the User and Server modules and some efficient routines for transferring data between a network stream and a disk stream.
.ssec |File Conventions|
The FTP package is distributed as file FTPPackage.dm, and contains the following files:
.begin nofill group tabs 5,30
User
\FtpUserProt.br\User protocol common to file and mail
\FtpUserProtFile.br\User file commands
\FtpUserProtMail.br\User mail commands
Server
\FtpServProt.br\Server protocol common to file and mail
\FtpServProtFile.br\Server file commands
\FtpServProtMail.br\Server mail commands
Property lists
\FtpPListProt.br\Property list protocol
\FtpPListImpl.br\Implements a 'standard' property list
\FtpPListInit.br\Initialization
Utility
\FtpUtilB.br\Common protocol
\FtpUtilXfer.br\Unformatted data transfer
\FtpUtilDmpLd.br\Dump/Load data transfer
\FtpUtilCompB.br\Binary compare data transfer
\FtpUtilCompA.br\Binary compare data transfer
\FtpUtilA.br\Assembly-language utility code
\FtpUtilInit.br\Initialization
Definitions
\FtpProt.decl\Protocol parameters and structures
Command files
\CompileFtpPackage.cm\Compiles all files
\DumpFtpPackage.cm\A list of all binary files
\FtpPackage.cm\A list of all source files
.end
All of these modules are swappable, and are broken up into pieces no larger than 1024 words. Modules whose names end in "init" are initialization code which should be executed once and thrown away.
The source files are kept with the subsystem sources in FTP.dm and are formatted for printing in a small fixed-pitch font such as Gacha8 (use the command 'Empress @FtpPackage.cm@').
.ssec |Other Packages|
FTP is a level 3 Pup protocol and this package uses a number of other Alto software packages. As always, files whose names end in "init" may be discarded after initialization (except ContextInit.br).
.begin group nofill tabs 5,25,45,60
Pup Package
\PupBSPOpenClose\PupBSPStreams.br\PupBSPProt.br
\\PupBSPa.br\PupBSPBlock.br
\PupRTP.br\PupRTPOpenClose\PupDummyGate.br\PupRoute.br
\Pup1b.br\Pup1OpenClose\PupAl1a.br\Pup1Init.br
\PupAlEthb.br\PupAlEtha.br\PupAlEthInit.br
Context Package
\Context.br\ContextInit.br
Interrupt Package
\Interrupt.br\InterruptInit.br
Queue, Timer, and ByteBLT Packages
\AltoQueue.br\AltoTimer.br\AltoByteBLT.br
Time Package
\TimeConvB.br\TimeConvA.br\TimeIO.br
CmdScan Package
\Keyword.br\KeywordInit.br
Strings and Template Packages
\StringUtil.br\Template.br
.end
.ssec |Principal Data Structures|
The following data structures are of interest to users, and together with the procedures described later, constitute the package interface.
.begin indent 0,9 tabs 10
PL\Property List, is this implementation's internal representation of the protocol-specified property list.
FTPI\File Transfer Package Interface, contains pointers to the network byte stream, user disk stream, log stream, the file buffer, and various flags.
FTPSI\FTP Server Interface, is a vector of user-supplied procedures constituting the interface between the protocol and environment-specific Server modules.
FtpCtx\FTP Context, is the process-private storage for an instance of a User or Server FTP. It consists of an FTPI, and if the process is a Server, an FTPSI. This is a convenient place for the user-supplied modules to keep process-private data. You can do this by adding items to the FtpCtx definition and then recompiling everything.
.end
The entire FtpCtx need not be filled in all of the time. For each group of procedures, the items they require will be specified. A general description of the contents of the FTPI part of an FtpCtx is in order here.
.begin indent 0,19 tabs 20
bspSoc\a pointer to a BSP socket open to a remote FTP process.
bspStream\a pointer to the stream in the above BSP socket. Pup package experts will recognize that this is redundant, but it is often convenient and makes the code clearer.
dspStream\a pointer to a stream to which this package will output generally useful information, including copious amounts of debugging information if debugFlag is true. The only operation that need be defined is 'Puts'.
debugFlag\a boolean. If true, the protocol exchanges for this context are output to dspStream as text, along with some other useful information. Use this! It will save you much head-scratching.
connFlag\a boolean. This should be true if bspSoc is open. The package will cooperate in maintaining this flag, which is valuble when finishing.
serverFlag\a boolean. This flag is tested by procedures in the shared modules to determine whether the caller is a User or Server.
getCmdString\a pointer to the last string read by the GetCommand procedure in the Utility module. Commands with string arguments are No, Yes, Version, MailBoxException, and Comment.
.end
The following items are used by the data transfer routines in the Utility module. The routines are Alto-specific and in some cases Ftp subsystem-specific, so these items need not be filled in if you do not use the routines.
.begin indent 0,19 tabs 20
diskStream\a pointer to a disk stream. It should always be opened in byte mode.
buffer\a pointer to a block of memory which can be used for block transfer I/O operations. The bigger this is the faster things will go.
bufferLength\the length in words of the above buffer.
.end
.ssec|Programming Conventions|
This package can be used with the Bcpl Overlay package. File FtpOEPInit.br contains a procedure which will help do this, but you should consult with the author.
This package does a lot of string manipulation, and uses the following conventions:
.begin indent 5,5
a. All strings are allocated from 'sysZone'.
b. Strings are represented in data structures (such as property lists) as addresses. Zero means no string is present.
.end
All of the procedures in this package expect to execute in contexts (in the sense of the Context package), and expect CtxRunning (defined by the Context package) to point to an appropriately filled in FtpCtx.
.ssec |Property Lists|
In most contexts, there are two property lists: one generated by the client of the package, and one generated by the package. A client-generated property list is referred to as a 'localPL', and it is the client's responsibility to free it when it is no longer needed. Property lists created by this package are referred to as 'remotePLs' since they are copies of property lists generated remotely; they should never be freed by the client.
.sec |Server|
The FTP Server module consists of three files: FtpServProt.br, routines common to the file and mail servers, FtpServProtFile.br, file commands, and FtpServProtMail.br, mail commands. The server module has one public procedure:
.begin indent 0,5
FtpServProt(timeout)
.continue
which carrys out protocol commands received over bspStream by calling the user-supplied procedures in FTPSI. When the BSP connection is closed by the remote User process, this procedure returns. FtpServProt passes 'timeout' to GetCommand (in the utility module) when waiting for top-level commands (retrieve, store, delete, etc.). This permits the server to break connections that don't seem to be doing anything.
.end
This module uses the following fields in FtpCtx: dspStream, bspStream, bspSoc, and FTPSI. The manifest constant MTP in FtpProt.decl conditionally compiles calls on the MTP commands. The package is released with this switch false, since I expect only IFS will need it. All of the FTP commands (Version, Store, Retrieve, etc.) must contain procedures (except the MTP ones if the MTP switch is false). If you do not wish to implement a command, it is sufficient to point the command at:
.begin nofill indent 5
and NYI(nil, nil) = valof
[
FTPM(markNo, 1, "Unimplemented Command")
resultis false
]
.end
in which case any subsidiary procedures for that command (such as StoreFile and StoreCleanup for the Store command) need not be filled in. FTPM is described in more detail below.
.ssec |Version Command|
By convention, Version is the first command exchanged over a newly opened FTP connection. The User sends its protocol version number and a string such as "Maxc Pup Ftp User 1.04 19-Mar-77". When FtpServProt receives this command, it replys with its protocol version number and then calls
.once indent 5
(CtxRunning>>FtpCtx.Version)(bspStream, nil)
which should generate some herald text:
.once indent 5
Wss(CtxRunning>>FtpCtx.bspStream, "Alto Pup FTP Server 1.13 14-May-77")
.ssec |Retrieve Command|
When the remote FTP User process sends the command 'Retrieve' and a property list describing the files it wants to retrieve, FtpServProt parses the property list and calls
.once indent 5
(CtxRunning>>FtpCtx.Retrieve)(remotePL, localPL)
which should decide whether to accept the command. Retrieve's decision may involve checking passwords, looking up files, and other actions using the information in remotePL plus other environment-specific information, such as whether the requester has the correct capabilities, etc. To refuse the request, Retrieve should call
.once indent 5
FTPM(markNo, code, string)
and return false. To accept the command, it should return a new property list, localPL, describing a file matching remotePL which Retrieve is willing to send. FtpServProt returns this PL to you as 'localPL' in the next call to Retrieve, so that you can free it. On the first call, localPL will be zero. Some FTP implementations require a minimum set of properties here, but the whole subject of who should specify what properties is rather involved and beyond the scope of this description. For more information, consult the FTP specification. This package provides a fast procedure (in the Utility module) for deciding the 'type' of a file (text or binary) which you may find useful.
Property lists in retrieve requests may specify multiple files, so FtpServProt will continue to call Retrieve until it returns false (no more files). On each call, remotePL will be the same original PL sent from the remote User, and localPL will be the last PL returned by Retrieve. If Retrieve supports multiple file requests then it must save some information so that the next time FtpServProt calls it, it can generate the next file. If Retrieve does not support multiple file requests then it should do its thing during the first call and remember that it is finished. The next time it is called it should return false having only deallocated localPL (it should not call FTPM).
If Retrieve returns a PL, FtpServProt sends it back to the User to more fully describe the file. At this point the User may back out of the transfer, in which case the next procedure will be skipped, and RetrieveCleanup will be called immediately. If the User indicates a willingness to proceed, FtpServProt then calls
.once indent 5
(CtxRunning>>FtpCtx.RetrieveFile)(localPL, remotePL)
to transfer the file data. This package provides a procedure (in the Utility module) for transferring data from a disk Stream to a BSP Stream, but you are free write your own. When RetrieveFile has finished the transfer, it should return true if everything went OK. If something bad happened, it should call
.once indent 5
FTPM(markNo, code, string)
and return false. In any case, FtpServProt calls
.once indent 5
(CtxRunning>>FtpCtx.RetrieveCleanup)(localPL, ok, remotePL)
where 'ok' is false if RetrieveFile returned false or the User backed out of the command. Note that if Retrieve returned true, RetrieveCleanup will always be called, but RetrieveFile may not. If Retrieve allocates any resources (such as opening a file) they should be deallocated here.
Finally, FtpServProt calls Retrieve again, and the process repeats until Retrieve returns false.
.ssec |Store Command|
When the remote FTP User process sends the command 'newStore' followed by a property list describing the file, FtpServProt parses the property list and calls
.once indent 5
(CtxRunning>>FtpCtx.Store)(remotePL)
which should decide whether to accept the command. To accept, Store should return a property list (referred to as localPL below) specifying the destination file (localPL will be passed to StoreCleanup so you can free it). To refuse the command Store should call FTPM(markNo, code, string) and return false, in which case the next procedure (StoreFile) is not called.
If Store returns true, FtpServProt sends the PL to the User and then calls
.once indent 5
(CtxRunning>>FtpCtx.StoreFile)(remotePL, localPL)
to transfer the file data. This package provides a procedure (in the Utility module) for transferring data from a BSP Stream to a disk Stream, but you may write your own. When StoreFile has finished the transfer, it should return true if everything went OK. If something bad happened, it should call
.once indent 5
FTPM(markNo, code, string)
and return false. Finally, FtpServProt calls
.once indent 5
(CtxRunning>>FtpCtx.StoreCleanup)(remotePL, ok, localPL)
where 'ok' is true if StoreFile returned true and the User indicated that everything went ok. If 'ok' is false, StoreCleanup should delete the file, since it is almost certainly damaged. Note that if Store returned true, StoreCleanup will always be called, but StoreFile may not. If Store allocates any resources (such as opening a file) they should be deallocated here.
.ssec |Delete Command|
When the remote FTP User process sends the command 'Delete' followed by a property list describing the files which it wants to delete, FtpServProt parses the property list and calls
.once indent 5
(CtxRunning>>FtpCtx.Delete)(remotePL, localPL)
which should decide whether to accept the command. Don't delete anything yet! The User may still back out. To refuse the delete request, Delete should call FTPM(markNo, code, string) and return false. To accept the command, it should return a new PL with every property it can find, so that the User can be sure of the identity of file to be deleted. FtpServProt will return this PL as 'localPL' in the next call to Delete, so that it can be deallocted.
Property lists in delete requests may specify multiple files, so FtpServProt will continue to call Delete until it returns false. On each call, remotePL will be the same original PL sent from the remote User, and localPL will be the last PL returned by Delete. If Delete supports multiple file requests then it must save some information so that the next time FtpServProt calls it, it can generate the PL for the next file. If Delete does not support multiple file requests then it should do its thing during the first call and remember that it is finished. The next time it is called it should return false having only deallocated localPL (it should not call FTPM).
If Delete returns a PL, FtpServProt sends it back to the User and waits for confirmation. If the User still wants to delete the file, FtpServProt calls
.once indent 5
(CtxRunning>>FtpCtx.DeleteFile)(localPL, remotePL)
which should delete the file and return true. If something goes wrong, it should call
.once indent 5
FTPM(markNo, code, string)
and return false. Finally, FtpServProtFile calls Delete again, and the process repeats until Delete returns false.
.ssec |Directory Command|
When the remote FTP User process sends the command 'Directory' followed by a property list naming the files about which it wants information, FtpServProt parses the property lists and calls
.once indent 5
(CtxRunning>>FtpCtx.Directory)(remotePL, localPL)
which should decide whether to accept the command. To refuse the request (because for example the requestor does not have the correct access capabilities) Directory should call FTPM(markNo, code, string) and return false. To accept the command it should return a PL describing a file.
Property lists in directory requests may specify multiple files, so FtpServProt will continue to call Directory until it returns false. If Directory supports multiple file requests then it must save some information so that the next time FtpServProt calls it, it can generate the PL for the next file. If Directory does not support multiple file requests then it should do its thing during the first call and remember that it is finished. The next time it is called it should return false having only deallocated localPL (it should not call FTPM).
.ssec |Rename Command|
When the remote FTP User process sends the command 'Rename' followed by two property lists describing the old and new files, FtpServProt parses the property lists and calls
.once indent 5
(CtxRunning>>FtpCtx.Rename)(oldPL, newPL)
which should decide whether to accept the command. The FTP protocol does not require that user access information be present in newPL, so access checking should be done on oldPl only. To refuse the rename request, Rename should call FTPM(markNo, code, string) and return false. Otherwise it should rename the file returning true if successful. If the rename operation fails, Rename should call
.once indent
FTPM(markNo, code, string)
and return false.
.ssec |Mail Protocol|
File FtpServProtMail.br implements the server part of the Mail Transfer Protocol. This description ignores various critical sections and other vital considerations which must be handled by the user-supplied routines in order to provide a reliable mail service. For the semantics of the protocol see [Maxc]<Pup>MailTransfer.press.
.ssec |StoreMail Command|
When the remote FTP User process sends the command 'StoreMail', FtpServProt parses the property lists which follow and for each one calls
.once indent 5
(CtxRunning>>FtpCtx.StoreMail)(remotePL)
which should return true or false. Returning true has nothing to do with whether the mailbox is valid, it just indicates that the command exchange may continue. If the mailbox is invalid, StoreMail should call FTPM(markMailboxException, code, string) and return true. Returning false terminates the exchange: StoreMailFile is skipped and StoreMailCleanup is called. StoreMail is called with a zero PL the last time so that it may reply No and return false if none of the mailboxes are valid.
If StoreMail always returns true, FtpServProt tells the User process to go ahead and send the mail, and then calls
.once indent 5
(CtxRunning>>FtpCtx.StoreMailMessage)()
to transfer the file data. When StoreMaiMessage has finished the transfer, it should return true if everything went OK. If something went wrong, it should call
.once indent
FTPM(markNo, code, string)
and return false. Finally, FtpServProt calls
.once indent 5
(CtxRunning>>FtpCtx.StoreMailCleanup)(ok)
where 'ok' is true if StoreMailMessage returned true and the remote User indicated that everything went ok. If 'ok' is false, StoreMailCleanup should not deliver the mail. Note that if StoreMail is ever called, StoreMailCleanup is always called, but StoreMailMessage may not be. If StoreMail allocates any resources (such as opening a file) they should be deallocated here.
.ssec |RetrieveMail Command|
When the remote FTP User process sends the command RetrieveMail followed by a property list describing the mailbox, FtpServProt parses the property list into 'remotePList' and then enters a loop:
.begin indent 9,9
First FtpServProt calls
.once indent 14
(CtxRunning>>FtpCtx.RetrieveMail)(remotePL, localPL)
which should return a PL describing the next message in the mailbox. If there are no more unread messages in the mailbox, RetrieveMail should return zero. On each call, remotePL is the same original PL sent from the remote User, and localPL is the last PL returned by RetrieveMail, which should be freed by the client. On the first call localPL is zero. If RetrieveMail returns a PL, FtpServProt calls.
.once indent 14
(CtxRunning>>FtpCtx.RetrieveMailMessage)(remotePL, localPL)
which should transfer the file and return true. If something goes wrong, it should call FTPM(markNo, code string) and return false.
.end
Finally, FtpServProt calls
.once indent 5
(CtxRunning>>FtpCtx.RetrieveMailCleanup)(remotePL, ok).
If 'ok' is true, then RetrieveMailCleanup should flush the mailbox. If this operation fails, RetrieveMailCleanup should call FTPM(markNo, code, string) and return false, otherwise it should return true. If any resouces were allocated during the command, they should be deallocated here.
.sec |User|
The FTP User module (files FtpUserProt.br, FtpUserProtFile.br, and FtpUserProtMail.br) implements the User protocol exchanges.
Many of the procedures in this module report results by returning a word containing an FTP mark code in the right byte and a subcode in the left byte (referred to below as 'subcode,,mark'). Marks and subcodes are the first two arguments to the FTPM procedure which is described in more detail in the Utility section. If the mark type is 'markNo', the subcode describes the reason why the Server refused; your modules may be able to fix the problem and retry the command. The package will output to dspStream text accompanying No, Version, and Comment marks.
.ssec|Common User Protocol|
File FtpUserProt.bcpl contains routines shared by FtpUserProtFile.br and FtpUserProtMail.br. It uses the bspStream, bspSoc, and dspStream fields in its FtpCtx and contains the following external procedures:
.begin indent 4,4
.once indent 0
UserOpen(Version) = true|false
.continue
UserOpen should be called after the BSP Connection is open. It sends a version command and aborts the connection returning false if the Server's protocol is incompatible. Otherwise it calls
.once indent 9
Version(stream, nil)
which should generate some herald text:
.once indent 9
Wss(stream, "Alto Pup FTP User 1.13, 4 June 78").
The herald string received from the Server is output to dspStream.
.once indent 0
UserClose(abortIt)
.continue
UserClose closes the FTP connection, aborting it if 'abortIt' is true.
.once indent 0
UserFlushEOC() = true|false
.continue
flushes bspStream up to the next command, and returns true if it is EndOfCommand. If the stream closes or times out, it returns false. It calls UserProtocolError if it encounters anything except an EOC.
.once indent 0
UserGetYesNo(flushEOC) = subcode,,mark
.continue
flushes bspStream up to the next command, which must be 'Yes' or 'No'. If flushEOC is true, it then calls UserFlushEOC and returns the Yes or No mark and accompanying subCode. If the stream closes or times out, it returns false. UserGetYesNo calls UserProtocolError if it encounters anything except Yes or No followed by EOC.
.once indent 0
UserProtocolError()
.continue
Writes an error message to dspStream and then calls UserClose to abort the connection.
.end
.ssec|User File Operations|
File FtpUserProtFile.br implements the User protocol for standard file operations. It uses the bspStream, bspSoc, and dspStream fields in its FtpCtx and contains the following external procedures:
.begin indent 4,4
.once indent 0
UserStore(localL, StoreFile) = subcode,,mark
.continue
Attempts to send the file described by 'localL' to the remote Server, calling the user-supplied procedure 'StoreFile' to transfer the data. It returns zero if something catastrophic happens (such as the Server aborts the connection), in which case retrying is probably futile.
UserStore sends PL to the Server for approval. The Server can refuse the command at this point, in which case UserStore returns subcode,,markNo. If the Server accepts the command, it returns a PL (remotePL) specifying the destination file, and UserStore calls
.once indent 9
StoreFile(localL, remotePL)
which should transfer the file data. This package provides procedures for transferring data from a disk stream to a network stream, but you are free to write your own. StoreFile should return true if the transfer went successfully. If some environment-specific thing goes wrong (such as an unrecoverable disk error), StoreFile should call FTPM(markNo, code, string, true) and return false. UserStore then asks the Server if the transfer went successfully and returns subcode,,mark. If mark is 'markYes', the file arrived at the Server safely.
.once indent 0
UserRetrieve(localPL, Retrieve) = subcode,,mark
.continue
Attempts to retrieve the file described by localPL from the remote Server, calling the user-supplied procedure 'RetrieveFile' to transfer the data. UserRetrieve returns zero if some catastrophic error occurs, markNo if the Server refuses the command, and markEndOfCommand if the everything goes OK.
UserRetrieve sends localPL to the Server and waits for approval. The Server can refuse the command at this point, in which case UserRetieve returns subcode,,markNo. If the Server can handle property lists that specify multiple files, then the following steps are taken for each file:
.begin indent 9,9
If the Server has no more files matching localPL, UserRetrieve returns subcode,,markEndOfCommand (subcode is undefined in this case). Otherwise the Server sends a fully-specified property list describing a file which it is willing to send. UserRetrieve parses this into remotePL and calls
.once indent 14
Retrieve(remotePL, localPL)
which should decide whether to accept the file. To skip the file, Retrieve should return false. UserRetrieve so informs the Server and then loops. To accept the file, Retrieve should return a procedure which UserRetrieve can call to transfer the data. Don't open the file yet, because the Server can still back out, in which case UserRetrieve skips the next step and just loops. If Retrieve returns true, UserRetrieve tells the Server to send the file and then calls
.once indent 14
RetrieveFile(remotePL, localPL)
which should open the file, transfer the data, and close the file. This package contains procedures for transferring data from a network stream to a disk stream, but you are free to write your own. When RetrieveFile is done, it should return true if everything went OK, or false after calling FTPM(markNo, code, string) if something went wrong. UserRetrieve then loops.
.end
.once indent 0
UserDelete(localPL, Delete) = subcode,,mark
.continue
Requests the remote Server to delete the files described by localPL, calling the user-supplied procedure DeleteFile before allowing the server to actually delete anything. UserDelete returns zero if some catastrophic error occurs, markNo if the Server refuses the command, and markEndOfCommand if the everything goes OK.
UserDelete sends localPL to the Server and waits for approval. The Server can refuse the command at this point, in which case UserDelete returns subcode,,markNo. If the Server can handle property lists that specify multiple files, then the following steps are taken for each file:
.begin indent 9,9
If the Server has no more files matching the original pList, UserDelete returns subcode,,markEndOfCommand. Otherwise the Server sends a fully-specified property list describing a file which it is willing to delete. UserDelete parses this into remotePL and calls
.once indent 14
Delete(remotePL, localPL)
which should return true to confirm deleting the file described by remotePL. UserDelete passes this answer on to the Server and then loops.
.end
.once indent 0
UserDirectory(localPL, Directory) = subcode,,mark
.continue
Requests the remote Server to describe in as much detail as it can files matching localPL, and then calls the user-supplied procedure Directory when the answers come back.
UserDirectory sends localPL to the Server and waits for an answer. The Server can refuse the command at this point, in which case UserDirectory returns subcode,,markNo. If the Server can handle property lists that specify multiple files, then the following steps are taken for each file:
.begin indent 9,9
If the Server has no more files matching localPL, UserDirectory returns subcode,,markEndOfCommand. Otherwise the Server sends a property list which UserDirectory parses into remotePL and calls
.once indent 14
Directory(remotePL, localPL)
and then loops.
.end
.end
.ssec |User Mail Operations|
File FtpUserProtMail.br implements the user part of the Mail Transfer Protocol. This description ignores various critical sections and other vital considerations which must be handled by the user-supplied routines in order to provide a reliable mail service. For the semantics of the protocol see <Pup>MailTransfer.ears.
.begin indent 4,4
.once indent 0
UserStoreMail(pListGen, ExcpHandler, Xfer)
.continue
Attempts to send mail to the mailboxes described by the pLists generated by pListGen. It returns zero if something catastrophic happens (such as the Server aborts the connection), in which case retrying is probably futile.
UserStoreMail repeatedly calls the client-supplied procedure pListGen which should supply a pList describing a recipient of the message. When the last recipient has been generated, pListGen should return zero. The Server can refuse the command at this point, in which case UserStoreMail returns subcode,,markNo. If the Server accepts the command, it may still object to some of the mailboxes, in which case UserStoreMail calls the client-supplied procedure
.once indent 9
ExcpHandler(subcode, index)
which should record the recipient as rejected. Recipients are numbered from one, in the order in which they were generated by pListGen. Index is the number of the rejected recipient. A string describing why the recipient was rejected is in FtpCtx.getCmdString.
If after rejecting any recipients, there are still some valid ones, UserStoreMail calls the client-supplied procedure
.once indent 9
Xfer()
which should transfer the message text. Xfer should return true if the transfer went successfully. If some environment-specific thing goes wrong (such as an unrecoverable disk error), Xfer should call FTPM(markNo, code, string, true) before returning false. UserStoreMail then asks the Server if the transfer went successfully. The server can reject some more recipients at this point, in which case UserStoreMail calls the client-supplied procedure ExcpHandler again. Finally UserStoreMail returns subcode,,mark. If mark is 'markYes', the mail arrived at the Server safely.
.once indent 0
UserRetrieveMail(pList, RetrieveMail) = subCode,,mark
.continue
Attempts to retrieve the contents of the mailbox described by 'pList' from the remote Server, calling the user-supplied procedure 'RetrieveMail' to transfer the data. UserRetrieveMail returns zero if some catastrophic error occurs, markNo if the Server refuses the command, and markEndOfCommand if the everything goes OK.
UserRetrieveMail sends pList to the Server and waits for approval. The Server can refuse the command at this point, in which case UserRetieveMail returns subcode,,markNo. Otherwise UserRetrieveMail calls
.once indent 9
RetrieveMail(pList)
which should transfer the file data. When RetrieveMail is done, it should return true if everything went OK.
.end
.sec |Utility Routines|
The utility module (files FtpUtilB.br, FtpUtilA.br, FtpUtilXfer, FtpUtilDmpLd, and FtpUtilInit.br) contains protocol routines shared by the User and Server modules, and some routines for efficiently manipulating disk streams.
.begin indent 4,4
.once indent 0
InitFtpUtil()
.continue
builds some internal tables and streams, getting space from sysZone. You must call this procedure before starting a Server or issuing any User commands.
.once indent 0
FTPM(mark, subCode [0], string [], eoc [false], par0, par1, par2, par3, par4)
.continue
sends the FTP command 'mark' to the remote FTP process, including 'subCode' if the command requires one, and 'string' if one is present. Then, if 'eoc' is true, an EOC command is sent. 'String' is written to bspStream using the Template package, and may contain imbedded format information. 'Par0' through 'par4' are passed as arguments to the PutTemplate call. The subcode and string arguments further explain certain commands. For markNo, subCode is a machine-readable explanation of why a request was refused, and 'String' is human-readable text such as "UserName and Password required". Codes are tabulated in an appendix to <Pup>FtpSpec.ears. New codes may be registered on request.
.once indent 0
GetCommand(timeout [-1]) = subCode,,mark
.continue
flushes bspStream up to the next command and returns the mark and subcode (if any). Returns false if the stream closes or it hangs for 'timeout' miliseconds while waiting for a byte (the default, -1, waits forever). Comment commands are ignored. GetCommand writes the strings accompanying Version, No, and Comment commands to dspStream and stores a pointer to them in FtpCtx.getCmdString.
.end
The utility module makes three 'process-relative streams' for use by the rest of the package. The only operation defined is 'Puts'.
.begin nofill group tabs 5,15
\lst\writes to dspStream
\dls\writes to dspStream if debugFlag is true
\dbls\writes to bspStream and if debugFlag to dspStream
.end
For example, Wss(dls,string) writes 'string' to the running process' dspStream if the process' debugFlag is set.
.ssec |Unformatted Data Transfer|
File FtpUtilXfer.br contains procedures for performing efficient operations on Alto OS disk Streams. They use the following fields in FtpCtx: bspSoc, bspStream, dspStream, diskStream, buffer, and bufferLength.
.begin indent 4,4
.once indent 0
DiskToNet(remotePL, localPL) = true|false
.continue
Transfers bytes from diskStream to bspStream up to end-of-file, and returns true if everything went OK.
.once indent 0
NetToDisk(remotePL, localPL) = true|false
.continue
Transfers bytes from bspStream to diskStream until it encounters another FTP command returning true if everything went OK.
.once indent 0
FileType() = Text|Binary
.continue
Resets diskStream, scans it looking for high order bits on, and then Resets it again. As soon as it encounters a byte with the high order bit on, it returns 'Binary', otherwise (having read the entire file) it returns 'Text'.
.once indent 0
PrintBegin(localPL)
.continue
Outputs the server filename in remotePL and the type and byte size from localPL to dspStream.
.end
.ssec |Dump Format Data Transfer|
File FtpUtilDmpLd.br contains procedures for transferring data between a disk and an FTP connection in dump format. They may be used as the inner loops of user-supplied data transfer procedures passed to UserStore and UserRetrieve and will create and unbundle dump-format files on the fly. If you don't want to handle dump format, you don't need this file. Dump-file format is described in an appendix to the Alto Executive documentation.
These procedures use the same fields in FtpCtx and the same Alto OS routines as the unformatted transfer routines. Buffer must be at least 130 words long. Making it longer does not speed up the transfer.
.begin indent 4,4
.once indent 0
DumpToNet(remotePL, localPL) = true|false
.continue
Moves a file from diskStream to bspStream converting it to dump format, returning true if things go OK. The filename is taken from the name-body field of localPL, and the creation date from the creation date field. To terminate a dump file, call DumpToNet with remotePL = 0.
.once indent 0
LoadFromNet(remotePL, localPL) = true|false
.continue
Moves a file from bspStream to diskStream converting it from dump format, returning false when it encounters an 'end block'. When it encounters a file, it returns true with the filename and creation date in remotePL. If the client wants the file, he should call LoadFromNet again with FtpCtx.diskStream non zero; to skip a file set diskStream to zero.
.end
.ssec |Binary Compare Data Transfer|
Files FtpUtilCompB.br and FtpUtilCompA.br implement a binary compare of a network stream and a disk stream. If you don't want to do this (not many people will, I suspect), then you don't need these files. FtpUtilCompA contains two Block comparison procedures: one uses a fast machine code loop and the other uses special microcode which you must load into the Alto's ram.
.begin indent 4,4
.once indent 0
CompareNetWithDisk(remotePL, localPL) = true|false
.continue
Compares diskStream with bspStream byte-by-byte and reports the results to dspStream. If the two streams are identical (and the same length), then a string of the form "xxx identical bytes" is output, otherwise a string of the form "First difference at byte pos xxx" is output. Returns true if everything went OK, false if something catastrophic happened to the network connection (note in particular that a result of true implies nothing about whether the two streams were identical).
The following fields in the FtpCtx must be set up: bspStream, dspStream, diskStream, buffer, and bufferLength.
.end
.sec |Property Lists|
The property list module (files FtpPListProt.br, FtpPList1.br, and FtpPListInit.br) translates between this package's internal representation of a property list and the protocol-specified network representation.
The FTP protocol specifies the syntax of a property list and the syntax of a set of properties sufficient for standard file operations, but states that property lists are extensible. Therefore the property list module comes in two parts: a part that knows the syntax of property lists, and a part which knows the syntax of individual properties. To add new properties you need only modify the latter.
The principal data structure in this module is the Property List Keyword Table, or pListKT. This table, built by InitFtpPlist, contains (propertyName, propertyObject) pairs. PropertyNames are strings such as "Byte-size". PropertyObjects know how to Scan (parse) properties into pLists, Generate properties from pLists, Initialize properties from a pList full of default values, and Free properties stored in pLists.
.ssec |Property List Protocol|
File FtpPlistProt.br implements four operations on property lists. This is the module that knows the syntax of a property list, but not the syntax of individual properties. Procedures in this file use the bspStream, bspSoc, and dspStream fields of the FtpCtx and contain the following external procedures:
.begin indent 4,4
.once indent 0
InitPList(defaultPL []) = PL
.continue
Creates an empty pList, and initializes it to be a copy of 'defaultPL' if one was supplied.
.once indent 0
FreePList(PL)
.continue
Destroys PL and returns 0 to facilite writing PL = FreePList(PL). If PL is zero, FreePList returns zero without doing anything.
.once indent 0
ScanPList() = PL|false
.continue
Expects to find a property list in bspStream. ScanPList parses this property list and returns a PL if it had proper syntax. If the property list is malformed, ScanPList calls FTPM(markNo, code, string) and returns false. If ScanPList encounters a mark before starting a PL or the connection closes or Gets times out, it returns false.
.once indent 0
GenPList(PL)
.continue
Generates a property list in network format from PL and sends it to bspStream.
.end
.ssec |The 'Standard' Properties|
Files FtpPlist1.br and FtpPlistInit.br implement the standard properties. These files know the syntax of individual properties; they contain the operation procedures for the standard property objects. These files are used by the FTP subsystem and IFS and are sufficient for performing 'standard' file operations. If you wish to add properties, these are the modules which you must change. In addition to the property operations which are rather specialized to their task, there are a few generally useful procedures which are made external:
.begin indent 0,4
InitFtpPList()
.continue
which makes the standard property objects and builds fplKT, getting space from sysZone. This procedure must be called before calling any of the procedures in FtpPlist.br (which typically means before starting a server or calling any procedures in the User module).
.once indent 0
Nin(string, lvDest) = true|false
.continue
Interprets 'string' as a decimal number and leaves the result in 'lvDest', ignoring leading blanks and terminating on end of string. A null string results in lvDest getting 0. Returns false if the string contains any characters other than 0-9 and <space>.
.once indent 0
ParseDate(string, lvRes) = true|false
.continue
Parses the string format date into an Alto format date which it puts into the two word vector at 'lvRes'. Returns true if it could parse the date. ParseDate expects the format of the string to bear some similarity to "day-month-year hour:minute:second".
.once indent 0
WriteDT(stream, dt)
.continue
converts 'dt' from 32 bit Alto date format to a string of the form "dd-mmm-yy hh:mm:ss" and writes it to 'stream'.
.end
.sec |Example|
The following example program makes use of most of the facilities in the User part of the Ftp Package. I have run it and it works. It is a rock-bottom minimal User Ftp with no redeeming features whatsoever. More extensive and realistic examples can be found by looking at the sources for the Ftp subsystem.
The main procedure FtpUserExample performs initialization, which consists of augmenting SysZone, initializing the Ftp and Pup packages, and creating and starting a context running the procedure 'User'.
User opens a BSP connection to Maxc, sets up its FtpCtx, gets and fills a blank pList, and calls UserRetrieve. When UserRetrieve returns, User closes the connection, releases its resources and commits suicide.
.begin verbatim
//FtpUserExample.bcpl - Example Ftp User
//last modified April 9, 1978 4:24 PM
// The load command file is:
// Bldr/l/v 600/W FtpUserExample ↑
// ↑
// FtpUserProt FtpUserProtFile ↑
// FtpPListProt FtpPList1 ↑
// FtpUtilb FtpUtila FtpUtilXfer ↑
// ↑
// PupBspOpenClose PupBspStreams PupBspProt PupBspBlock PupBspA ↑
// PupRtpOpenClose PupRtp PupNameLookup ↑
// Pup1OpenClose Pup1B PupAl1A PupRoute PupDummyGate ↑
// PupAlEthB PupAlEthA ↑
// ↑
// Context ContextInit Interrupt ↑
// AltoQueue AltoTimer AltoByteBlt ↑
// Template CTime StringUtil Keyword ↑
// ↑
// FtpPlistInit FtpUtilInit KeywordInit ↑
// Pup1Init PupAlEthInit InterruptInit
get "FtpProt.decl"
get "Pup.decl"
external
[
//incoming procedures
InitFtpUtil; InitFtpPList; InitPupLevel1
GetFixed; CallSwat; AddToZone; Allocate; Free
InitializeContext; CallContextList; Enqueue
GetPartner; OpenLevel1Socket; OpenRTPSocket; CreateBSPStream
InitPList; FreePList; NetToDisk
UserRetrieve; UserOpen; UserClose; NetToDisk
ExtractSubstring; OpenFile; Closes; Wss
//incoming statics
sysZone; dsp; CtxRunning; UserName; UserPassword
]
let FtpUserExample() be
[
let v = GetFixed(10000)
if v eq 0 then CallSwat("GetFixed failed")
AddToZone(sysZone, v, 10000)
let ctxQ = vec 1; ctxQ!0 = 0
InitFtpUtil()
InitFtpPList()
InitPupLevel1(sysZone, ctxQ, 10)
Enqueue(ctxQ, InitializeContext(Allocate(sysZone, 1000), 1000,
User, lenExtraCtx))
CallContextList(ctxQ!0) repeat
]
and User(ctx) be //a context
[
let soc = Allocate(sysZone, lenBSPSoc)
let maxcPort = vec lenPort
unless GetPartner("Maxc", dsp, maxcPort, 0, socketFTP) do
CallSwat("GetPartner failed")
OpenLevel1Socket(soc, 0, maxcPort)
unless OpenRTPSocket(soc) do
CallSwat("OpenRTPSocket failed")
CtxRunning>>FtpCtx.bspStream = CreateBSPStream(soc)
CtxRunning>>FtpCtx.bspSoc = soc
CtxRunning>>FtpCtx.dspStream = dsp
CtxRunning>>FtpCtx.buffer = Allocate(sysZone, 256)
CtxRunning>>FtpCtx.bufferLength = 256
CtxRunning>>FtpCtx.debugFlag = true
unless UserOpen(Version) do
CallSwat("UserOpen failed")
let localPL = InitPList()
localPL>>PL.UNAM = ExtractSubstring(UserName)
localPL>>PL.UPSW = ExtractSubstring(UserPassword)
localPL>>PL.SFIL = ExtractSubstring("<system>Pup-Network.txt")
let mark = UserRetrieve(localPL, Retrieve)
if mark ne markEndOfCommand then
CallSwat("UserRetrieve failed")
FreePList(localPL)
UserClose()
Free(sysZone, soc)
Free(sysZone, CtxRunning>>FtpCtx.buffer)
finish
]
and Version(stream, nil) be Wss(stream, "Example FTP User")
and Retrieve(remotePL, localPL) = RetrieveFile
and RetrieveFile(remotePL, localPL) = valof
[
let s = OpenFile(remotePL>>PL.NAMB, ksTypeWriteOnly, charItem)
CtxRunning>>FtpCtx.diskStream = s
unless NetToDisk(remotePL, localPL) do CallSwat("NetToDisk failed")
Closes(s)
resultis true
]
.end
.sec |Revision History|
March 30, 1977
First release.
May 15, 1977
Added Directory and Rename commands. Server now handles property lists which specify multiple files. Added User and Server mail operations.
June 8, 1977
Overlay machinery was changed and some bugs were fixed. Some structure definitions changed, so recompilation of user programs is necessary.
July 17, 1977
DiskToNet and NetToDisk moved out of FtpUtilb into a new file FtpUtilXfer. Property lists reorganized, causing changes to the calling interface in FTPSI. Plist module now uses the Keyword routines in the CmdScan package. Recompilation of user programs is necessary. FtpUserDmpLd renamed FtpUtilDmpLd. Timeouts cleaned up.
October 24, 1977
Example program added.
February 14, 1978
Files FtpUtilCompB and FtpUtilCompA, implementing a byte-by-byte compare of the net stream with a disk stream added.
April 9, 1978
Implemented the new form of Store in which the Server returns a property list specifying the destination file. The old form is still supported, but no longer documented.
June 1, 1978
FtpServProt.bcpl split out of FtpServProtFile.bcpl. FtpServProtMail.bcpl updated to the current MTP. Many data structures changed so recompiliation of user programs is necessary.
September 20, 1980
Parameters passed to client routines changed. Both property lists are passed now. Recompilation is necessary.
December 16, 1980
Timeouts reworked. Statics 'getCmdTimeout' and 'getPutTimeout', and their default values, manifests 'defGetCmdTimeout' and 'defGetPutTimeout' were removed, since Pup byte stream activity timeouts, added about a year ago, do the same job. FtpServProt now takes a timeout which it uses while waiting for top level commands.
December 12, 1981
Internal data structure rearranged. Recompilation is necessary.