Command Scanner Package This package consists of an interactive command scanner and a collection of command interpretation procedures. Among the important features of this package are: 1. The editing facilities are fairly sophisticated. One can provide defaults and modify the break and echo sets on a per- phrase basis. The user is permitted to backspace over phrases that have already been parsed. Phrases may be interspersed with "noise" text that is retained with the command line while not logically a part of it. 2. Error recovery and retry facilities are provided by means of some rather tricky BCPL control structure. 3. The package is modular, and not all modules necessarily need be loaded. Also, specialized knowledge about the Alto display is confined to one module, which may be replaced by a different module that deals with other media such as hardcopy terminals or network streams. The Command Scanner Package is intended for use in programs with relatively sophisticated needs, and is fairly large (just the basic command editor and Alto display handling modules together amount to about 1500 words of code). Programmers with simpler needs or tight memory constraints might be better off using Bruce Parsley's Simple Dialoging Package. 1. Organization The package is distributed as a dump-format file CmdScan.dm, which contains the following files: CmdScan.decl Declarations that may be needed in order to use the package. CmdScan.br The main control module. This must always be loaded. CmdScanEdit.br Editing operations invoked from the main control module. This also must always be loaded. CmdScanDisplay.br Operations specific to the Alto environment (display and keyboard). This or some equivalent module must always be loaded. CmdScanTty.br Equivalent operations oriented toward a minimal terminal stream interface. ------------ Copyright Xerox Corporation 1981 Command Scanner Package November 21, 1981 2 CmdScanAux.br Higher-level command interpretation procedures for dealing with such things as numbers, strings, filenames, and keywords. This module is required only if its facilities are desired. Keyword.br Primitives to look up and enumerate keywords in a keyword table. Procedures in this module are called from the CmdScanAux module. KeywordInit.br Procedures to construct and manipulate keyword tables. This module may be discarded after all desired keyword tables have been created. CmdScanOEP.br Declarations of Overlay Entry Points (OEPs) in the CmdScan modules. This module is needed only if the CmdScan modules are loaded into overlays. KeywordOEP.br OEP declarations for the Keyword modules. The CmdScanAux module requires that the Timer and Context packages also be loaded. If one is not using contexts, one may omit the Context package and instead define an external procedure Block() that just returns immediately. 2. Basic Command Scanner Command scanning is done within the confines of a Command State (cs) object, which accumulates the text of a command and maintains state from one phrase to the next. A command consists of a sequence of phrases, possibly interspersed with "noise" text not part of any phrase. Each phrase consists of zero or more non-terminating characters followed by a terminating character. Editing is done on a phrase-by-phrase basis. For each phrase, the GetPhrase procedure is called to input a new phrase from the keyboard (terminated by a break character) and append it to the command line. GetPhrase returns when the terminating character is typed. At this point, the caller may call Gets(cs) to read the characters of the phrase (using Endofs(cs) to test for end of phrase). While control is inside GetPhrase, if the user backspaces past the beginning of the current phrase, control is sent back to an earlier point of interpretation so as to reparse the previous phrase now being editted. There exists a facility for regaining control when this happens so as to release resources acquired during command interpretation. Between phrases, one may output "noise" text by means of Puts(cs). This text is displayed and maintained in the command line but does not participate in editing operations. That is, if one is positioned at the end of a "noise" string and backspaces one character, the entire "noise" string is erased along with the real command character preceding it. Command Scanner Package November 21, 1981 3 2.1. Getting Phrases The following procedures are defined in CmdScan.br and CmdScanEdit.br: InitCmd(maxChars, maxPhrases, WordBreak [DefBreak], PhraseTerminator [DefBreak], Echo [DefEcho], keyS [keys], dspS [dsp], Erase [DefErase], Error [DefError], zone [sysZone]) = cs or 0 Creates and returns a Command State (cs) structure capable of holding at most maxChars characters grouped into at most maxPhrases phrases. keyS and dspS are the keyboard and display streams for the command scanner. The structure is allocated from zone. The remaining arguments (all of which are procedures) control the command scanner in various ways. These procedures are described below under "Edit Control Procedures". When InitCmd is called, it always returns a cs. However, if the command is deleted (by the user striking the Delete character) during later command typein, the cs is destroyed and InitCmd returns again with zero as its result. This is discussed below under "Backing Up and Catch Phrases". Closes(cs) Destroys the Command State structure cs, returning it to the zone from which it was allocated. GetPhrase(cs, WordBreak [default], PhraseTerminator [default], Echo [default], Help , helpArg ) = numChars Readies the next phrase to be interpreted, inputting one from the keyboard if necessary. Returns the number of characters in the phrase, not including the terminating character. The WordBreak, PhraseTerminator, and Echo procedures, if provided, override the ones declared to InitCmd for this phrase only. If Help is provided then upon typein of a question mark the call Help(dspS, helpArg) is executed; this is expected to output a helpful message to the stream dspS, not preceded or followed by a carriage return. (Typically the message is just a string, which may most easily be output by providing a Help procedure of Wss and a helpArg of the string itself.) Gets(cs) = char Returns the next character of the current phrase, i.e., the one most recently input by means of GetPhrase. If the phrase is exhausted (i.e., the next character would be the phrase terminator), Errors(cs, ecEndOfPhrase) is called. Endofs(cs) = true|false Returns true if the current phrase is exhausted. Puts(cs, char) Appends the "noise" character to the command line and outputs it to the command's display stream. Puts should be called only between phrases, i.e., after reading all characters of one phrase and before calling GetPhrase for the next. Resets(cs) Resets the command scanner to the beginning of the current phrase, such that the next call to GetPhrase will return the same phrase again. Command Scanner Package November 21, 1981 4 TerminatingChar(cs) = char Returns the character that terminated the current phrase. 2.2. Default Phrases DefaultPhrase(cs, string, char ) Supplies a default value (the string) for the next phrase; that is, the next call to GetPhrase will cause the text from that string to be returned. The string is appended to the command line and output to the command display stream. The string should not contain a terminating character. If char is supplied, it is used as the terminating character and the next call to GetPhrase will return without giving the user any opportunity to edit the phrase. If char is omitted, the next GetPhrase will wait for the user to either type a terminating character (in which case the default phrase will be returned) or provide a replacement phrase followed by a terminating character. BeginDefaultPhrase(cs) Begins a default phrase. All occurrences of Puts(cs, char) between calls to BeginDefaultPhrase and EndDefaultPhrase are included in the default phrase rather than treated as "noise" characters. This permits default phrases to be generated by arbitrary stream output. EndDefaultPhrase(cs, char ) Ends a default phrase started by BeginDefaultPhrase. If char is supplied, it is used as the terminating character, as described above under DefaultPhrase. BeginDefaultPhrase and EndDefaultPhrase must be paired and there must be no calls to GetPhrase between them. 2.3. Edit Control Procedures These procedures control the operation of the command scanner in various ways. The procedures are passed as arguments to InitCmd, and some of them to GetPhrase. The default procedures are all defined in CmdScanDisplay.br, but the programmer is free to substitute other ones when appropriate. The file CmdScanTty.br is an alternate to CmdScanDisplay.br, but oriented toward a minimal terminal stream interface. The only operations required are Gets and Resets on the keyboard stream and Puts on the display stream. WordBreak(cs, char) = true|false Returns true if char is a word break character and false otherwise. This controls the action of the control-W editing character and has no other effect. The default WordBreak procedure returns true only for space, escape, and carriage return. PhraseTerminator(cs, char) = true|false Returns true if char is a phrase terminating character and false otherwise. This controls the definition of a phrase, which is zero or more non-terminating characters followed by a terminating character. The default PhraseTerminator procedure returns true only for space, escape, and carriage return. Command Scanner Package November 21, 1981 5 Echo(cs, char) = true|false Returns true if char should be echoed when it is typed in and false otherwise. The default Echo procedure returns true if char is not a phrase terminator and false if it is (using whatever definition of phrase terminator is currently in effect). Erase(cs, first, last, context) Erases characters cs>>CS.buf>>Buf↑first through cs>>CS.buf>>Buf↑last (inclusive) from the output stream in whatever manner is appropriate for the medium. This interval may include both real phrase consituent characters and "noise" characters. Characters that were not echoed (i.e., not actually sent to the output stream) have #200 added to them and should be ignored. The context argument indicates the context in which the Erase procedure is being called; this may be useful in determining the correct action. eraseChar A single character is being erased. (It is the character cs>>CS.buf>>Buf↑first; any other characters are "noise".) eraseWord A word (or phrase) is being erased. eraseTerminator The terminating character of the current phrase is being erased to permit additional editing on the phrase. The default Erase procedure in the CmdScanDisplay module erases characters from the Alto display by means of EraseBits. If it is necessary to erase past the left margin (i.e., past a carriage return or a line wrap-around), the entire display window is erased and the command line is regenerated, thereby losing any text displayed before the beginning of the current command line. (This is necessary because the Operating System's display streams package generally does not permit one to manipulate other than the current display line.) The default Erase procedure in the CmdScanTty module prints a backslash followed by the erased character in the eraseChar case and a left arrow in the eraseWord case. Error(cs, ec) This is the stream error procedure for cs and is called under a variety of exceptional conditions. The error codes (ec) are defined in CmdScan.decl. Most of them indicate a specific error condition. However, a few simply request a certain action and are therefore generally useful in client command parsing routines. ecCmdDelete (called from GetPhrase) The Delete character has been typed. The Error procedure should take appropriate action and should not return. The default Error procedure types "XXX" and forces a return from the call to InitCmd with value zero. ecCmdTooLong The command line buffer is full and an attempt has been made to append another character to it. The maximum length is the maxChars Command Scanner Package November 21, 1981 6 argument to InitCmd. If the Error procedure returns the excess character is thrown away. The default Error procedure blinks the display, resets the keyboard, and returns. (The CmdScanTty module outputs a bell to the display stream.) ecTooManyPhrases Attempt to put more than maxPhrases phrases into the command line (maxPhrases is passed to InitCmd). This is an unrecoverable error, and the default Error procedure calls SysErr. ecEndOfPhrase Attempt to read characters past the end of the current phrase (by Gets(cs)). If the Error procedure returns, the result value is returned by Gets. The default Error procedure calls SysErr. ecKeyAmbiguous (called from GetKeyword, described later) An ambiguous keyword has been typed in. The default Error procedure blinks the display, resets the keyboard, and sends control back to an earlier point of interpretation so as to permit the user to type in more characters. ecBackupReplace This and the following error codes are not associated with specific errors but simply request that a certain action be performed. This one requests that control be sent back to the beginning of the current phrase to permit typein of a replacement phrase. ecBackupAppend Requests that control be sent back to the current phrase to permit the user to append to or edit it. ecCmdDestroy Requests that control be sent back to the InitCmd that began this command, forcing it to return zero. This is the same as ecCmdDelete except that "XXX" is not typed. other Any error code not listed above is assumed to be some sort of syntax error arising from a higher-level command interpreter (such as the ones in the CmdScanAux module). The default Error procedure handles all of them in the same way: it displays a question mark, blinks the display, resets the keyboard, and sends control back to an earlier point of interpretation so as to permit the user to replace or modify the current phrase. The following additional Alto display-specific procedures are defined in CmdScanDisplay.br: CmdError(cs, string ) If a string is supplied, outputs it to the command display stream. Then blinks the display window and issues a Resets operation on the command keyboard stream. (This procedure is also defined in the Command Scanner Package November 21, 1981 7 CmdScanTty module, but it outputs a bell to the display stream rather than blinking it.) InvertWindow(ds) Inverts the polarity of the display stream ds. That is, if it is now being displayed black on white, changes it to white on black or vice versa. 2.4. Backing Up and Catch Phrases When it becomes necessary to edit a phrase that has already been parsed (i.e., passed to the client program via GetPhrase and Gets), it is necessary to back up the interpretation of the command line to an earlier point so as to permit the modified phrase to be reparsed. This situation arises in several cases: the user backspaces past the beginning of the current phrase or deletes the entire command, or a syntax error is detected and the current phrase or a previous phrase must be replaced or modified. Rather than requiring GetPhrase and every higher-level procedure that calls GetPhrase to provide a failure indication (which the caller must then test after every call), the Command Scanner Package makes use of some devious control transfer primitives to back up control to an earlier point of interpretation, usually without the client program's being aware of it. In the simplest case, control is sent all the way back to the call to InitCmd that created the Command State (cs). InitCmd returns again with the same cs as before, and the entire command line is reparsed by the client program. Each call to GetPhrase (up to the phrase that is being modified) returns a phrase saved away in the command state, just as if it had just been typed in. The effects of the command scanner procedures during a reparse are indistinguishable from those during the initial parse. This control structure does have certain consequences that the programmer must be aware of. The first is that the context of the call to InitCmd must remain valid throughout the lifetime of the cs; that is, the procedure that called InitCmd must not return until the cs has been destroyed. Second, the interpretation of a given command line must have constant effects. That is, the result of reparsing the command must be indistinguishable from the result of parsing it initially--there must be no incremental or time-dependent variations in interpretation. There are situations in which resources are allocated during the course of command interpretation, e.g., storage blocks or open files. In some cases, when control is sent to an earlier point of interpretation, it is necessary to release such resources. The package provides a "catch phrase" mechanism by means of which the program can regain control so as to perform such cleanup. (The name is borrowed from Mesa, but the facility is not really very much like the Mesa "signal" and "catch phrase" machinery.) The catch phrase mechanism is accessed through the following procedures: EnableCatch(cs) = true|false Command Scanner Package November 21, 1981 8 When this call is encountered during normal interpretation, EnableCatch saves away the current frame and pc in storage associated with the next phrase (the phrase that will be read by the next call to GetPhrase). In this context, EnableCatch always returns false. While interpretation is being backed up, if a phrase is encountered for which an EnableCatch has been done, control is sent to that point; i.e., EnableCatch returns, but with value true rather than false. The programmer should write a statement of the form: if EnableCatch(cs) then [ <catch phrase>; EndCatch(cs) ] where <catch phrase> is code that performs the necessary cleanup. EndCatch(cs) Should be included at the end of every catch phrase. If control is being returned to a point of interpretation at or after the current phrase, EndCatch simply returns, thereby starting the reparse of succeeding phrases. However, if control is being sent back to a phrase before the current one, EndParse resumes the reverse transfer of control. Hence catch phrases are executed in reverse order, and the backing up of interpretation terminates at the latest catch phrase preceding the first phrase that must be reparsed. DisableCatch(cs) Undoes the effect of a previous EnableCatch for the current phrase. It may be issued before or after the GetPhrase that reads the current phrase. It is useful in situations where resources are allocated temporarily, across only one call to GetPhrase. The typical context is something like: if EnableCatch(cs) then [ <release resources>; EndCatch(cs) ] <allocate resources> GetPhrase(cs) <release resources> DisableCatch(cs) CmdErrorCode(cs) = ec If control is being backed up due to an error (including a command delete), this procedure returns the error code. If the user backspaced past the beginning of a phrase, zero is returned. This procedure returns a valid result only in the context of a catch phrase. As is the case for InitCmd, the context of every call to EnableCatch must remain valid during subsequent command interpretation. Effectively this means that calls to EnableCatch must be at the same or successively increasing depths of procedure calls. Also, only one catch phrase may be enabled per phrase in the command line. The call to EnableCatch must precede the call to GetPhrase for the particular phrase, though it may either precede or follow a DefaultPhrase providing a default value for that phrase. This restriction makes inclusion of catch phrases within iterations somewhat tricky, though it is still possible. A backup of interpretation is normally initiated only from within the Command Scanner Package November 21, 1981 9 Command Scanner Package itself, or from within an Error procedure called due to a syntax error. However, one may explicitly back up control by means of one of the following procedures: BackupPhrase(cs, nPh , editControl [editReplace], char ) Sends control backward nPh phrases relative to the current phrase (the default, zero, means restart interpretation of the current phrase). Note that BackupPhrase never returns. editControl determines the disposition of the current phrase, and may have one of the following values: editNew Discard the phrase and start over. (This option is not usually meaningful in the context of BackupPhrase, but is in ErasePhrase, described below.) editAppend Discard the phrase terminator and permit the user to append more characters to the phrase (or otherwise edit it). editReplace Discard the phrase terminator. If the first character typed by the user is a non-terminating, non-editing character, erase the entire phrase and start over (treating that character as the first character of the phrase); if it is an editing character, permit the user to edit the phrase as it stands; if it is a terminator, attempt to parse the phrase again with that terminator. If char is provided, it is effectively inserted at the front of the command keyboard stream and is used the next time GetPhrase needs to input a character from the user. ErasePhrase(cs, nPh , editControl [editReplace], char ) Same as BackupPhrase, but first erases all intervening phrases (both from the command line buffer and from the display). In this case, the editControl parameter applies to the target phrase rather than to the current phrase. The target phrase is erased only if editControl is editNew. 3. Auxiliary Command Interpreters The procedures in the CmdScanAux module each read a phrase (by calling GetPhrase) and interpret it in some way. While they are useful in their own right, they also serve as a good model for additional command interpretation procedures. In general the procedures return only if successful and call Errors with an appropriate error code otherwise. As previously explained, the default handling for these errors consists of backing up control to the beginning of the current phrase and permitting the user to replace or modify the phrase. Also, these procedures interpret only the phrase itself, not the terminating character. It is the caller's reponsibility to check the terminator if required. GetNumber(cs, radix , lvNumber ) = number Command Scanner Package November 21, 1981 10 Interprets the next phrase as a (possibly signed) integer in the specified radix. If lvNumber is provided, a 32-bit number is accepted and stored in @lvNumber; if lvNumber is not provided, a 16-bit number is accepted. In any event, GetNumber returns the low-order 16 bits of the number. If an error occurs, Errors(cs, ec) is called with one of the following error codes: ecEmptyNumber The phrase is empty. ecNonNumericChar The phrase contains a character that is not a digit in the specified radix. ecNumberOverflow The number is too large. GetString(cs, PhraseTerminator [default], Help , helpArg , Echo [default]) = string Returns the next phrase as a BCPL string. The optional arguments, if supplied, are passed to GetPhrase. The string is allocated from the same zone used to create cs. GetFile(cs, ksType, itemSize, versionControl, hintFp, errRtn, zone, logInfo, disk) = stream Calls OpenFile on the file whose name is the next phrase. All the arguments after cs are optional and are defaulted precisely as in OpenFile. If the file cannot be opened, calls Errors(cs, ecCantOpenFile). Confirm(cs, string ) = true|false Outputs the message "[Confirm]" preceded by the string if supplied. Then inputs a confirmation character and returns true if it is "Y" or carriage return and false if it is "N". Any other (non-editing) character causes Errors(cs, ecBadConfirmingChar) to be called. (Note that if Delete is typed, Confirm will not return but rather the entire command will be aborted.) GetKeyword(cs, kt, returnOnFail [false], PhraseTerminator [default]) = entry Looks up the next phrase in the keyword table kt (described later) and returns a pointer to the corresponding table entry. If the phrase is ambiguous, calls Errors(cs, ecKeyAmbiguous). If the phrase is not found, normally calls Errors(cs, ecKeyNotFound); however, if returnOnFail is true then returns zero in this case. If a unique initial substring match occurs and the terminating character has not been echoed, appends the remainder of the matching keyword to the command line and to the display as if it had been typed in. 4. Keyword Package This portion of the Command Scanner Package implements operations on an object called a Keyword Table. It is independent of the rest of the package and does not make use of any of its facilities. However, the CmdScanAux module does require the Keyword Package or some other package implementing equivalent operations. Command Scanner Package November 21, 1981 11 The Keyword Package consists of two principal modules. File KeywordInit.br contains procedures to create and modify a keyword table, while Keyword.br contains procedures to look up keywords and to enumerate and destroy the table. The reason for this division is to permit one to create all needed keyword tables at program initialization time and then to discard the code (which accounts for more than half the total size of the package). This package requires the StringUtil module of the Strings package, which in turn requires the ByteBlt package. All keyword table operations except CreateKeywordTable are actually accessed through the Calls mechanism (Call0, Call1, etc.), so alternate implementations of the same interface are possible. In particular, the CmdScanAux module requires only that the LookupKeyword and EnumerateKeywordTable operations be provided. A keyword table is an ordered set of <key, entry> pairs. The keys are BCPL strings and are maintained in alphabetical order for efficient lookup. The entries are fixed-length records whose interpretation is not defined by the package. While the lookup operation is efficient, the insert and delete operations are not, so this package is not suitable for maintaining large dictionaries or symbol tables. Its principal use is maintaining tables of keywords for applications such as command interpreters. Procedures contained in the KeywordInit module are: CreateKeywordTable(maxEntries, lenEntry , zone [sysZone]) = kt Creates and returns a keyword table (kt) capable of holding a maximum of maxEntries entries of lenEntry words each. The keyword table is allocated from the supplied zone and is initialized to empty. InsertKeyword(kt, key) = entry Inserts the supplied key (a BCPL string) into the keyword table kt and returns a pointer to the corresponding entry, which is initialized to all zeroes. The key string is copied; storage for the copy is obtained from the zone passed to CreateKeywordTable. It is the caller's responsibility to appropriately initialize the contents of the entry. If the keyword table is full or a duplicate entry is inserted, SysErr is called. DeleteKeyword(kt, key) Deletes the specified key (and its corresponding entry) from the keyword table kt. It is the caller's responsibility to dispose of any allocated objects pointed to by the deleted entry. If the key is not present in the table, SysErr is called. Procedures contained in the Keyword module are: LookupKeyword(kt, key, lvTableKey ) = entry Looks up the supplied key in the keyword table kt, returning a pointer to the corresponding entry if successful and zero if unsuccessful. For a successful lookup, the supplied key must either completely match a key in the table or be an initial substring of exactly one key. Upper- and lower-case letters are considered equivalent. Command Scanner Package November 21, 1981 12 If lvTableKey is supplied, a pointer to the full text of the matching keyword is stored in @lvTableKey if either a successful match or an ambiguous substring match occurs (zero is stored otherwise). In the case of an ambiguous substring match, the key stored is the first one that matches. This string is the one actually kept in the table (not a copy), so the caller must not modify it. EnumerateKeywordTable(kt, Proc, arg) Calls Proc(entry, kt, key, arg) for each entry in the keyword table kt. The called procedure may modify the entry but must not insert or delete keys. DestroyKeywordTable(kt) Destroys the keyword table kt, returning the table object and all keys to the zone from which they were allocated. It is the caller's responsibility to dispose of any allocated objects pointed to by entries in the table. Additionally, the following procedure (defined in Keyword.br) may be of interest: BinarySearch(key, tbl, lenTbl, Compare) = index Searches for key in the sorted table tbl, which has entries numbered zero to lenTbl-1 (inclusive). The comparison procedure Compare(key, tbl, i) is expected to compare key against entry i in the table and return a negative number if the key is "less than" the entry, zero if "equal", or a positive number if "greater than". All knowledge of the format of key and tbl is vested in the Compare procedure. If the requested key is found, BinarySearch returns the index of the matching entry in the table. If the key is not found, -i-1 (= not i) is returned, where i is the index of the first entry greater than the requested one (i.e., the key before which the requested key should be inserted). 5. Revision history July 14, 1977. First release. November 21, 1981. GetNumber accepts numbers that are either signed or unsigned and that are either 16 or 32 bits.