Processes
*********

In the terminology of operating systems, a process is a space in which a program can execute.  Emacs runs in a process.  Emacs Lisp programs can invoke other programs in processes of their own.  These are called subprocesses or child processes of the Emacs process, which is their parent process.

A subprocess of Emacs may be synchronous or asynchronous, depending on how it is created.  When you create a synchronous subprocess, the Lisp program waits for the subprocess to terminate before continuing execution.  When you create an asynchronous subprocess, it can run in parallel with the Lisp program.  This kind of subprocess is represented within Emacs by a Lisp object which is also called a ``process.''  Lisp programs can use this object to communicate with the subprocess or to control it.  For example, you can send signals, obtain status information, receive output from the process, or send input to it.

 -- Function: processp object
     This function returns t if OBJECT represents an Emacs subprocess, nil otherwise.

In addition to subprocesses of the current Emacs session, you can also access other processes running on your machine.  See System Processes.

Functions that Create Subprocesses
==================================

There are three primitives that create a new subprocess in which to run a program.  One of them, start-process, creates an asynchronous process and returns a process object (see Asynchronous Processes). The other two, call-process and call-process-region, create a synchronous process and do not return a process object (see Synchronous Processes).

Synchronous and asynchronous processes are explained in the following sections.  Since the three functions are all called in a similar fashion, their common arguments are described here.

In all cases, the function's PROGRAM argument specifies the program to be run.  An error is signaled if the file is not found or cannot be executed.  If the file name is relative, the variable exec-path contains a list of directories to search.  Emacs initializes exec-path when it starts up, based on the value of the environment variable PATH.  The standard file name constructs, ~, ., and .., are interpreted as usual in exec-path, but environment variable substitutions ($HOME, etc.) are not recognized; use substitute-in-file-name to perform them (see File Name Expansion).  nil in this list refers to default-directory.

Executing a program can also try adding suffixes to the specified name:

 -- Variable: exec-suffixes
     This variable is a list of suffixes (strings) to try adding to the specified program file name.  The list should include "" if you want the name to be tried exactly as specified.  The default value is system-dependent.

Please note: The argument PROGRAM contains only the name of the program; it may not contain any command-line arguments.  You must use ARGS to provide those.

Each of the subprocess-creating functions has a BUFFER-OR-NAME argument which specifies where the standard output from the program will go.  It should be a buffer or a buffer name; if it is a buffer name, that will create the buffer if it does not already exist.  It can also be nil, which says to discard the output unless a filter function handles it.  (see Filter Functions, and Read and Print.) Normally, you should avoid having multiple processes send output to the same buffer because their output would be intermixed randomly.

All three of the subprocess-creating functions have a &rest argument, ARGS.  The ARGS must all be strings, and they are supplied to PROGRAM as separate command line arguments.  Wildcard characters and other shell constructs have no special meanings in these strings, since the strings are passed directly to the specified program.

The subprocess gets its current directory from the value of default-directory (see File Name Expansion).

The subprocess inherits its environment from Emacs, but you can specify overrides for it with process-environment.  See System Environment.

 -- Variable: exec-directory
     The value of this variable is a string, the name of a directory that contains programs that come with GNU Emacs, programs intended for Emacs to invoke.  The program movemail is an example of such a program; Rmail uses it to fetch new mail from an inbox.

 -- User Option: exec-path
     The value of this variable is a list of directories to search for programs to run in subprocesses.  Each element is either the name of a directory (i.e., a string), or nil, which stands for the default directory (which is the value of default-directory).

     The value of exec-path is used by call-process and start-process when the PROGRAM argument is not an absolute file name.

Shell Arguments
===============

Lisp programs sometimes need to run a shell and give it a command that contains file names that were specified by the user.  These programs ought to be able to support any valid file name.  But the shell gives special treatment to certain characters, and if these characters occur in the file name, they will confuse the shell.  To handle these characters, use the function shell-quote-argument:

 -- Function: shell-quote-argument argument
     This function returns a string which represents, in shell syntax, an argument whose actual contents are ARGUMENT.  It should work reliably to concatenate the return value into a shell command and then pass it to a shell for execution.

     Precisely what this function does depends on your operating system.  The function is designed to work with the syntax of your system's standard shell; if you use an unusual shell, you will need to redefine this function.

          ;; This example shows the behavior on GNU and Unix systems.
          (shell-quote-argument "foo > bar")
               => "foo\\ \\>\\ bar"

          ;; This example shows the behavior on MS-DOS and MS-Windows.
          (shell-quote-argument "foo > bar")
               => "\"foo > bar\""

     Here's an example of using shell-quote-argument to construct a shell command:

          (concat "diff -c "
                  (shell-quote-argument oldfile)
                  " "
                  (shell-quote-argument newfile))

The following two functions are useful for creating shell commands from individual argument strings, and taking shell command lines apart into individual arguments.

 -- Function: split-string-and-unquote string &optional separators
     This function splits STRING into substrings at matches for the regular expression SEPARATORS, like split-string does (see Creating Strings); in addition, it removes quoting from the substrings.  It then makes a list of the substrings and returns it.

     If SEPARATORS is omitted or nil, it defaults to "\\s-+", which is a regular expression that matches one or more characters with whitespace syntax (see Syntax Class Table).

     This function performs two types of quoting: enclosing a whole string in double quotes "...", and quoting individual characters with a backslash escape \.  The latter is also used in Lisp strings, so this function can handle those as well.

 -- Function: combine-and-quote-strings list-of-strings &optional separator
     This function concatenates LIST-OF-STRINGS into a single string, quoting each string as necessary.  It also sticks the SEPARATOR string between each pair of strings; if SEPARATOR is omitted or nil, it defaults to " ".  The return value is the resulting string.

     The strings in LIST-OF-STRINGS that need quoting are those that include SEPARATOR as their substring.  Quoting a string encloses it in double quotes "...".  In the simplest case, if you are consing a shell command from the individual command-line arguments, every argument that includes embedded blanks will be quoted.

Creating a Synchronous Process
==============================

After a synchronous process is created, Emacs waits for the process to terminate before continuing.  Starting Dired on GNU or Unix (On other systems, Emacs uses a Lisp emulation of ls; see Contents of Directories.) is an example of this: it runs ls in a synchronous process, then modifies the output slightly.  Because the process is synchronous, the entire directory listing arrives in the buffer before Emacs tries to do anything with it.

While Emacs waits for the synchronous subprocess to terminate, the user can quit by typing C-g.  The first C-g tries to kill the subprocess with a SIGINT signal; but it waits until the subprocess actually terminates before quitting.  If during that time the user types another C-g, that kills the subprocess instantly with SIGKILL and quits immediately (except on MS-DOS, where killing other processes doesn't work).  See Quitting.

The synchronous subprocess functions return an indication of how the process terminated.

The output from a synchronous subprocess is generally decoded using a coding system, much like text read from a file.  The input sent to a subprocess by call-process-region is encoded using a coding system, much like text written into a file.  See Coding Systems.

 -- Function: call-process program &optional infile destination display &rest args
     This function calls PROGRAM in a separate process and waits for it to finish.

     The standard input for the process comes from file INFILE if INFILE is not nil, and from the null device otherwise. The argument DESTINATION says where to put the process output. Here are the possibilities:

     a buffer
          Insert the output in that buffer, before point.  This includes both the standard output stream and the standard error stream of the process.

     a string
          Insert the output in a buffer with that name, before point.

     t
          Insert the output in the current buffer, before point.

     nil
          Discard the output.

     0
          Discard the output, and return nil immediately without waiting for the subprocess to finish.

          In this case, the process is not truly synchronous, since it can run in parallel with Emacs; but you can think of it as synchronous in that Emacs is essentially finished with the subprocess as soon as this function returns.

          MS-DOS doesn't support asynchronous subprocesses, so this option doesn't work there.

     (REAL-DESTINATION ERROR-DESTINATION)
          Keep the standard output stream separate from the standard error stream; deal with the ordinary output as specified by REAL-DESTINATION, and dispose of the error output according to ERROR-DESTINATION. If ERROR-DESTINATION is nil, that means to discard the error output, t means mix it with the ordinary output, and a string specifies a file name to redirect error output into.

          You can't directly specify a buffer to put the error output in; that is too difficult to implement.  But you can achieve this result by sending the error output to a temporary file and then inserting the file into a buffer.

     If DISPLAY is non-nil, then call-process redisplays the buffer as output is inserted.  (However, if the coding system chosen for decoding output is undecided, meaning deduce the encoding from the actual data, then redisplay sometimes cannot continue once non-ASCII characters are encountered.  There are fundamental reasons why it is hard to fix this; see Output from Processes.)

     Otherwise the function call-process does no redisplay, and the results become visible on the screen only when Emacs redisplays that buffer in the normal course of events.

     The remaining arguments, ARGS, are strings that specify command line arguments for the program.

     The value returned by call-process (unless you told it not to wait) indicates the reason for process termination.  A number gives the exit status of the subprocess; 0 means success, and any other value means failure.  If the process terminated with a signal, call-process returns a string describing the signal.

     In the examples below, the buffer foo is current.

          (call-process "pwd" nil t)
               => 0

          ---------- Buffer: foo ----------
          /usr/user/lewis/manual
          ---------- Buffer: foo ----------

          (call-process "grep" nil "bar" nil "lewis" "/etc/passwd")
               => 0

          ---------- Buffer: bar ----------
          lewis:5LTsHm66CSWKg:398:21:Bil Lewis:/user/lewis:/bin/csh

          ---------- Buffer: bar ----------

     Here is a good example of the use of call-process, which used to be found in the definition of insert-directory:

          (call-process insert-directory-program nil t nil SWITCHES
                        (if full-directory-p
                            (concat (file-name-as-directory file) ".")
                          file))

 -- Function: process-file program &optional infile buffer display &rest args
     This function processes files synchronously in a separate process.  It is similar to call-process but may invoke a file handler based on the value of the variable default-directory.  The current working directory of the subprocess is default-directory.

     The arguments are handled in almost the same way as for call-process, with the following differences:

     Some file handlers may not support all combinations and forms of the arguments INFILE, BUFFER, and DISPLAY.  For example, some file handlers might behave as if DISPLAY were nil, regardless of the value actually passed.  As another example, some file handlers might not support separating standard output and error output by way of the BUFFER argument.

     If a file handler is invoked, it determines the program to run based on the first argument PROGRAM.  For instance, consider that a handler for remote files is invoked.  Then the path that is used for searching the program might be different than exec-path.

     The second argument INFILE may invoke a file handler.  The file handler could be different from the handler chosen for the process-file function itself.  (For example, default-directory could be on a remote host, whereas INFILE is on another remote host.  Or default-directory could be non-special, whereas INFILE is on a remote host.)

     If BUFFER is a list of the form (REAL-DESTINATION ERROR-DESTINATION), and ERROR-DESTINATION names a file, then the same remarks as for INFILE apply.

     The remaining arguments (ARGS) will be passed to the process verbatim.  Emacs is not involved in processing file names that are present in ARGS.  To avoid confusion, it may be best to avoid absolute file names in ARGS, but rather to specify all file names as relative to default-directory.  The function file-relative-name is useful for constructing such relative file names.

 -- Variable: process-file-side-effects
     This variable indicates, whether a call of process-file changes remote files.

     Per default, this variable is always set to t, meaning that a call of process-file could potentially change any file on a remote host.  When set to nil, a file handler could optimize its behaviour with respect to remote file attributes caching.

     This variable should never be changed by setq.  Instead of, it shall be set only by let-binding.

 -- Function: call-process-region start end program &optional delete destination display &rest args
     This function sends the text from START to END as standard input to a process running PROGRAM.  It deletes the text sent if DELETE is non-nil; this is useful when DESTINATION is t, to insert the output in the current buffer in place of the input.

     The arguments DESTINATION and DISPLAY control what to do with the output from the subprocess, and whether to update the display as it comes in.  For details, see the description of call-process, above.  If DESTINATION is the integer 0, call-process-region discards the output and returns nil immediately, without waiting for the subprocess to finish (this only works if asynchronous subprocesses are supported).

     The remaining arguments, ARGS, are strings that specify command line arguments for the program.

     The return value of call-process-region is just like that of call-process: nil if you told it to return without waiting; otherwise, a number or string which indicates how the subprocess terminated.

     In the following example, we use call-process-region to run the cat utility, with standard input being the first five characters in buffer foo (the word input).  cat copies its standard input into its standard output.  Since the argument DESTINATION is t, this output is inserted in the current buffer.

          ---------- Buffer: foo ----------
          input-!-
          ---------- Buffer: foo ----------

          (call-process-region 1 6 "cat" nil t)
               => 0

          ---------- Buffer: foo ----------
          inputinput-!-
          ---------- Buffer: foo ----------

     The shell-command-on-region command uses call-process-region like this:

          (call-process-region
           start end
           shell-file-name      ; Name of program.
           nil                  ; Do not delete region.
           buffer               ; Send output to buffer.
           nil                  ; No redisplay during output.
           "-c" command)        ; Arguments for the shell.

 -- Function: call-process-shell-command command &optional infile destination display &rest args
     This function executes the shell command COMMAND synchronously in a separate process.  The final arguments ARGS are additional arguments to add at the end of COMMAND.  The other arguments are handled as in call-process.

 -- Function: process-file-shell-command command &optional infile destination display &rest args
     This function is like call-process-shell-command, but uses process-file internally.  Depending on default-directory, COMMAND can be executed also on remote hosts.

 -- Function: shell-command-to-string command
     This function executes COMMAND (a string) as a shell command, then returns the command's output as a string.

 -- Function: process-lines program &rest args
     This function runs PROGRAM in a separate process, waits for it to finish, and returns its output as a list of strings.  Each string in the list holds a single line of text output by the program; the end-of-line characters are stripped from each line.  The arguments beyond PROGRAM, ARGS, are strings that specify command-line arguments with which to run the program.

     If PROGRAM exits with a non-zero exit status, this function signals an error.

     This function works by calling call-process, so program output is decoded in the same way as for call-process.

Creating an Asynchronous Process
================================

After an asynchronous process is created, Emacs and the subprocess both continue running immediately.  The process thereafter runs in parallel with Emacs, and the two can communicate with each other using the functions described in the following sections.  However, communication is only partially asynchronous: Emacs sends data to the process only when certain functions are called, and Emacs accepts data from the process only when Emacs is waiting for input or for a time delay.

Here we describe how to create an asynchronous process.

 -- Function: start-process name buffer-or-name program &rest args
     This function creates a new asynchronous subprocess and starts the program PROGRAM running in it.  It returns a process object that stands for the new subprocess in Lisp.  The argument NAME specifies the name for the process object; if a process with this name already exists, then NAME is modified (by appending <1>, etc.) to be unique.  The buffer BUFFER-OR-NAME is the buffer to associate with the process.

     The remaining arguments, ARGS, are strings that specify command line arguments for the program.

     In the example below, the first process is started and runs (rather, sleeps) for 100 seconds.  Meanwhile, the second process is started, and given the name my-process<1> for the sake of uniqueness.  It inserts the directory listing at the end of the buffer foo, before the first process finishes.  Then it finishes, and a message to that effect is inserted in the buffer.  Much later, the first process finishes, and another message is inserted in the buffer for it.

          (start-process "my-process" "foo" "sleep" "100")
               => #<process my-process>

          (start-process "my-process" "foo" "ls" "-l" "/user/lewis/bin")
               => #<process my-process<1>>

          ---------- Buffer: foo ----------
          total 2
          lrwxrwxrwx  1 lewis     14 Jul 22 10:12 gnuemacs --> /emacs
          -rwxrwxrwx  1 lewis     19 Jul 30 21:02 lemon

          Process my-process<1> finished

          Process my-process finished
          ---------- Buffer: foo ----------

 -- Function: start-file-process name buffer-or-name program &rest args
     Like start-process, this function starts a new asynchronous subprocess running PROGRAM in it, and returns its process object---when default-directory is not a magic file name.

     If default-directory is magic, the function invokes its file handler instead.  This handler ought to run PROGRAM, perhaps on the local host, perhaps on a remote host that corresponds to default-directory.  In the latter case, the local part of default-directory becomes the working directory of the process.

     This function does not try to invoke file name handlers for PROGRAM or for the PROGRAM-ARGS.

     Depending on the implementation of the file handler, it might not be possible to apply process-filter or process-sentinel to the resulting process object (see Filter Functions, see Sentinels).

     Some file handlers may not support start-file-process (for example ange-ftp-hook-function).  In such cases, the function does nothing and returns nil.

 -- Function: start-process-shell-command name buffer-or-name command
     This function is like start-process except that it uses a shell to execute the specified command.  The argument COMMAND is a shell command name.  The variable shell-file-name specifies which shell to use.

     The point of running a program through the shell, rather than directly with start-process, is so that you can employ shell features such as wildcards in the arguments.  It follows that if you include an arbitrary user-specified arguments in the command, you should quote it with shell-quote-argument first, so that any special shell characters do not have their special shell meanings.  See Shell Arguments.

 -- Function: start-file-process-shell-command name buffer-or-name command
     This function is like start-process-shell-command, but uses start-file-process internally.  By this, COMMAND can be executed also on remote hosts, depending on default-directory.

 -- Variable: process-connection-type
     This variable controls the type of device used to communicate with asynchronous subprocesses.  If it is non-nil, then PTYs are used, when available.  Otherwise, pipes are used.

     PTYs are usually preferable for processes visible to the user, as in Shell mode, because they allow job control (C-c, C-z, etc.) to work between the process and its children, whereas pipes do not.  For subprocesses used for internal purposes by programs, it is often better to use a pipe, because they are more efficient.  In addition, the total number of PTYs is limited on many systems and it is good not to waste them.

     The value of process-connection-type takes effect when start-process is called.  So you can specify how to communicate with one subprocess by binding the variable around the call to start-process.

          (let ((process-connection-type nil))  ; Use a pipe.
            (start-process ...))

     To determine whether a given subprocess actually got a pipe or a PTY, use the function process-tty-name (see Process Information).

Deleting Processes
==================

Deleting a process disconnects Emacs immediately from the subprocess.  Processes are deleted automatically after they terminate, but not necessarily right away.  You can delete a process explicitly at any time.  If you delete a terminated process explicitly before it is deleted automatically, no harm results.  Deleting a running process sends a signal to terminate it (and its child processes if any), and calls the process sentinel if it has one.  See Sentinels.

When a process is deleted, the process object itself continues to exist as long as other Lisp objects point to it.  All the Lisp primitives that work on process objects accept deleted processes, but those that do I/O or send signals will report an error.  The process mark continues to point to the same place as before, usually into a buffer where output from the process was being inserted.

 -- User Option: delete-exited-processes
     This variable controls automatic deletion of processes that have terminated (due to calling exit or to a signal).  If it is nil, then they continue to exist until the user runs list-processes.  Otherwise, they are deleted immediately after they exit.

 -- Function: delete-process process
     This function deletes a process, killing it with a SIGKILL signal.  The argument may be a process, the name of a process, a buffer, or the name of a buffer.  (A buffer or buffer-name stands for the process that get-buffer-process returns.)  Calling delete-process on a running process terminates it, updates the process status, and runs the sentinel (if any) immediately.  If the process has already terminated, calling delete-process has no effect on its status, or on the running of its sentinel (which will happen sooner or later).

          (delete-process "*shell*")
               => nil

Process Information
===================

Several functions return information about processes. list-processes is provided for interactive use.

 -- Command: list-processes &optional query-only
     This command displays a listing of all living processes.  In addition, it finally deletes any process whose status was Exited or Signaled.  It returns nil.

     If QUERY-ONLY is non-nil then it lists only processes whose query flag is non-nil.  See Query Before Exit.

 -- Function: process-list
     This function returns a list of all processes that have not been deleted.

          (process-list)
               => (#<process display-time> #<process shell>)

 -- Function: get-process name
     This function returns the process named NAME, or nil if there is none.  An error is signaled if NAME is not a string.

          (get-process "shell")
               => #<process shell>

 -- Function: process-command process
     This function returns the command that was executed to start PROCESS.  This is a list of strings, the first string being the program executed and the rest of the strings being the arguments that were given to the program.

          (process-command (get-process "shell"))
               => ("/bin/csh" "-i")

 -- Function: process-contact process &optional key

     This function returns information about how a network or serial process was set up.  For a network process, when KEY is nil, it returns (HOSTNAME SERVICE) which specifies what you connected to.  For a serial process, when KEY is nil, it returns (PORT SPEED).  For an ordinary child process, this function always returns t.

     If KEY is t, the value is the complete status information for the connection, server, or serial port; that is, the list of keywords and values specified in make-network-process or make-serial-process, except that some of the values represent the current status instead of what you specified.

     For a network process:

     :buffer
          The associated value is the process buffer.

     :filter
          The associated value is the process filter function.

     :sentinel
          The associated value is the process sentinel function.

     :remote
          In a connection, the address in internal format of the remote peer.

     :local
          The local address, in internal format.

     :service
          In a server, if you specified t for SERVICE, this value is the actual port number.

     :local and :remote are included even if they were not specified explicitly in make-network-process.

     For a serial process, see make-serial-process and serial-process-configure for a list of keys.

     If KEY is a keyword, the function returns the value corresponding to that keyword.

 -- Function: process-id process
     This function returns the PID of PROCESS.  This is an integer that distinguishes the process PROCESS from all other processes running on the same computer at the current time.  The PID of a process is chosen by the operating system kernel when the process is started and remains constant as long as the process exists.

 -- Function: process-name process
     This function returns the name of PROCESS.

 -- Function: process-status process-name
     This function returns the status of PROCESS-NAME as a symbol. The argument PROCESS-NAME must be a process, a buffer, or a process name (a string).

     The possible values for an actual subprocess are:

     run
          for a process that is running.

     stop
          for a process that is stopped but continuable.

     exit
          for a process that has exited.

     signal
          for a process that has received a fatal signal.

     open
          for a network connection that is open.

     closed
          for a network connection that is closed.  Once a connection is closed, you cannot reopen it, though you might be able to open a new connection to the same place.

     connect
          for a non-blocking connection that is waiting to complete.

     failed
          for a non-blocking connection that has failed to complete.

     listen
          for a network server that is listening.

     nil
          if PROCESS-NAME is not the name of an existing process.

          (process-status (get-buffer "*shell*"))
               => run

          x
               => #<process xx<1>>
          (process-status x)
               => exit

     For a network connection, process-status returns one of the symbols open or closed.  The latter means that the other side closed the connection, or Emacs did delete-process.

 -- Function: process-type process
     This function returns the symbol network for a network connection or server, serial for a serial port connection, or real for a real subprocess.

 -- Function: process-exit-status process
     This function returns the exit status of PROCESS or the signal number that killed it.  (Use the result of process-status to determine which of those it is.)  If PROCESS has not yet terminated, the value is 0.

 -- Function: process-tty-name process
     This function returns the terminal name that PROCESS is using for its communication with Emacs---or nil if it is using pipes instead of a terminal (see process-connection-type in Asynchronous Processes).

 -- Function: process-coding-system process
     This function returns a cons cell describing the coding systems in use for decoding output from PROCESS and for encoding input to PROCESS (see Coding Systems).  The value has this form:

          (CODING-SYSTEM-FOR-DECODING . CODING-SYSTEM-FOR-ENCODING)

 -- Function: set-process-coding-system process &optional decoding-system encoding-system
     This function specifies the coding systems to use for subsequent output from and input to PROCESS.  It will use DECODING-SYSTEM to decode subprocess output, and ENCODING-SYSTEM to encode subprocess input.

Every process also has a property list that you can use to store miscellaneous values associated with the process.

 -- Function: process-get process propname
     This function returns the value of the PROPNAME property of PROCESS.

 -- Function: process-put process propname value
     This function sets the value of the PROPNAME property of PROCESS to VALUE.

 -- Function: process-plist process
     This function returns the process plist of PROCESS.

 -- Function: set-process-plist process plist
     This function sets the process plist of PROCESS to PLIST.

Sending Input to Processes
==========================

Asynchronous subprocesses receive input when it is sent to them by Emacs, which is done with the functions in this section.  You must specify the process to send input to, and the input data to send.  The data appears on the ``standard input'' of the subprocess.

Some operating systems have limited space for buffered input in a PTY.  On these systems, Emacs sends an EOF periodically amidst the other characters, to force them through.  For most programs, these EOFs do no harm.

Subprocess input is normally encoded using a coding system before the subprocess receives it, much like text written into a file.  You can use set-process-coding-system to specify which coding system to use (see Process Information).  Otherwise, the coding system comes from coding-system-for-write, if that is non-nil; or else from the defaulting mechanism (see Default Coding Systems).

Sometimes the system is unable to accept input for that process, because the input buffer is full.  When this happens, the send functions wait a short while, accepting output from subprocesses, and then try again.  This gives the subprocess a chance to read more of its pending input and make space in the buffer.  It also allows filters, sentinels and timers to run---so take account of that in writing your code.

In these functions, the PROCESS argument can be a process or the name of a process, or a buffer or buffer name (which stands for a process via get-buffer-process).  nil means the current buffer's process.

 -- Function: process-send-string process string
     This function sends PROCESS the contents of STRING as standard input.  If it is nil, the current buffer's process is used.

     The function returns nil.

          (process-send-string "shell<1>" "ls\n")
               => nil

          ---------- Buffer: *shell* ----------
          ...
          introduction.texi               syntax-tables.texi~
          introduction.texi~              text.texi
          introduction.txt                text.texi~
          ...
          ---------- Buffer: *shell* ----------

 -- Function: process-send-region process start end
     This function sends the text in the region defined by START and END as standard input to PROCESS.

     An error is signaled unless both START and END are integers or markers that indicate positions in the current buffer.  (It is unimportant which number is larger.)

 -- Function: process-send-eof &optional process
     This function makes PROCESS see an end-of-file in its input.  The EOF comes after any text already sent to it.

     The function returns PROCESS.

          (process-send-eof "shell")
               => "shell"

 -- Function: process-running-child-p &optional process
     This function will tell you whether a PROCESS has given control of its terminal to its own child process.  The value is t if this is true, or if Emacs cannot tell; it is nil if Emacs can be certain that this is not so.

Sending Signals to Processes
============================

Sending a signal to a subprocess is a way of interrupting its activities.  There are several different signals, each with its own meaning.  The set of signals and their names is defined by the operating system.  For example, the signal SIGINT means that the user has typed C-c, or that some analogous thing has happened.

Each signal has a standard effect on the subprocess.  Most signals kill the subprocess, but some stop or resume execution instead.  Most signals can optionally be handled by programs; if the program handles the signal, then we can say nothing in general about its effects.

You can send signals explicitly by calling the functions in this section.  Emacs also sends signals automatically at certain times: killing a buffer sends a SIGHUP signal to all its associated processes; killing Emacs sends a SIGHUP signal to all remaining processes.  (SIGHUP is a signal that usually indicates that the user hung up the phone.)

Each of the signal-sending functions takes two optional arguments: PROCESS and CURRENT-GROUP.

The argument PROCESS must be either a process, a process name, a buffer, a buffer name, or nil.  A buffer or buffer name stands for a process through get-buffer-process.  nil stands for the process associated with the current buffer.  An error is signaled if PROCESS does not identify a process.

The argument CURRENT-GROUP is a flag that makes a difference when you are running a job-control shell as an Emacs subprocess.  If it is non-nil, then the signal is sent to the current process-group of the terminal that Emacs uses to communicate with the subprocess.  If the process is a job-control shell, this means the shell's current subjob.  If it is nil, the signal is sent to the process group of the immediate subprocess of Emacs.  If the subprocess is a job-control shell, this is the shell itself.

The flag CURRENT-GROUP has no effect when a pipe is used to communicate with the subprocess, because the operating system does not support the distinction in the case of pipes.  For the same reason, job-control shells won't work when a pipe is used.  See process-connection-type in Asynchronous Processes.

 -- Function: interrupt-process &optional process current-group
     This function interrupts the process PROCESS by sending the signal SIGINT.  Outside of Emacs, typing the ``interrupt character'' (normally C-c on some systems, and DEL on others) sends this signal.  When the argument CURRENT-GROUP is non-nil, you can think of this function as ``typing C-c'' on the terminal by which Emacs talks to the subprocess.

 -- Function: kill-process &optional process current-group
     This function kills the process PROCESS by sending the signal SIGKILL.  This signal kills the subprocess immediately, and cannot be handled by the subprocess.

 -- Function: quit-process &optional process current-group
     This function sends the signal SIGQUIT to the process PROCESS.  This signal is the one sent by the ``quit character'' (usually C-b or C-\) when you are not inside Emacs.

 -- Function: stop-process &optional process current-group
     This function stops the process PROCESS by sending the signal SIGTSTP.  Use continue-process to resume its execution.

     Outside of Emacs, on systems with job control, the ``stop character'' (usually C-z) normally sends this signal.  When CURRENT-GROUP is non-nil, you can think of this function as ``typing C-z'' on the terminal Emacs uses to communicate with the subprocess.

 -- Function: continue-process &optional process current-group
     This function resumes execution of the process PROCESS by sending it the signal SIGCONT.  This presumes that PROCESS was stopped previously.

 -- Function: signal-process process signal
     This function sends a signal to process PROCESS.  The argument SIGNAL specifies which signal to send; it should be an integer.

     The PROCESS argument can be a system process ID; that allows you to send signals to processes that are not children of Emacs.  See System Processes.

Receiving Output from Processes
===============================

There are two ways to receive the output that a subprocess writes to its standard output stream.  The output can be inserted in a buffer, which is called the associated buffer of the process, or a function called the filter function can be called to act on the output.  If the process has no buffer and no filter function, its output is discarded.

When a subprocess terminates, Emacs reads any pending output, then stops reading output from that subprocess.  Therefore, if the subprocess has children that are still live and still producing output, Emacs won't receive that output.

Output from a subprocess can arrive only while Emacs is waiting: when reading terminal input, in sit-for and sleep-for (see Waiting), and in accept-process-output (see Accepting Output).  This minimizes the problem of timing errors that usually plague parallel programming.  For example, you can safely create a process and only then specify its buffer or filter function; no output can arrive before you finish, if the code in between does not call any primitive that waits.

 -- Variable: process-adaptive-read-buffering
     On some systems, when Emacs reads the output from a subprocess, the output data is read in very small blocks, potentially resulting in very poor performance.  This behavior can be remedied to some extent by setting the variable PROCESS-ADAPTIVE-READ-BUFFERING to a non-nil value (the default), as it will automatically delay reading from such processes, thus allowing them to produce more output before Emacs tries to read it.

It is impossible to separate the standard output and standard error streams of the subprocess, because Emacs normally spawns the subprocess inside a pseudo-TTY, and a pseudo-TTY has only one output channel.  If you want to keep the output to those streams separate, you should redirect one of them to a file---for example, by using an appropriate shell command.

Process Buffers
---------------

A process can (and usually does) have an associated buffer, which is an ordinary Emacs buffer that is used for two purposes: storing the output from the process, and deciding when to kill the process.  You can also use the buffer to identify a process to operate on, since in normal practice only one process is associated with any given buffer. Many applications of processes also use the buffer for editing input to be sent to the process, but this is not built into Emacs Lisp.

Unless the process has a filter function (see Filter Functions), its output is inserted in the associated buffer.  The position to insert the output is determined by the process-mark, which is then updated to point to the end of the text just inserted.  Usually, but not always, the process-mark is at the end of the buffer.

Killing the associated buffer of a process also kills the process. Emacs asks for confirmation first, if the process's process-query-on-exit-flag is non-nil (see Query Before Exit).  This confirmation is done by the function process-kill-buffer-query-function, which is run from kill-buffer-query-functions (see Killing Buffers).

 -- Function: process-buffer process
     This function returns the associated buffer of the process PROCESS.

          (process-buffer (get-process "shell"))
               => #<buffer *shell*>

 -- Function: process-mark process
     This function returns the process marker for PROCESS, which is the marker that says where to insert output from the process.

     If PROCESS does not have a buffer, process-mark returns a marker that points nowhere.

     Insertion of process output in a buffer uses this marker to decide where to insert, and updates it to point after the inserted text.  That is why successive batches of output are inserted consecutively.

     Filter functions normally should use this marker in the same fashion as is done by direct insertion of output in the buffer.  A good example of a filter function that uses process-mark is found at the end of the following section.

     When the user is expected to enter input in the process buffer for transmission to the process, the process marker separates the new input from previous output.

 -- Function: set-process-buffer process buffer
     This function sets the buffer associated with PROCESS to BUFFER.  If BUFFER is nil, the process becomes associated with no buffer.

 -- Function: get-buffer-process buffer-or-name
     This function returns a nondeleted process associated with the buffer specified by BUFFER-OR-NAME.  If there are several processes associated with it, this function chooses one (currently, the one most recently created, but don't count on that).  Deletion of a process (see delete-process) makes it ineligible for this function to return.

     It is usually a bad idea to have more than one process associated with the same buffer.

          (get-buffer-process "*shell*")
               => #<process shell>

     Killing the process's buffer deletes the process, which kills the subprocess with a SIGHUP signal (see Signals to Processes).

Process Filter Functions
------------------------

A process filter function is a function that receives the standard output from the associated process.  If a process has a filter, then all output from that process is passed to the filter.  The process buffer is used directly for output from the process only when there is no filter.

The filter function can only be called when Emacs is waiting for something, because process output arrives only at such times.  Emacs waits when reading terminal input, in sit-for and sleep-for (see Waiting), and in accept-process-output (see Accepting Output).

A filter function must accept two arguments: the associated process and a string, which is output just received from it.  The function is then free to do whatever it chooses with the output.

Quitting is normally inhibited within a filter function---otherwise, the effect of typing C-g at command level or to quit a user command would be unpredictable.  If you want to permit quitting inside a filter function, bind inhibit-quit to nil.  In most cases, the right way to do this is with the macro with-local-quit.  See Quitting.

If an error happens during execution of a filter function, it is caught automatically, so that it doesn't stop the execution of whatever program was running when the filter function was started.  However, if debug-on-error is non-nil, the error-catching is turned off.  This makes it possible to use the Lisp debugger to debug the filter function.  See Debugger.

Many filter functions sometimes or always insert the text in the process's buffer, mimicking the actions of Emacs when there is no filter.  Such filter functions need to use set-buffer in order to be sure to insert in that buffer.  To avoid setting the current buffer semipermanently, these filter functions must save and restore the current buffer.  They should also update the process marker, and in some cases update the value of point.  Here is how to do these things:

     (defun ordinary-insertion-filter (proc string)
       (with-current-buffer (process-buffer proc)
         (let ((moving (= (point) (process-mark proc))))

           (save-excursion
             ;; Insert the text, advancing the process marker.
             (goto-char (process-mark proc))
             (insert string)
             (set-marker (process-mark proc) (point)))
           (if moving (goto-char (process-mark proc))))))

The reason to use with-current-buffer, rather than using save-excursion to save and restore the current buffer, is so as to preserve the change in point made by the second call to goto-char.

To make the filter force the process buffer to be visible whenever new text arrives, insert the following line just before the with-current-buffer construct:

     (display-buffer (process-buffer proc))

To force point to the end of the new output, no matter where it was previously, eliminate the variable moving and call goto-char unconditionally.

In earlier Emacs versions, every filter function that did regular expression searching or matching had to explicitly save and restore the match data.  Now Emacs does this automatically for filter functions; they never need to do it explicitly.  See Match Data.

A filter function that writes the output into the buffer of the process should check whether the buffer is still alive.  If it tries to insert into a dead buffer, it will get an error.  The expression (buffer-name (process-buffer PROCESS)) returns nil if the buffer is dead.

The output to the function may come in chunks of any size.  A program that produces the same output twice in a row may send it as one batch of 200 characters one time, and five batches of 40 characters the next.  If the filter looks for certain text strings in the subprocess output, make sure to handle the case where one of these strings is split across two or more batches of output; one way to do this is to insert the received text into a temporary buffer, which can then be searched.

 -- Function: set-process-filter process filter
     This function gives PROCESS the filter function FILTER.  If FILTER is nil, it gives the process no filter.

 -- Function: process-filter process
     This function returns the filter function of PROCESS, or nil if it has none.

Here is an example of use of a filter function:

     (defun keep-output (process output)
        (setq kept (cons output kept)))
          => keep-output

     (setq kept nil)
          => nil

     (set-process-filter (get-process "shell") 'keep-output)
          => keep-output

     (process-send-string "shell" "ls ~/other\n")
          => nil
     kept
          => ("lewis@slug[8] % "

     "FINAL-W87-SHORT.MSS    backup.otl              kolstad.mss~
     address.txt             backup.psf              kolstad.psf
     backup.bib~             david.mss               resume-Dec-86.mss~
     backup.err              david.psf               resume-Dec.psf
     backup.mss              dland                   syllabus.mss
     "
     "#backups.mss#          backup.mss~             kolstad.mss
     ")

Decoding Process Output
-----------------------

When Emacs writes process output directly into a multibyte buffer, it decodes the output according to the process output coding system. If the coding system is raw-text or no-conversion, Emacs converts the unibyte output to multibyte using string-to-multibyte, and inserts the resulting multibyte text.

You can use set-process-coding-system to specify which coding system to use (see Process Information).  Otherwise, the coding system comes from coding-system-for-read, if that is non-nil; or else from the defaulting mechanism (see Default Coding Systems).  If the text output by a process contains null bytes, Emacs by default uses no-conversion for it; see inhibit-null-byte-detection, for how to control this behavior.

Warning: Coding systems such as undecided which determine the coding system from the data do not work entirely reliably with asynchronous subprocess output.  This is because Emacs has to process asynchronous subprocess output in batches, as it arrives.  Emacs must try to detect the proper coding system from one batch at a time, and this does not always work.  Therefore, if at all possible, specify a coding system that determines both the character code conversion and the end of line conversion---that is, one like latin-1-unix, rather than undecided or latin-1.

When Emacs calls a process filter function, it provides the process output as a multibyte string or as a unibyte string according to the process's filter coding system.  Emacs decodes the output according to the process output coding system, which usually produces a multibyte string, except for coding systems such as binary and raw-text

Accepting Output from Processes
-------------------------------

Output from asynchronous subprocesses normally arrives only while Emacs is waiting for some sort of external event, such as elapsed time or terminal input.  Occasionally it is useful in a Lisp program to explicitly permit output to arrive at a specific point, or even to wait until output arrives from a process.

 -- Function: accept-process-output &optional process seconds millisec just-this-one
     This function allows Emacs to read pending output from processes.  The output is inserted in the associated buffers or given to their filter functions.  If PROCESS is non-nil then this function does not return until some output has been received from PROCESS.

     The arguments SECONDS and MILLISEC let you specify timeout periods.  The former specifies a period measured in seconds and the latter specifies one measured in milliseconds.  The two time periods thus specified are added together, and accept-process-output returns after that much time, whether or not there has been any subprocess output.

     The argument MILLISEC is semi-obsolete nowadays because SECONDS can be a floating point number to specify waiting a fractional number of seconds.  If SECONDS is 0, the function accepts whatever output is pending but does not wait.

     If PROCESS is a process, and the argument JUST-THIS-ONE is non-nil, only output from that process is handled, suspending output from other processes until some output has been received from that process or the timeout expires.  If JUST-THIS-ONE is an integer, also inhibit running timers.  This feature is generally not recommended, but may be necessary for specific applications, such as speech synthesis.

     The function accept-process-output returns non-nil if it did get some output, or nil if the timeout expired before output arrived.

Sentinels: Detecting Process Status Changes
===========================================

A process sentinel is a function that is called whenever the associated process changes status for any reason, including signals (whether sent by Emacs or caused by the process's own actions) that terminate, stop, or continue the process.  The process sentinel is also called if the process exits.  The sentinel receives two arguments: the process for which the event occurred, and a string describing the type of event.

The string describing the event looks like one of the following:

  * "finished\n".

  * "exited abnormally with code EXITCODE\n".

  * "NAME-OF-SIGNAL\n".

  * "NAME-OF-SIGNAL (core dumped)\n".

A sentinel runs only while Emacs is waiting (e.g., for terminal input, or for time to elapse, or for process output).  This avoids the timing errors that could result from running them at random places in the middle of other Lisp programs.  A program can wait, so that sentinels will run, by calling sit-for or sleep-for (see Waiting), or accept-process-output (see Accepting Output).  Emacs also allows sentinels to run when the command loop is reading input.  delete-process calls the sentinel when it terminates a running process.

Emacs does not keep a queue of multiple reasons to call the sentinel of one process; it records just the current status and the fact that there has been a change.  Therefore two changes in status, coming in quick succession, can call the sentinel just once.  However, process termination will always run the sentinel exactly once.  This is because the process status can't change again after termination.

Emacs explicitly checks for output from the process before running the process sentinel.  Once the sentinel runs due to process termination, no further output can arrive from the process.

A sentinel that writes the output into the buffer of the process should check whether the buffer is still alive.  If it tries to insert into a dead buffer, it will get an error.  If the buffer is dead, (buffer-name (process-buffer PROCESS)) returns nil.

Quitting is normally inhibited within a sentinel---otherwise, the effect of typing C-g at command level or to quit a user command would be unpredictable.  If you want to permit quitting inside a sentinel, bind inhibit-quit to nil.  In most cases, the right way to do this is with the macro with-local-quit. See Quitting.

If an error happens during execution of a sentinel, it is caught automatically, so that it doesn't stop the execution of whatever programs was running when the sentinel was started.  However, if debug-on-error is non-nil, the error-catching is turned off.  This makes it possible to use the Lisp debugger to debug the sentinel.  See Debugger.

While a sentinel is running, the process sentinel is temporarily set to nil so that the sentinel won't run recursively. For this reason it is not possible for a sentinel to specify a new sentinel.

In earlier Emacs versions, every sentinel that did regular expression searching or matching had to explicitly save and restore the match data. Now Emacs does this automatically for sentinels; they never need to do it explicitly.  See Match Data.

 -- Function: set-process-sentinel process sentinel
     This function associates SENTINEL with PROCESS.  If SENTINEL is nil, then the process will have no sentinel. The default behavior when there is no sentinel is to insert a message in the process's buffer when the process status changes.

     Changes in process sentinel take effect immediately---if the sentinel is slated to be run but has not been called yet, and you specify a new sentinel, the eventual call to the sentinel will use the new one.

          (defun msg-me (process event)
             (princ
               (format "Process: %s had the event `%s'" process event)))
          (set-process-sentinel (get-process "shell") 'msg-me)
               => msg-me

          (kill-process (get-process "shell"))
               -| Process: #<process shell> had the event `killed'
               => #<process shell>

 -- Function: process-sentinel process
     This function returns the sentinel of PROCESS, or nil if it has none.

 -- Function: waiting-for-user-input-p
     While a sentinel or filter function is running, this function returns non-nil if Emacs was waiting for keyboard input from the user at the time the sentinel or filter function was called, nil if it was not.

Querying Before Exit
====================

When Emacs exits, it terminates all its subprocesses by sending them the SIGHUP signal.  Because subprocesses may be doing valuable work, Emacs normally asks the user to confirm that it is ok to terminate them.  Each process has a query flag which, if non-nil, says that Emacs should ask for confirmation before exiting and thus killing that process.  The default for the query flag is t, meaning do query.

 -- Function: process-query-on-exit-flag process
     This returns the query flag of PROCESS.

 -- Function: set-process-query-on-exit-flag process flag
     This function sets the query flag of PROCESS to FLAG.  It returns FLAG.

          ;; Don't query about the shell process
          (set-process-query-on-exit-flag (get-process "shell") nil)
               => t

 -- Function: process-kill-without-query process &optional do-query
     This function clears the query flag of PROCESS, so that Emacs will not query the user on account of that process.

     Actually, the function does more than that: it returns the old value of the process's query flag, and sets the query flag to DO-QUERY. Please don't use this function to do those things any more---please use the newer, cleaner functions process-query-on-exit-flag and set-process-query-on-exit-flag in all but the simplest cases. The only way you should use process-kill-without-query nowadays is like this:

          ;; Don't query about the shell process
          (process-kill-without-query (get-process "shell"))

Accessing Other Processes
=========================

In addition to accessing and manipulating processes that are subprocesses of the current Emacs session, Emacs Lisp programs can also access other processes running on the same machine.  We call these system processes, to distinguish between them and Emacs subprocesses.

Emacs provides several primitives for accessing system processes. Not all platforms support these primitives; on those which don't, these primitives return nil.

 -- Function: list-system-processes
     This function returns a list of all the processes running on the system.  Each process is identified by its PID, a numerical process ID that is assigned by the OS and distinguishes the process from all the other processes running on the same machine at the same time.

 -- Function: process-attributes pid
     This function returns an alist of attributes for the process specified by its process ID PID.  Each association in the alist is of the form (KEY . VALUE), where KEY designates the attribute and VALUE is the value of that attribute.  The various attribute KEY's that this function can return are listed below. Not all platforms support all of these attributes; if an attribute is not supported, its association will not appear in the returned alist. Values that are numbers can be either integer or floating-point, depending on the magnitude of the value.

     euid
          The effective user ID of the user who invoked the process.  The corresponding VALUE is a number.  If the process was invoked by the same user who runs the current Emacs session, the value is identical to what user-uid returns (see User Identification).

     user
          User name corresponding to the process's effective user ID, a string.

     egid
          The group ID of the effective user ID, a number.

     group
          Group name corresponding to the effective user's group ID, a string.

     comm
          The name of the command that runs in the process.  This is a string that usually specifies the name of the executable file of the process, without the leading directories.  However, some special system processes can report strings that do not correspond to an executable file of a program.

     state
          The state code of the process.  This is a short string that encodes the scheduling state of the process.  Here's a list of the most frequently seen codes:

          "D"
               uninterruptible sleep (usually I/O)

          "R"
               running

          "S"
               interruptible sleep (waiting for some event)

          "T"
               stopped, e.g., by a job control signal

          "Z"
               ``zombie'': a process that terminated, but was not reaped by its parent

          For the full list of the possible states, see the manual page of the ps command.

     ppid
          The process ID of the parent process, a number.

     pgrp
          The process group ID of the process, a number.

     sess
          The session ID of the process.  This is a number that is the process ID of the process's session leader.

     ttname
          A string that is the name of the process's controlling terminal.  On Unix and GNU systems, this is normally the file name of the corresponding terminal device, such as /dev/pts65.

     tpgid
          The numerical process group ID of the foreground process group that uses the process's terminal.

     minflt
          The number of minor page faults caused by the process since its beginning.  (Minor page faults are those that don't involve reading from disk.)

     majflt
          The number of major page faults caused by the process since its beginning.  (Major page faults require a disk to be read, and are thus more expensive than minor page faults.)

     cminflt
     cmajflt
          Like minflt and majflt, but include the number of page faults for all the child processes of the given process.

     utime
          Time spent by the process in the user context, for running the application's code.  The corresponding VALUE is in the (HIGH LOW MICROSEC) format, the same format used by functions current-time (see current-time) and file-attributes (see File Attributes).

     stime
          Time spent by the process in the system (kernel) context, for processing system calls.  The corresponding VALUE is in the same format as for utime.

     time
          The sum of utime and stime.  The corresponding VALUE is in the same format as for utime.

     cutime
     cstime
     ctime
          Like utime, stime, and time, but include the times of all the child processes of the given process.

     pri
          The numerical priority of the process.

     nice
          The nice value of the process, a number.  (Processes with smaller nice values get scheduled more favorably.)

     thcount
          The number of threads in the process.

     start
          The time the process was started, in the (HIGH LOW MICROSEC) format used by current-time and file-attributes.

     etime
          The time elapsed since the process started, in the (HIGH LOW MICROSEC) format.

     vsize
          The virtual memory size of the process, measured in kilobytes.

     rss
          The size of the process's resident set, the number of kilobytes occupied by the process in the machine's physical memory.

     pcpu
          The percentage of the CPU time used by the process since it started. The corresponding VALUE is a floating-point number between 0 and 100.

     pmem
          The percentage of the total physical memory installed on the machine used by the process's resident set.  The value is a floating-point number between 0 and 100.

     args
          The command-line with which the process was invoked.  This is a string in which individual command-line arguments are separated by blanks; whitespace characters that are embedded in the arguments are quoted as appropriate for the system's shell: escaped by backslash characters on GNU and Unix, and enclosed in double quote characters on Windows. Thus, this command-line string can be directly used in primitives such as shell-command.

Transaction Queues
==================

You can use a transaction queue to communicate with a subprocess using transactions.  First use tq-create to create a transaction queue communicating with a specified process.  Then you can call tq-enqueue to send a transaction.

 -- Function: tq-create process
     This function creates and returns a transaction queue communicating with PROCESS.  The argument PROCESS should be a subprocess capable of sending and receiving streams of bytes.  It may be a child process, or it may be a TCP connection to a server, possibly on another machine.

 -- Function: tq-enqueue queue question regexp closure fn &optional delay-question
     This function sends a transaction to queue QUEUE.  Specifying the queue has the effect of specifying the subprocess to talk to.

     The argument QUESTION is the outgoing message that starts the transaction.  The argument FN is the function to call when the corresponding answer comes back; it is called with two arguments: CLOSURE, and the answer received.

     The argument REGEXP is a regular expression that should match text at the end of the entire answer, but nothing before; that's how tq-enqueue determines where the answer ends.

     If the argument DELAY-QUESTION is non-nil, delay sending this question until the process has finished replying to any previous questions.  This produces more reliable results with some processes.

     The return value of tq-enqueue itself is not meaningful.

 -- Function: tq-close queue
     Shut down transaction queue QUEUE, waiting for all pending transactions to complete, and then terminate the connection or child process.

Transaction queues are implemented by means of a filter function. See Filter Functions.

Network Connections
===================

Emacs Lisp programs can open stream (TCP) and datagram (UDP) network connections to other processes on the same machine or other machines. A network connection is handled by Lisp much like a subprocess, and is represented by a process object.  However, the process you are communicating with is not a child of the Emacs process, so it has no process ID, and you can't kill it or send it signals.  All you can do is send and receive data.  delete-process closes the connection, but does not kill the program at the other end; that program must decide what to do about closure of the connection.

Lisp programs can listen for connections by creating network servers.  A network server is also represented by a kind of process object, but unlike a network connection, the network server never transfers data itself.  When it receives a connection request, it creates a new network connection to represent the connection just made.  (The network connection inherits certain information, including the process plist, from the server.)  The network server then goes back to listening for more connection requests.

Network connections and servers are created by calling make-network-process with an argument list consisting of keyword/argument pairs, for example :server t to create a server process, or :type 'datagram to create a datagram connection.  See Low-Level Network, for details.  You can also use the open-network-stream function described below.

To distinguish the different types of processes, the process-type function returns the symbol network for a network connection or server, serial for a serial port connection, or real for a real subprocess.

The process-status function returns open, closed, connect, and failed for network connections.  For a network server, the status is always listen.  None of those values is possible for a real subprocess.  See Process Information.

You can stop and resume operation of a network process by calling stop-process and continue-process.  For a server process, being stopped means not accepting new connections.  (Up to 5 connection requests will be queued for when you resume the server; you can increase this limit, unless it is imposed by the operating system.)  For a network stream connection, being stopped means not processing input (any arriving input waits until you resume the connection).  For a datagram connection, some number of packets may be queued but input may be lost.  You can use the function process-command to determine whether a network connection or server is stopped; a non-nil value means yes.

 -- Function: open-network-stream name buffer-or-name host service
     This function opens a TCP connection, and returns a process object that represents the connection.

     The NAME argument specifies the name for the process object.  It is modified as necessary to make it unique.

     The BUFFER-OR-NAME argument is the buffer to associate with the connection.  Output from the connection is inserted in the buffer, unless you specify a filter function to handle the output.  If BUFFER-OR-NAME is nil, it means that the connection is not associated with any buffer.

     The arguments HOST and SERVICE specify where to connect to; HOST is the host name (a string), and SERVICE is the name of a defined network service (a string) or a port number (an integer).

Network Servers
===============

You create a server by calling make-network-process with :server t.  The server will listen for connection requests from clients.  When it accepts a client connection request, that creates a new network connection, itself a process object, with the following parameters:

  * The connection's process name is constructed by concatenating the server process' NAME with a client identification string.  The client identification string for an IPv4 connection looks like <A.B.C.D:P>.  Otherwise, it is a unique number in brackets, as in <NNN>.  The number is unique for each connection in the Emacs session.

  * If the server's filter is non-nil, the connection process does not get a separate process buffer; otherwise, Emacs creates a new buffer for the purpose.  The buffer name is the server's buffer name or process name, concatenated with the client identification string.

     The server's process buffer value is never used directly by Emacs, but it is passed to the log function, which can log connections by inserting text there.

  * The communication type and the process filter and sentinel are inherited from those of the server.  The server never directly uses its filter and sentinel; their sole purpose is to initialize connections made to the server.

  * The connection's process contact info is set according to the client's addressing information (typically an IP address and a port number). This information is associated with the process-contact keywords :host, :service, :remote.

  * The connection's local address is set up according to the port number used for the connection.

  * The client process' plist is initialized from the server's plist.

Datagrams
=========

A datagram connection communicates with individual packets rather than streams of data.  Each call to process-send sends one datagram packet (see Input to Processes), and each datagram received results in one call to the filter function.

The datagram connection doesn't have to talk with the same remote peer all the time.  It has a remote peer address which specifies where to send datagrams to.  Each time an incoming datagram is passed to the filter function, the peer address is set to the address that datagram came from; that way, if the filter function sends a datagram, it will go back to that place.  You can specify the remote peer address when you create the datagram connection using the :remote keyword.  You can change it later on by calling set-process-datagram-address.

 -- Function: process-datagram-address process
     If PROCESS is a datagram connection or server, this function returns its remote peer address.

 -- Function: set-process-datagram-address process address
     If PROCESS is a datagram connection or server, this function sets its remote peer address to ADDRESS.

Low-Level Network Access
========================

You can also create network connections by operating at a lower level than that of open-network-stream, using make-network-process.

make-network-process
--------------------

The basic function for creating network connections and network servers is make-network-process.  It can do either of those jobs, depending on the arguments you give it.

 -- Function: make-network-process &rest args
     This function creates a network connection or server and returns the process object that represents it.  The arguments ARGS are a list of keyword/argument pairs.  Omitting a keyword is always equivalent to specifying it with value nil, except for :coding, :filter-multibyte, and :reuseaddr.  Here are the meaningful keywords:

     :name NAME
          Use the string NAME as the process name.  It is modified if necessary to make it unique.

     :type TYPE
          Specify the communication type.  A value of nil specifies a stream connection (the default); datagram specifies a datagram connection; seqpacket specifies a ``sequenced packet stream'' connection.  Both connections and servers can be of these types.

     :server SERVER-FLAG
          If SERVER-FLAG is non-nil, create a server.  Otherwise, create a connection.  For a stream type server, SERVER-FLAG may be an integer which then specifies the length of the queue of pending connections to the server.  The default queue length is 5.

     :host HOST
          Specify the host to connect to.  HOST should be a host name or Internet address, as a string, or the symbol local to specify the local host.  If you specify HOST for a server, it must specify a valid address for the local host, and only clients connecting to that address will be accepted.

     :service SERVICE
          SERVICE specifies a port number to connect to, or, for a server, the port number to listen on.  It should be a service name that translates to a port number, or an integer specifying the port number directly.  For a server, it can also be t, which means to let the system select an unused port number.

     :family FAMILY
          FAMILY specifies the address (and protocol) family for communication.  nil means determine the proper address family automatically for the given HOST and SERVICE. local specifies a Unix socket, in which case HOST is ignored.  ipv4 and ipv6 specify to use IPv4 and IPv6 respectively.

     :local LOCAL-ADDRESS
          For a server process, LOCAL-ADDRESS is the address to listen on. It overrides FAMILY, HOST and SERVICE, and you may as well not specify them.

     :remote REMOTE-ADDRESS
          For a connection, REMOTE-ADDRESS is the address to connect to. It overrides FAMILY, HOST and SERVICE, and you may as well not specify them.

          For a datagram server, REMOTE-ADDRESS specifies the initial setting of the remote datagram address.

          The format of LOCAL-ADDRESS or REMOTE-ADDRESS depends on the address family:

            - An IPv4 address is represented as a five-element vector of four 8-bit integers and one 16-bit integer [A B C D P] corresponding to numeric IPv4 address A.B.C.D and port number P.

            - An IPv6 address is represented as a nine-element vector of 16-bit integers [A B C D E F G H P] corresponding to numeric IPv6 address A:B:C:D:E:F:G:H and port number P.

            - A local address is represented as a string which specifies the address in the local address space.

            - An ``unsupported family'' address is represented by a cons (F . AV), where F is the family number and AV is a vector specifying the socket address using one element per address data byte.  Do not rely on this format in portable code, as it may depend on implementation defined constants, data sizes, and data structure alignment.

     :nowait BOOL
          If BOOL is non-nil for a stream connection, return without waiting for the connection to complete.  When the connection succeeds or fails, Emacs will call the sentinel function, with a second argument matching "open" (if successful) or "failed".  The default is to block, so that make-network-process does not return until the connection has succeeded or failed.

     :stop STOPPED
          Start the network connection or server in the `stopped' state if STOPPED is non-nil.

     :buffer BUFFER
          Use BUFFER as the process buffer.

     :coding CODING
          Use CODING as the coding system for this process.  To specify different coding systems for decoding data from the connection and for encoding data sent to it, specify (DECODING . ENCODING) for CODING.

          If you don't specify this keyword at all, the default is to determine the coding systems from the data.

     :noquery QUERY-FLAG
          Initialize the process query flag to QUERY-FLAG. See Query Before Exit.

     :filter FILTER
          Initialize the process filter to FILTER.

     :sentinel SENTINEL
          Initialize the process sentinel to SENTINEL.

     :log LOG
          Initialize the log function of a server process to LOG.  The log function is called each time the server accepts a network connection from a client.  The arguments passed to the log function are SERVER, CONNECTION, and MESSAGE, where SERVER is the server process, CONNECTION is the new process for the connection, and MESSAGE is a string describing what has happened.

     :plist PLIST
          Initialize the process plist to PLIST.

     The original argument list, modified with the actual connection information, is available via the process-contact function.

Network Options
---------------

The following network options can be specified when you create a network process.  Except for :reuseaddr, you can also set or modify these options later, using set-network-process-option.

For a server process, the options specified with make-network-process are not inherited by the client connections, so you will need to set the necessary options for each child connection as it is created.

:bindtodevice DEVICE-NAME
     If DEVICE-NAME is a non-empty string identifying a network interface name (see network-interface-list), only handle packets received on that interface.  If DEVICE-NAME is nil (the default), handle packets received on any interface.

     Using this option may require special privileges on some systems.

:broadcast BROADCAST-FLAG
     If BROADCAST-FLAG is non-nil for a datagram process, the process will receive datagram packet sent to a broadcast address, and be able to send packets to a broadcast address.  Ignored for a stream connection.

:dontroute DONTROUTE-FLAG
     If DONTROUTE-FLAG is non-nil, the process can only send to hosts on the same network as the local host.

:keepalive KEEPALIVE-FLAG
     If KEEPALIVE-FLAG is non-nil for a stream connection, enable exchange of low-level keep-alive messages.

:linger LINGER-ARG
     If LINGER-ARG is non-nil, wait for successful transmission of all queued packets on the connection before it is deleted (see delete-process).  If LINGER-ARG is an integer, it specifies the maximum time in seconds to wait for queued packets to be sent before closing the connection.  Default is nil which means to discard unsent queued packets when the process is deleted.

:oobinline OOBINLINE-FLAG
     If OOBINLINE-FLAG is non-nil for a stream connection, receive out-of-band data in the normal data stream.  Otherwise, ignore out-of-band data.

:priority PRIORITY
     Set the priority for packets sent on this connection to the integer PRIORITY.  The interpretation of this number is protocol specific, such as setting the TOS (type of service) field on IP packets sent on this connection.  It may also have system dependent effects, such as selecting a specific output queue on the network interface.

:reuseaddr REUSEADDR-FLAG
     If REUSEADDR-FLAG is non-nil (the default) for a stream server process, allow this server to reuse a specific port number (see :service) unless another process on this host is already listening on that port.  If REUSEADDR-FLAG is nil, there may be a period of time after the last use of that port (by any process on the host), where it is not possible to make a new server on that port.

 -- Function: set-network-process-option process option value &optional no-error
     This function sets or modifies a network option for network process PROCESS.  See make-network-process for details of options OPTION and their corresponding values VALUE.  If NO-ERROR is non-nil, this function returns nil instead of signaling an error if OPTION is not a supported option.  If the function successfully completes, it returns t.

     The current setting of an option is available via the process-contact function.

Testing Availability of Network Features
----------------------------------------

To test for the availability of a given network feature, use featurep like this:

     (featurep 'make-network-process '(KEYWORD VALUE))

The result of the first form is t if it works to specify KEYWORD with value VALUE in make-network-process. The result of the second form is t if KEYWORD is supported by make-network-process.  Here are some of the KEYWORD---VALUE pairs you can test in this way.

(:nowait t)
     Non-nil if non-blocking connect is supported.

(:type datagram)
     Non-nil if datagrams are supported.

(:family local)
     Non-nil if local (a.k.a. ``UNIX domain'') sockets are supported.

(:family ipv6)
     Non-nil if IPv6 is supported.

(:service t)
     Non-nil if the system can select the port for a server.

To test for the availability of a given network option, use featurep like this:

     (featurep 'make-network-process 'KEYWORD)

Here are some of the options you can test in this way.

:bindtodevice
:broadcast
:dontroute
:keepalive
:linger
:oobinline
:priority
:reuseaddr
     That particular network option is supported by make-network-process and set-network-process-option.

Misc Network Facilities
=======================

These additional functions are useful for creating and operating on network connections.  Note that they are supported only on some systems.

 -- Function: network-interface-list
     This function returns a list describing the network interfaces of the machine you are using.  The value is an alist whose elements have the form (NAME . ADDRESS). ADDRESS has the same form as the LOCAL-ADDRESS and REMOTE-ADDRESS arguments to make-network-process.

 -- Function: network-interface-info ifname
     This function returns information about the network interface named IFNAME.  The value is a list of the form (ADDR BCAST NETMASK HWADDR FLAGS).

     addr
          The Internet protocol address.

     bcast
          The broadcast address.

     netmask
          The network mask.

     hwaddr
          The layer 2 address (Ethernet MAC address, for instance).

     flags
          The current flags of the interface.

 -- Function: format-network-address address &optional omit-port
     This function converts the Lisp representation of a network address to a string.

     A five-element vector [A B C D P] represents an IPv4 address A.B.C.D and port number P.  format-network-address converts that to the string "A.B.C.D:P".

     A nine-element vector [A B C D E F G H P] represents an IPv6 address along with a port number.  format-network-address converts that to the string "[A:B:C:D:E:F:G:H]:P".

     If the vector does not include the port number, P, or if OMIT-PORT is non-nil, the result does not include the :P suffix.

Communicating with Serial Ports
===============================

Emacs can communicate with serial ports.  For interactive use, M-x serial-term opens a terminal window.  In a Lisp program, make-serial-process creates a process object.

The serial port can be configured at run-time, without having to close and re-open it.  The function serial-process-configure lets you change the speed, bytesize, and other parameters.  In a terminal window created by serial-term, you can click on the mode line for configuration.

A serial connection is represented by a process object which can be used similar to a subprocess or network process.  You can send and receive data and configure the serial port.  A serial process object has no process ID, you can't send signals to it, and the status codes are different from other types of processes. delete-process on the process object or kill-buffer on the process buffer close the connection, but this does not affect the device connected to the serial port.

The function process-type returns the symbol serial for a process object representing a serial port connection.

Serial ports are available on GNU/Linux, Unix, and Windows systems.

 -- Command: serial-term port speed
     Start a terminal-emulator for a serial port in a new buffer. PORT is the name of the serial port to which to connect.  For example, this could be /dev/ttyS0 on Unix.  On Windows, this could be COM1, or \\.\COM10 (double the backslashes in Lisp strings).

     SPEED is the speed of the serial port in bits per second.  9600 is a common value.  The buffer is in Term mode; see Term Mode (The GNU Emacs Manual), for the commands to use in that buffer. You can change the speed and the configuration in the mode line menu.

 -- Function: make-serial-process &rest args
     This function creates a process and a buffer.  Arguments are specified as keyword/argument pairs.  Here's the list of the meaningful keywords:

     :port PORT (mandatory)
          This is the name of the serial port.  On Unix and GNU systems, this is a file name such as /dev/ttyS0.  On Windows, this could be COM1, or \\.\COM10 for ports higher than COM9 (double the backslashes in Lisp strings).

     :speed SPEED (mandatory)
          The speed of the serial port in bits per second.  This function calls serial-process-configure to handle the speed.

     :name NAME
          The name of the process.  If NAME is not given, PORT will serve as the process name as well.

     :buffer BUFFER
          The buffer to associate with the process.  The value could be either a buffer or a string that names a buffer.  Process output goes at the end of that buffer, unless you specify an output stream or filter function to handle the output.  If BUFFER is not given, the process buffer's name is taken from the value of the :name keyword.

     :coding CODING
          If CODING is a symbol, it specifies the coding system used for both reading and writing for this process.  If CODING is a cons (decoding . encoding), DECODING is used for reading, and ENCODING is used for writing.  If not specified, the default is to determine the coding systems from data itself.

     :noquery QUERY-FLAG
          Initialize the process query flag to QUERY-FLAG.  See Query Before Exit.  The flags defaults to nil if unspecified.

     :stop BOOL
          Start process in the stopped state if BOOL is non-nil.  In the stopped state, a serial process does not accept incoming data, but you can send outgoing data.  The stopped state is cleared by continue-process and set by stop-process.

     :filter FILTER
          Install FILTER as the process filter.

     :sentinel SENTINEL
          Install SENTINEL as the process sentinel.

     :plist PLIST
          Install PLIST as the initial plist of the process.

     :speed
     :bytesize
     :parity
     :stopbits
     :flowcontrol
          These arguments are handled by serial-process-configure, which is called by make-serial-process.

     The original argument list, possibly modified by later configuration, is available via the function process-contact.

     Examples:

          (make-serial-process :port "/dev/ttyS0" :speed 9600)

          (make-serial-process :port "COM1" :speed 115200 :stopbits 2)

          (make-serial-process :port "\\\\.\\COM13" :speed 1200
                               :bytesize 7 :parity 'odd)

          (make-serial-process :port "/dev/tty.BlueConsole-SPP-1"
                               :speed nil)

 -- Function: serial-process-configure &rest args

     This functions configures a serial port connection.  Arguments are specified as keyword/argument pairs.  Attributes that are not given are re-initialized from the process's current configuration (available via the function process-contact) or set to reasonable default values.  The following arguments are defined:

     :process PROCESS
     :name NAME
     :buffer BUFFER
     :port PORT
          Any of these arguments can be given to identify the process that is to be configured.  If none of these arguments is given, the current buffer's process is used.

     :speed SPEED
          The speed of the serial port in bits per second, a.k.a. baud rate.  The value can be any number, but most serial ports work only at a few defined values between 1200 and 115200, with 9600 being the most common value.  If SPEED is nil, the function ignores all other arguments and does not configure the port.  This may be useful for special serial ports such as Bluetooth-to-serial converters which can only be configured through AT commands sent through the connection.  The value of nil for SPEED is valid only for connections that were already opened by a previous call to make-serial-process or serial-term.

     :bytesize BYTESIZE
          The number of bits per byte, which can be 7 or 8.  If BYTESIZE is not given or nil, it defaults to 8.

     :parity PARITY
          The value can be nil (don't use parity), the symbol odd (use odd parity), or the symbol even (use even parity).  If PARITY is not given, it defaults to no parity.

     :stopbits STOPBITS
          The number of stopbits used to terminate a transmission of each byte.  STOPBITS can be 1 or 2.  If STOPBITS is not given or nil, it defaults to 1.

     :flowcontrol FLOWCONTROL
          The type of flow control to use for this connection, which is either nil (don't use flow control), the symbol hw (use RTS/CTS hardware flow control), or the symbol sw (use XON/XOFF software flow control).  If FLOWCONTROL is not given, it defaults to no flow control.

     serial-process-configure is called by make-serial-process for the initial configuration of the serial port.

     Examples:

          (serial-process-configure :process "/dev/ttyS0" :speed 1200)

          (serial-process-configure :buffer "COM1" :stopbits 1
                                    :parity 'odd :flowcontrol 'hw)

          (serial-process-configure :port "\\\\.\\COM13" :bytesize 7)

Packing and Unpacking Byte Arrays
=================================

This section describes how to pack and unpack arrays of bytes, usually for binary network protocols.  These functions convert byte arrays to alists, and vice versa.  The byte array can be represented as a unibyte string or as a vector of integers, while the alist associates symbols either with fixed-size objects or with recursive sub-alists.

Conversion from byte arrays to nested alists is also known as deserializing or unpacking, while going in the opposite direction is also known as serializing or packing.

Describing Data Layout
----------------------

To control unpacking and packing, you write a data layout specification, a special nested list describing named and typed fields.  This specification controls length of each field to be processed, and how to pack or unpack it.  We normally keep bindat specs in variables whose names end in -bindat-spec; that kind of name is automatically recognized as ``risky.''

A field's type describes the size (in bytes) of the object that the field represents and, in the case of multibyte fields, how the bytes are ordered within the field.  The two possible orderings are ``big endian'' (also known as ``network byte ordering'') and ``little endian.''  For instance, the number #x23cd (decimal 9165) in big endian would be the two bytes #x23 #xcd; and in little endian, #xcd #x23.  Here are the possible type values:

u8
byte
     Unsigned byte, with length 1.

u16
word
short
     Unsigned integer in network byte order, with length 2.

u24
     Unsigned integer in network byte order, with length 3.

u32
dword
long
     Unsigned integer in network byte order, with length 4. Note: These values may be limited by Emacs' integer implementation limits.

u16r
u24r
u32r
     Unsigned integer in little endian order, with length 2, 3 and 4, respectively.

str LEN
     String of length LEN.

strz LEN
     Zero-terminated string, in a fixed-size field with length LEN.

vec LEN [TYPE]
     Vector of LEN elements of type TYPE, or bytes if not TYPE is specified. The TYPE is any of the simple types above, or another vector specified as a list (vec LEN [TYPE]).

ip
     Four-byte vector representing an Internet address.  For example: [127 0 0 1] for localhost.

bits LEN
     List of set bits in LEN bytes.  The bytes are taken in big endian order and the bits are numbered starting with 8 * LEN - 1 and ending with zero.  For example: bits 2 unpacks #x28 #x1c to (2 3 4 11 13) and #x1c #x28 to (3 5 10 11 12).

(eval FORM)
     FORM is a Lisp expression evaluated at the moment the field is unpacked or packed.  The result of the evaluation should be one of the above-listed type specifications.

For a fixed-size field, the length LEN is given as an integer specifying the number of bytes in the field.

When the length of a field is not fixed, it typically depends on the value of a preceding field.  In this case, the length LEN can be given either as a list (NAME ...) identifying a field name in the format specified for bindat-get-field below, or by an expression (eval FORM) where FORM should evaluate to an integer, specifying the field length.

A field specification generally has the form ([NAME] HANDLER).  The square braces indicate that NAME is optional.  (Don't use names that are symbols meaningful as type specifications (above) or handler specifications (below), since that would be ambiguous.)  NAME can be a symbol or the expression (eval FORM), in which case FORM should evaluate to a symbol.

HANDLER describes how to unpack or pack the field and can be one of the following:

TYPE
     Unpack/pack this field according to the type specification TYPE.

eval FORM
     Evaluate FORM, a Lisp expression, for side-effect only.  If the field name is specified, the value is bound to that field name.

fill LEN
     Skip LEN bytes.  In packing, this leaves them unchanged, which normally means they remain zero.  In unpacking, this means they are ignored.

align LEN
     Skip to the next multiple of LEN bytes.

struct SPEC-NAME
     Process SPEC-NAME as a sub-specification.  This describes a structure nested within another structure.

union FORM (TAG SPEC)...
     Evaluate FORM, a Lisp expression, find the first TAG that matches it, and process its associated data layout specification SPEC.  Matching can occur in one of three ways:

       * If a TAG has the form (eval EXPR), evaluate EXPR with the variable tag dynamically bound to the value of FORM.  A non-nil result indicates a match.

       * TAG matches if it is equal to the value of FORM.

       * TAG matches unconditionally if it is t.

repeat COUNT FIELD-SPECS...
     Process the FIELD-SPECS recursively, in order, then repeat starting from the first one, processing all the specs COUNT times overall.  The COUNT is given using the same formats as a field length---if an eval form is used, it is evaluated just once. For correct operation, each spec in FIELD-SPECS must include a name.

For the (eval FORM) forms used in a bindat specification, the FORM can access and update these dynamically bound variables during evaluation:

last
     Value of the last field processed.

bindat-raw
     The data as a byte array.

bindat-idx
     Current index (within bindat-raw) for unpacking or packing.

struct
     The alist containing the structured data that have been unpacked so far, or the entire structure being packed.  You can use bindat-get-field to access specific fields of this structure.

count
index
     Inside a repeat block, these contain the maximum number of repetitions (as specified by the COUNT parameter), and the current repetition number (counting from 0).  Setting count to zero will terminate the inner-most repeat block after the current repetition has completed.

Functions to Unpack and Pack Bytes
----------------------------------

In the following documentation, SPEC refers to a data layout specification, bindat-raw to a byte array, and STRUCT to an alist representing unpacked field data.

 -- Function: bindat-unpack spec bindat-raw &optional bindat-idx
     This function unpacks data from the unibyte string or byte array bindat-raw according to SPEC.  Normally this starts unpacking at the beginning of the byte array, but if BINDAT-IDX is non-nil, it specifies a zero-based starting position to use instead.

     The value is an alist or nested alist in which each element describes one unpacked field.

 -- Function: bindat-get-field struct &rest name
     This function selects a field's data from the nested alist STRUCT.  Usually STRUCT was returned by bindat-unpack.  If NAME corresponds to just one argument, that means to extract a top-level field value.  Multiple NAME arguments specify repeated lookup of sub-structures.  An integer name acts as an array index.

     For example, if NAME is (a b 2 c), that means to find field c in the third element of subfield b of field a.  (This corresponds to struct.a.b[2].c in C.)

Although packing and unpacking operations change the organization of data (in memory), they preserve the data's total length, which is the sum of all the fields' lengths, in bytes.  This value is not generally inherent in either the specification or alist alone; instead, both pieces of information contribute to its calculation.  Likewise, the length of a string or array being unpacked may be longer than the data's total length as described by the specification.

 -- Function: bindat-length spec struct
     This function returns the total length of the data in STRUCT, according to SPEC.

 -- Function: bindat-pack spec struct &optional bindat-raw bindat-idx
     This function returns a byte array packed according to SPEC from the data in the alist STRUCT.  Normally it creates and fills a new byte array starting at the beginning.  However, if BINDAT-RAW is non-nil, it specifies a pre-allocated unibyte string or vector to pack into.  If BINDAT-IDX is non-nil, it specifies the starting offset for packing into bindat-raw.

     When pre-allocating, you should make sure (length BINDAT-RAW) meets or exceeds the total length to avoid an out-of-range error.

 -- Function: bindat-ip-to-string ip
     Convert the Internet address vector IP to a string in the usual dotted notation.

          (bindat-ip-to-string [127 0 0 1])
               => "127.0.0.1"

Examples of Byte Unpacking and Packing
--------------------------------------

Here is a complete example of byte unpacking and packing:

     (defvar fcookie-index-spec
       '((:version  u32)
         (:count    u32)
         (:longest  u32)
         (:shortest u32)
         (:flags    u32)
         (:delim    u8)
         (:ignored  fill 3)
         (:offset   repeat (:count)
                    (:foo u32)))
       "Description of a fortune cookie index file's contents.")

     (defun fcookie (cookies &optional index)
       "Display a random fortune cookie from file COOKIES.
     Optional second arg INDEX specifies the associated index
     filename, which is by default constructed by appending
     \".dat\" to COOKIES.  Display cookie text in possibly
     new buffer \"*Fortune Cookie: BASENAME*\" where BASENAME
     is COOKIES without the directory part."
       (interactive "fCookies file: ")
       (let* ((info (with-temp-buffer
                      (insert-file-contents-literally
                       (or index (concat cookies ".dat")))
                      (bindat-unpack fcookie-index-spec
                                     (buffer-string))))
              (sel (random (bindat-get-field info :count)))
              (beg (cdar (bindat-get-field info :offset sel)))
              (end (or (cdar (bindat-get-field info
                                               :offset (1+ sel)))
                       (nth 7 (file-attributes cookies)))))
         (switch-to-buffer
          (get-buffer-create
           (format "*Fortune Cookie: %s*"
                   (file-name-nondirectory cookies))))
         (erase-buffer)
         (insert-file-contents-literally
          cookies nil beg (- end 3))))

     (defun fcookie-create-index (cookies &optional index delim)
       "Scan file COOKIES, and write out its index file.
     Optional second arg INDEX specifies the index filename,
     which is by default constructed by appending \".dat\" to
     COOKIES.  Optional third arg DELIM specifies the unibyte
     character which, when found on a line of its own in
     COOKIES, indicates the border between entries."
       (interactive "fCookies file: ")
       (setq delim (or delim ?%))
       (let ((delim-line (format "\n%c\n" delim))
             (count 0)
             (max 0)
             min p q len offsets)
         (unless (= 3 (string-bytes delim-line))
           (error "Delimiter cannot be represented in one byte"))
         (with-temp-buffer
           (insert-file-contents-literally cookies)
           (while (and (setq p (point))
                       (search-forward delim-line (point-max) t)
                       (setq len (- (point) 3 p)))
             (setq count (1+ count)
                   max (max max len)
                   min (min (or min max) len)
                   offsets (cons (1- p) offsets))))
         (with-temp-buffer
           (set-buffer-multibyte nil)
           (insert
            (bindat-pack
             fcookie-index-spec
             `((:version . 2)
               (:count . ,count)
               (:longest . ,max)
               (:shortest . ,min)
               (:flags . 0)
               (:delim . ,delim)
               (:offset . ,(mapcar (lambda (o)
                                     (list (cons :foo o)))
                                   (nreverse offsets))))))
           (let ((coding-system-for-write 'raw-text-unix))
             (write-file (or index (concat cookies ".dat")))))))

Following is an example of defining and unpacking a complex structure. Consider the following C structures:

     struct header {
         unsigned long    dest_ip;
         unsigned long    src_ip;
         unsigned short   dest_port;
         unsigned short   src_port;
     };

     struct data {
         unsigned char    type;
         unsigned char    opcode;
         unsigned short   length;  /* In network byte order */
         unsigned char    id[8];   /* null-terminated string  */
         unsigned char    data[/* (length + 3) & ~3 */];
     };

     struct packet {
         struct header    header;
         unsigned long    counters[2];  /* In little endian order */
         unsigned char    items;
         unsigned char    filler[3];
         struct data      item[/* items */];

     };

The corresponding data layout specification:

     (setq header-spec
           '((dest-ip   ip)
             (src-ip    ip)
             (dest-port u16)
             (src-port  u16)))

     (setq data-spec
           '((type      u8)
             (opcode    u8)
             (length    u16)  ;; network byte order
             (id        strz 8)
             (data      vec (length))
             (align     4)))

     (setq packet-spec
           '((header    struct header-spec)
             (counters  vec 2 u32r)   ;; little endian order
             (items     u8)
             (fill      3)
             (item      repeat (items)
                        (struct data-spec))))

A binary data representation:

     (setq binary-data
           [ 192 168 1 100 192 168 1 101 01 28 21 32
             160 134 1 0 5 1 0 0 2 0 0 0
             2 3 0 5 ?A ?B ?C ?D ?E ?F 0 0 1 2 3 4 5 0 0 0
             1 4 0 7 ?B ?C ?D ?E ?F ?G 0 0 6 7 8 9 10 11 12 0 ])

The corresponding decoded structure:

     (setq decoded (bindat-unpack packet-spec binary-data))
          =>
     ((header
       (dest-ip   . [192 168 1 100])
       (src-ip    . [192 168 1 101])
       (dest-port . 284)
       (src-port  . 5408))
      (counters . [100000 261])
      (items . 2)
      (item ((data . [1 2 3 4 5])
             (id . "ABCDEF")
             (length . 5)
             (opcode . 3)
             (type . 2))
            ((data . [6 7 8 9 10 11 12])
             (id . "BCDEFG")
             (length . 7)
             (opcode . 4)
             (type . 1))))

Fetching data from this structure:

     (bindat-get-field decoded 'item 1 'id)
          => "BCDEFG"