GDP

<?xml version="1.0" encoding="US-ASCII"?>

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"

"http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd">

<article class="techreport">

  <articleinfo>

    <title>Notes on "EP" Library</title>

    <author>

      <firstname>Eric</firstname>

      <surname>Allman</surname>

    </author>

    <copyright>

      <year>2008, 2014&ndash;2018 Eric P. Allman. All rights reserved.</year>

    </copyright>

    <date>2018-08-15</date>

  </articleinfo>

  <section>

    <title>INTRODUCTION</title>

    <para>This document describes the Enhanced Portability library. This is a

    reduced version of Eric's Portability library (designed for sendmail),

    removing many things that didn't work out or proved unnecessary, e.g., the

    entire I/O subsystem. It was originally intended to be used in sendmail,

    so some of the terminology is geared toward the email world; none the

    less, it should be generally useful.</para>

    <para>Programs using <symbol>libep</symbol> should be able to minimize

    <symbol>#ifdef</symbol> since often non-portable functionality is wrapped

    in portability routines; for example, BSD-derived systems and

    Linux-derived systems differ in how you get the name of the currently

    running program. These differences are hidden if the application calls

    <function>ep_app_getprogname</function>.</para>

    <section>

      <title>Design Goals</title>

      <itemizedlist>

        <listitem>

          <para>Portable, to the extent possible. Where not possible, there

          needs to be a clearly codified way to represent the externally

          visible semantic differences.</para>

        </listitem>

        <listitem>

          <para>Efficient.</para>

        </listitem>

        <listitem>

          <para>Customizable -- try to implement mechanism, not policy.</para>

        </listitem>

        <listitem>

          <para>TBD: It should be entirely UTF-8 internally. Any translations

          to other character sets should be done on input or output, and then

          only as strictly necessary.</para>

        </listitem>

      </itemizedlist>

    </section>

    <section>

      <title>Assumptions</title>

      <para>Use of this library requires that you have a C compiler that is

      compliant with ANSI C as defined by ANSI/ISO 9899-1999. Also requires an

      environment that is at least Posix based on Posix.1-2008.</para>

    </section>

    <section>

      <title>Conventions</title>

      <para><itemizedlist>

          <listitem>

            <para>All externally visible names (i.e., those not declared

            "static" in a file) shall be named ep_* (for routine names) or Ep*

            (for variable names). In a few cases, the names may begin with

            __ep or __Ep; such names would be in the global namespace, but

            would be intended for use internal to the library only. There are

            a few cases where "standard" names (such as

            <function>strlcpy</function>) are defined if they are not included

            in the standard library.</para>

          </listitem>

        </itemizedlist></para>

    </section>

    <section>

      <title>Terminology</title>

      <section>

        <title>Warning, Error, Severe, Abort</title>

        <para>These words get used fairly loosely, so they are worth defining.

        In the context of libep:</para>

        <itemizedlist>

          <listitem>

            <para>Warning means a condition that is expected in normal

            operations, but is not the usual case. Reading an end-of-file on a

            file might be a warning. Applications need to be aware of these,

            but are expected to either ignore them or recover easily. This can

            also be used for temporary errors which are likely to recover

            after a delay. For example, the inability to open a connection to

            a remote server might recover automatically if that server is

            re-started. However, this sort of warnings that persist should

            turn into permanent errors in a fashion appropriate for the

            application.</para>

          </listitem>

          <listitem>

            <para>Error means a situation that should not occur, but isn't

            terribly unusual. For example, an attempt to open a file that

            isn't accessible would be an error. Applications must be aware of

            such conditions and handle them gracefully.</para>

          </listitem>

          <listitem>

            <para>Severe means a situation that should not occur iand requires

            exceptional handling. Severe errors are drastic conditions, but

            are not so severe that the application can't take some reasonable

            backout action.</para>

          </listitem>

          <listitem>

            <para>Abort means a situation so drastic that an application

            cannot be expected to make any reasonable recovery. These might

            include assertion errors and memory allocation failures during a

            critical step (e.g., something where backing out a single thread

            won't solve the problem). About the only thing an application can

            reasonably do is log an abort error and exit. In particular, an

            abort is appropriate when an attempt for an application to recover

            is likely to do additional damage. These should be extremely

            rare.</para>

          </listitem>

        </itemizedlist>

      </section>

    </section>

  </section>

  <section>

    <title>GENERAL ISSUES</title>

    <para>All files using this library must use "<code>#include

    &lt;ep/ep.h&gt;</code>".</para>

  </section>

  <section>

    <title>STATUS CODES</title>

    <para>Almost all functions return an <type>EP_STAT</type> value. This is a

    short (integer-encoded) status value that gives you a brief idea of how

    severe the problem was and some idea of what it was, but not much else.

    Think of it as an errno equivalent. Functions returning any status other

    than OK are expected to provide some other way of returning detailed

    data.</para>

    <para><type>EP_STAT</type>s are also used as message identifiers for

    logging (below).</para>

    <para>Status codes are defined in

    <filename>&lt;ep/ep_stat.h&gt;</filename>.</para>

    <section>

      <title>Severities</title>

      <para>Severities are:</para>

      <variablelist termlength="20">

        <varlistentry>

          <term><errortype>EP_STAT_SEV_OK</errortype></term>

          <listitem>

            <para>Everything is fine. Detail may contain info. For messages,

            can be used for debugging.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><errortype>EP_STAT_SEV_WARN</errortype></term>

          <listitem>

            <para>The function partially succeeded, but there is something

            that the application should be aware of, e.g., an end of file or a

            short data read. Alternatively, the functionfailed, but it might

            work again on a later try.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><errortype>EP_STAT_SEV_ERROR</errortype></term>

          <listitem>

            <para>A normal error status. The call failed.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><errortype>EP_STAT_SEV_SEVERE</errortype></term>

          <listitem>

            <para>A severe error status. The call failed, and the caller

            should try to back out.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><errortype>EP_STAT_SEV_ABORT</errortype></term>

          <listitem>

            <para>A critical error occured &mdash; you should clean up and

            exit as soon as possible; the program cannot be expected to

            operate correctly.</para>

          </listitem>

        </varlistentry>

      </variablelist>

      <para>Some functions for testing values:</para>

      <variablelist>

        <varlistentry>

          <term><function>EP_STAT_SEV_ISOK</function>(st)</term>

          <listitem>

            <para>Returns true if this is an

            <errortype>EP_STAT_SEV_OK</errortype> status code.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_SEV_WARN</function>(st)</term>

          <listitem>

            <para>Returns true if this is an "warning" severity status code:

            <errortype>EP_STAT_SEV_WARN</errortype>.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_SEV_ISERROR</function>(st)</term>

          <listitem>

            <para>Returns true if this is an "error" severity status code:

            <errortype>EP_STAT_SEV_ERROR</errortype></para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_SEV_ISFAIL</function>(st)</term>

          <listitem>

            <para>Returns true if this message is a "failure" severity status

            code: <errortype>EP_STAT_SEV_ERROR</errortype> or higher.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_SEV_ISSEVERE</function>(st)</term>

          <listitem>

            <para>Returns true if this is an "severe" severity status code:

            <errortype>EP_STAT_SEV_SEVERE</errortype></para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_SEV_ISSFAIL</function>(st)</term>

          <listitem>

            <para>Returns true if this message is a "major" severity status

            code: <errortype>EP_STAT_SEV_SEVERE</errortype> or higher</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_SEV_ISABORT</function>(st)</term>

          <listitem>

            <para>Returns true if this is an "abort" severity status code:

            <errortype>EP_STAT_SEV_ABORT</errortype></para>

          </listitem>

        </varlistentry>

      </variablelist>

    </section>

    <section>

      <title>Status Code Representation</title>

      <para>Status codes are represented as four-part values: severity,

      registry, module, and detail. The severities are described above.

      Registries are globally registered by neophilic.com and are defined in

      ep_registry.h. There are some registries for general use; in particular,

      registry numbers between 0x001 and 0x1FF are available for local

      (non-global) registry at the corporate or local level. Modules are

      defined by registries, and detail is defined by module. It is

      <emphasis>never</emphasis> acceptable to look at detail unless you

      recognize the module. (OK, you can print it out for debugging.) Severity

      = 3 bits, registry = 13 bits, module = 6 bits, detail = 10 bits.</para>

      <para>Any severity where the top bit is zero is considered "OK", and the

      rest of the word is available to encode a non-negative integer.</para>

      <para>Status codes are represented as structures to ensure type safety.

      Occassionally you might want to convert a status to or from a long

      int:</para>

      <programlisting>int      EP_STAT_TO_INT(EP_STAT stat)          // convert status to unsigned int

EP_STAT  EP_STAT_FROM_INT(unsigned int istat)  // convert unsigned integer to status</programlisting>

      <para>The constituent parts of the status code can also be

      extracted:</para>

      <variablelist>

        <varlistentry>

          <term><function>EP_STAT_SEV</function>(st)</term>

          <listitem>

            <para>Returns the severity part of the status code.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_REGISTRY</function>(st)</term>

          <listitem>

            <para>Returns the registry part of the status code.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_MODULE</function>(st)</term>

          <listitem>

            <para>Returns the module part of the status code.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><function>EP_STAT_DETAIL</function>(st)</term>

          <listitem>

            <para>Returns the detail part of the status code.</para>

          </listitem>

        </varlistentry>

      </variablelist>

      <para>To compare two statuses for equality, use

      <function>EP_STAT_IS_SAME</function>(a, b).</para>

      <para>As a special case, if the severity is

      <errortype>EP_STAT_SEV_OK</errortype> the rest of the word is ignored;

      this can be used to pass small integers (no more than 31 bits) of

      information.</para>

    </section>

    <section>

      <title>Predefined Status Codes</title>

      <para>All status codes from this library are in the

      <constant>EP_REGISTRY_EPLIB</constant> registery. There are several

      predefined status codes for generic use, all using module

      <constant>EP_STAT_MOD_GENERIC</constant>:</para>

      <informaltable border="1">

        <col width="4*"/>

        <col width="6*"/>

        <tr>

          <td>EP_STAT_OK</td>

          <td>No error (also integer 0)</td>

        </tr>

        <tr>

          <td>EP_STAT_WARN</td>

          <td>Generic warning status</td>

        </tr>

        <tr>

          <td>EP_STAT_ERROR</td>

          <td>Generic error status</td>

        </tr>

        <tr>

          <td>EP_STAT_SEVERE</td>

          <td>Generic severe error status</td>

        </tr>

        <tr>

          <td>EP_STAT_ABORT</td>

          <td>Generic abortive status</td>

        </tr>

        <tr>

          <td>EP_STAT_OUT_OF_MEMORY</td>

          <td>Out of memory</td>

        </tr>

        <tr>

          <td>EP_STAT_ARG_OUT_OF_RANGE</td>

          <td>An argument was out of range</td>

        </tr>

        <tr>

          <td>EP_STAT_END_OF_FILE</td>

          <td>End of input</td>

        </tr>

        <tr>

          <td>EP_STAT_TIME_BADFORMAT</td>

          <td>Couldn't parse a date/time string</td>

        </tr>

        <tr>

          <td>EP_STAT_BUF_OVERFLOW</td>

          <td>Buffer overflow averted</td>

        </tr>

        <tr>

          <td>EP_STAT_ASSERT_ABORT</td>

          <td>Assertion failiure: backout now!</td>

        </tr>

      </informaltable>

      <para>There is also a special module

      <constant>EP_STAT_MOD_ERRNO</constant> that encodes Posix-style errnos

      (i.e., use <function>EP_STAT_DETAIL</function> on codes returned by that

      module to get the Posix errno code).</para>

    </section>

    <section>

      <title>Manipulating Status Codes</title>

      <para>There are several routines to print error codes or create them on

      the fly. Note that <function>ep_stat_tostr</function> returns the buffer

      itself.</para>

      <programlisting>// create status code from UNIX errno

extern EP_STAT  ep_stat_from_errno(

                        int uerrno);

// return string representation of status

char            *ep_stat_tostr(

                        EP_STAT estat,

                        char *buf,

                        size_t bsize);

// return string representation of severity (in natural language)

const char      *ep_stat_sev_tostr(

                        int sev);

// print a status code and abort (never returns)

void            ep_stat_abort(

                        EP_STAT estat);</programlisting>

    </section>

    <section>

      <title>Creating New Status Codes</title>

      <para>Libraries and applications can create their own specific error

      codes. There are four steps to do this:</para>

      <orderedlist>

        <listitem>

          <para>Determine the registry. The registry name space is divided as

          follows:</para>

          <informaltable border="1">

            <col width="4*"/>

            <col width="6*"/>

            <tr>

              <td>0x000 (<constant>EP_REGISTRY_GENERIC</constant>)</td>

              <td>reserved for generic status codes</td>

            </tr>

            <tr>

              <td>0x001 (<constant>EP_REGISTRY_USER</constant>)</td>

              <td>available for internal use to an application</td>

            </tr>

            <tr>

              <td>0x002&ndash;0x07F</td>

              <td>available for local, unregistered use, such as separate

              applications within an application suite</td>

            </tr>

            <tr>

              <td>0x080&ndash;0x0FF</td>

              <td>available for internal corporate registry, but not

              registered globally; conflicts may occur between organizations

              but not within an organization</td>

            </tr>

            <tr>

              <td>0x100 (<constant>EP_REGISTRY_EPLIB</constant>)</td>

              <td>reserved for libep</td>

            </tr>

            <tr>

              <td>0x101&ndash;0x6FF</td>

              <td>available for centrally managed global registry &mdash;

              contact the libep maintainers for an allocation</td>

            </tr>

            <tr>

              <td>0x700&ndash;0x7FF</td>

              <td>reserved</td>

            </tr>

          </informaltable>

        </listitem>

        <listitem>

          <para>Determine the module(s) in which the error code should exist.

          These must be unique within a registry and in the range

          0x00&ndash;0xFF.</para>

        </listitem>

        <listitem>

          <para>Define the error codes you want to use using

          <function>EP_STAT_NEW</function>, e.g.,</para>

          <programlisting>#define FOO_STAT_ELEPHANT    EP_STAT_NEW(ERROR, EP_REGISTRY_FOO, MOD_BAR, 1)

#define FOO_STAT_GIRAFFE     EP_STAT_NEW(SEVERE, EP_REGISTRY_FOO, MOD_BAR, 2)</programlisting>

        </listitem>

        <listitem>

          <para>(Optional step) Define the strings associated with the error

          codes when they are printed. These are done by populating a table

          and then calling <function>ep_stat_register_strings</function>. For

          example:</para>

          <programlisting>struct ep_stat_reg_strings  FooStatusCodes[] =

{

    FOO_STAT_ELEPHANT,    "elephant in the room",        },

    FOO_STAT_GIRAFFE,     "too tall",                    },

    EP_STAT_OK,           NULL,                          }

}

ep_stat_reg_strings(FooStatusCodes);</programlisting>

        </listitem>

      </orderedlist>

    </section>

  </section>

  <section>

    <title>INITIALIZATION</title>

    <para>Although libep will generally work without initialization, in some

    cases you may need to give it information about your usage. To do this

    call <function>ep_lib_init</function>:</para>

    <programlisting>#include &lt;ep/ep.h&gt;

EP_STAT

ep_lib_init(uint32_t flags)</programlisting>

    <para>Flags can be:</para>

    <informaltable>

      <tgroup cols="2">

        <tbody>

          <row>

            <entry>EP_LIB_USEPTHREADS</entry>

            <entry>Initialize the thread support</entry>

          </row>

        </tbody>

      </tgroup>

    </informaltable>

  </section>

  <section>

    <title>MEMORY ALLOCATION AND RESOURCE POOLS</title>

    <section>

      <title>Memory</title>

      <para>Memory support is much like malloc/free, but with some additional

      functionality. One crucial difference is that most of these routines do

      not return if memory is exhausted; instead they can call a cleanup

      routine that might (for example) eliminate some old cache entries, or

      pick a "victim" thread to kill and reclaim its memory. If successful

      they can continue, otherwise the process is aborted.</para>

      <programlisting>    #include &lt;ep/ep_mem.h&gt;

    void *

    ep_mem_malloc(size_t nbytes)       // allocate uninitialized memory

    void *

    ep_mem_zalloc(size_t nbytes)       // allocate zeroed memory

    void *

    ep_mem_ralloc(size_t nbytes)       // allocate randomized memory

    void *

    ep_mem_ealloc(size_t nbytes)       // allocate memory, failure OK

    void *

    ep_mem_realloc(size_t nbytes,      // reallocate (extend) memory

                void *curmem)

    void *

    ep_mem_falloc(size_t nbytes,       // allocate memory (see flags)

                uint32_t flags)

    void

    ep_mem_mfree(void *mem)            // free indicated memory

    struct ep_malloc_functions

    {

        void    *(*m_malloc)(size_t);

        void    *(*m_realloc)(void*, size_t);

        void    *(*m_valloc)(size_t);

        void    (*m_free)(void*);

    };

    void

    ep_mem_set_malloc_functions(       // set underlying malloc functions

                struct ep_malloc_functions *funcs)</programlisting>

      <para>The <function>ep_mem_malloc</function>,

      <function>ep_mem_zalloc</function>, <function>ep_mem_ralloc</function>,

      and <function>ep_mem_realloc</function> are all implemented in terms of

      <function>ep_mem_falloc</function>, which uses flags to tune the

      behavior (see below). The primary interface is

      <function>ep_mem_malloc</function>, which returns uninitialized data;

      <function>ep_mem_zalloc</function> returns zeroed memory, and

      <function>ep_mem_ralloc</function> returns memory that is initialized to

      random or some other nonsensical data. The last would probably be used

      only for debugging, and can be turned on at runtime using a debug flag

      <remark>XXX TBD</remark>.</para>

      <para>In all allocation schemes, the function returns a pointer to the

      allocated data — they cannot normally return

      <constant>NULL</constant> (but see below). If they cannot allocate the

      memory, they <remark>do error recovery (XXX describe)</remark>. If

      recovery fails, the allocation system will abort the process. However,

      <function>ep_mem_ealloc</function> can return <constant>NULL</constant>

      on memory allocation failure, as can <function>ep_mem_falloc</function>

      if the <constant>EP_MEM_F_FAILOK</constant> flag bit is set (see

      below).</para>

      <para>Flag bits are as follows:</para>

      <variablelist>

        <varlistentry>

          <term><constant>EP_MEM_F_FAILOK</constant></term>

          <listitem>

            <para>Permits the routine to return <constant>NULL</constant> on

            failure. This modifies the behavior described above. Note that if

            this is set every call to the <function>ep_*malloc</function>

            routines may potentially fail.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><constant>EP_MEM_F_ZERO</constant></term>

          <listitem>

            <para>Zero any returned memory.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><constant>EP_MEM_F_TRASH</constant></term>

          <listitem>

            <para>Randomize any returned memory.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><constant>EP_MEM_F_ALIGN</constant></term>

          <listitem>

            <para>The application would prefer that the allocation is

            page-aligned. This is not available on all architectures, and

            other architectures do it automatically if the allocation is at

            least as large as a page.</para>

          </listitem>

        </varlistentry>

        <varlistentry>

          <term><constant>EP_MEM_F_WAIT</constant></term>

          <listitem>

            <para>If memory is unavailable, try to wait for it to become

            available (e.g., because another thread has released memory).

            <remark>This is not yet implemented.</remark></para>

          </listitem>

        </varlistentry>

      </variablelist>

      <para>Specifying <constant>EP_MEM_F_ZERO</constant> and

      <constant>EP_MEM_F_TRASH</constant> at the same time is

      undefined.</para>

      <para>Since <function>ep_mem_[mzr]alloc</function> are implemented as

      macros, they can't be used as pointers to functions (e.g., for

      specifying a memory allocator callback to a third party app). For this

      reason, there are also <function>ep_mem_[mzr]alloc_f</function> "real"

      functions to be used in this context.</para>

      <para>Generally, unthreaded code and most application code will probably

      be happy with the defaults. Threaded server code (which cannot be

      permitted to die) is expected to catch the out of memory condition, do

      some recovery operation such as terminating a task, and return

      <errorcode>EP_MEM_STAT_TRYAGAIN</errorcode> so the memory allocation can

      retry.</para>

      <para>[[[XXX Document ep_set_malloc_functions XXX]]]</para>

    </section>

    <section>

      <title>Resource Pools</title>

      <para>Resources are allocatable global entities such as memory, file

      descriptors, etc. Resources can be collected together into pools and

      then freed in one call. Memory is specially handled to allow fast

      allocation from a pool --- specifically, a chunk of memory can be

      allocated from the heap to a pool and then sub-allocated as needed.

      Allocating memory from resource pools is particularly fast for small

      allocations. Also, pool allocations that are of a size that is a

      multiple of the page size are guaranteed to return a page-aligned

      pointer. This is particularly useful to allow the I/O level to implement

      zero-copy I/O.</para>

      <para>The heap used is the one that is current when

      <function>ep_rpool_new</function> is invoked.</para>

      <programlisting>    #include &lt;ep/ep_mem.h&gt;

    EP_RPOOL *

    ep_rpool_new(const char *name,              // for debugging

                size_t qsize)                   // min memory allocation quantum

    EP_STAT

    ep_rpool_free(EP_RPOOL *rp)                 // free pool and all resources

    void *

    ep_rpool_malloc(EP_RPOOL *rp,               // the pool to allocate from

                size_t nbytes)                  // number of bytes

    void *

    ep_rpool_zalloc(EP_RPOOL *rp,               // the pool to allocate from

                size_t nbytes)                  // number of bytes

    void *

    ep_rpool_xalloc(EP_RPOOL *rp,               // the pool to allocate from

                size_t nbytes,                  // number of bytes

                const char *filename,           // file name (for debugging)

                int lineno,                     // line number (for debugging)

                uint32_t flags)                 // flag bits (see below)

    void *

    ep_rpool_strdup(EP_RPOOL *rp,               // the pool to allocate from

                char *str)                      // the string to save

    void *

    ep_rpool_realloc(EP_RPOOL *rp,              // pool to allocate from

                void *old mem,                  // old memory pointer

                size_t oldsize,                 // old allocation size

                size_t newsize)                 // new allocation size

    void

    ep_rpool_mfree(EP_RPOOL *rp,                // the pool to release to

                void *p);                       // the memory

    void

    ep_rpool_mfreeto(EP_RPOOL *rp,              // the pool to release to

                void *p);                       // restore up the pool to here

    EP_STAT

    ep_rpool_attach(EP_RPOOL *rp,               // the resource pool

                void freefunc(void *arg),       // a function to call on free

                void *arg)                      // argument to pass to it</programlisting>

      <para>The <function>ep_rpool_mfreeto</function>() routine lets you treat

      rpool memory like a stack; this call releases everything allocated back

      to (and including) the pointer given. If <computeroutput>p ==

      NULL</computeroutput>, the entire memory contents of the rpool are

      freed, but the rpool itself is still active. Deep care needs to be taken

      here: if a subordinate routine is called that allocates memory from the

      rpool, you may end up deallocating memory that is still in use.

      <remark>Not implemented at this time.</remark></para>

      <para>The <function>ep_rpool_attach</function>() routine is used to

      associate other resources (such as files) with a pool. The corresponding

      free functions will be invoked when the pool is freed.</para>

      <para>In most cases, passing in <parameter>rp</parameter> ==

      <constant>NULL</constant> treats the call like the corresponding heap

      allocation. In this case the caller is responsible for freeing the

      memory. For example,

      <function>ep_rpool_malloc</function>(<constant>NULL</constant>,

      <varname>nbytes</varname>) is equivalent to

      <function>ep_mem_malloc</function>(<varname>nbytes</varname>).</para>

      <para>The distinction between multiple heaps and resource pools are that

      heaps are not intended for application use other than for doing recovery

      for out-of-memory conditions. Pools are intended for general use. Pools

      are fast at allocation time (since they just grab space from the end of

      the pool) and fast at free time (since the entire pool can be

      deallocated at once); heaps are comparatively slow.</para>

      <para>When any memory collections (heaps or pools) are freed, all

      objects allocated from that collection are freed (i.e., their

      destructors are automatically invoked).</para>

    </section>

    <section>

      <title>Opening Memory as a File</title>

      <programlisting>    FILE *

    ep_fopen_smem(void *buf,          // block of memory to open

        size_t bsize,                 // size of that memory

        const char *mode)             // fopen(3) mode string</programlisting>

    </section>

  </section>

  <section>

    <title>TIME</title>

    <para>The ep library has a separate time abstraction. This is for two

    reasons: first, it guarantees that the number of seconds since January 1,

    1970 will be sufficiently long to last past 2038 (this varies from system

    to system), and it includes a "<structfield>tv_accuracy</structfield>"

    (type float) to indicate the approximate accuracy of the clock relative to

    absolute time. For example, a clock synchronized from a GPS clock might be

    accurate within perhaps 100nsec, whereas a standard crystal clock

    synchronized once a day might only have an accuracy of a few

    seconds.</para>

    <programlisting>#include &lt;ep/ep_time.h&gt;

typedef struct

{

        int64_t     tv_sec;          // seconds since Jan 1, 1970

        int32_t     tv_nsec;         // nanoseconds

        float       tv_accuracy;     // clock accuracy in seconds

} EP_TIME_SPEC;

#define EP_TIME_NOTIME      (-INT64_MAX)

#define EP_TIME_MAXTIME     (INT64_MAX)

EP_STAT

ep_time_now(                         // return current time

        EP_TIME_SPEC *tv);

EP_STAT

ep_time_deltanow(                    // return time in the future (or past)

        uint64_t delta_nanoseconds,

        EP_TIME_SPEC *tv);

void

ep_time_add_delta(                   // add a delta to a time (delta may be negative)

        EP_TIME_SPEC *delta,

        EP_TIME_SPEC *tv);

bool

ep_time_before(                      // determine if A occurred before B

        EP_TIME_SPEC *a,

        EP_TIME_SPEC *b);

void

ep_time_from_nsec(                   // create a time from a scalar number of nanoseconds

        int64_t nsec,

        EP_TIME_SPEC *tv);

void

ep_time_from_sec(                    // create a time from a scalar number of seconds

        int64_t sec,

        EP_TIME_SPEC *tv);

float

ep_time_accuracy(void);              // return putative clock accuracy

void

ep_time_setaccuracy(                 // set the clock accuracy (may not be available)

        float accuracy);

void

ep_time_format(                      // format a time string into a buffer

        EP_TIME_SPEC *tv,

        char *buf,

        size_t bufsize,

        uint32_t flags);

void

ep_time_print(                       // format a time string to a file

        EP_TIME_SPEC *tv,

        FILE *fp,

        uint32_t flags);

// values for ep_time_format and ep_time_print flags

#define EP_TIME_FMT_DEFAULT     0               // pseudo-flag

#define EP_TIME_FMT_HUMAN       0x00000001      // format for humans

#define EP_TIME_FMT_NOFUZZ      0x00000002      // suppress accuracy printing

EP_STAT

ep_time_parse(                       // parse a time string

        const char *timestr,

        EP_TIME_SPEC *tv,

        uint32_t flags);

// values for ep_time_parse flags

#define EP_TIME_USE_UTC         0x00000000      // assume UTC (default)

#define EP_TIME_USE_LOCALTIME   0x00000001      // assume times in local zone

EP_STAT

ep_time_nanosleep(                   // sleep for the indicated number of nanoseconds

        int64_t nanoseconds);

bool

EP_TIME_IS_VALID(                    // test to see if a timestamp is valid

        EP_TIME_SPEC *tv);

void

EP_TIME_INVALIDATE(                  // invalidate a timestamp

        EP_TIME_SPEC *tv);</programlisting>

    <para>"Human" formatted times are intended to be human readable, and may

    use non-ASCII characters. Otherwise the format is intended to be machine

    readable, e.g., using <function>ep_time_parse</function>.</para>

  </section>

  <section>

    <title>DATA STRUCTURES</title>

    <section>

      <title>Property Lists</title>

      <para><remark>Not implemented at this time.</remark> A series of

      key=value pairs. Used for many things, including configuration files.

      For example, looking in the "configuration" property list for

      "<varname>mailer.local.timeout.connect</varname>" would return the

      connect timeout for the local mailer. <remark>[[How does this deal with

      nested defaults — e.g., looking for timeout.connect if the full

      path cannot be found?]]</remark></para>

      <programlisting>    EP_PLIST *

    ep_plist_new(

                const char *name)              // for printing

    EP_STAT

    ep_plist_load(

                EP_PLIST *plp,                 // the list to read into

                FILE *sp,                      // the stream to load from

                const char *prefix)            // prefix added to all properties

    EP_STAT

    ep_plist_set(

                EP_PLIST *plp,                 // the plist in which to set

                const char *keyname,           // the name of the key to set

                const char *value)             // the value to set (will be copied)

    const char *

    ep_plist_get(

                EP_PLIST *plp,                 // the plist to search

                const char *keyname)           // the name of the key to get

    void

    ep_plist_dump(

                EP_PLIST *plp,                 // plist to print

                FILE *sp)                      // stream to print to

    void

    ep_plist_free(

                EP_PLIST *plp)                 // plist to free</programlisting>

      <para>A property list can be loaded from an external stream using

      <function>ep_plist_load</function>. The syntax of the file is a simple

      text file with "key=value" pairs on separate lines, with blank lines and

      those with # at the beginning of the line ignored. The values are

      strictly strings. <remark>[[Does it make sense to type

      them?]]</remark></para>

      <para><remark>[[Note the overlap between plists and the

      <function>ep_adm</function> interface. Does this make

      sense?]]</remark></para>

      <para>Property lists can be printed using

      <function>ep_plist_dump</function>. The output format will be readable

      by <function>ep_plist_load</function>. For the time being,

      <varname>flags</varname> should always be <constant>0</constant>.</para>

      <warning>

        <para>The property list is not guaranteed to be dumped in the same

        order items are inserted.</para>

      </warning>

    </section>

    <section>

      <title>Hashes</title>

      <programlisting>    #include &lt;ep/ep_hash.h&gt;

    EP_HASH *

    ep_hash_new(

                const char *name,               // for printing

                EP_HASH_HASH_FUNCP *hfunc,      // alternate hash function

                int tabsize)                    // hash table function size

    void

    ep_hash_free(

                EP_HASH *hp)                    // hash to free

    void *

    ep_hash_search(

                const EP_HASH *hp,              // hash to search

                size_t keylen,                  // length of key

                const void *key)                // pointer to key

    void *

    ep_hash_insert(                     // returns old value for key

                EP_HASH *hp,                    // hash to modify

                size_t keylen,                  // length of key

                const void *key,                // pointer to key

                void *val)                      // value to insert

    ep_hash_forall(EP_HASH *hp,                 // hash to walk

                void (func)(                    // function to call

                        int keylen,                // key length

                        const void *key,           // key value

                        void *val,                 // value

                        void *closure),            // from caller

                void *closure)                  // passed to func

    ep_hash_dump(EP_TREE *tree,                 // tree to dump

                FILE *sp)                       // stream to print on</programlisting>

      <para><remark>[[Should <function>ep_hash_dump</function> take the same

      parameters as the usual object print routine? For that matter, should

      there be a separate <function>ep_hash_dump</function> routine, or should

      it just be a generic <function>ep_obj_dump</function>? Note that

      <function>ep_hash_dump</function> is not implemented at this time, but

      an internal (object-based) dump is.]]</remark></para>

    </section>

    <section>

      <title>Function Lists</title>

      <programlisting>    #include &lt;ep/ep_funclist.h&gt;

    EP_FUNCLIST *

    ep_funclist_new(

                const char *name)       // name for printing/debugging

    void

    ep_funclist_free(EP_FUNCLIST *fp)   // list to free

    void

    ep_funclist_push(EP_FUNCLIST *fp,   // list to push to

                void (*func)(           // the function to invoke

                        void *closure,  // from the ep_funclist_push call

                        void *arg),     // from the ep_funclist_invoke call

                void *closure)          // the closure arg to pass to it

    void

    ep_funclist_pop(EP_FUNCLIST *fp)    // list to pop from, value discarded

    void

    ep_funclist_clear(EP_FUNCLIST *fp)  // list to clear

    void

    ep_funclist_invoke(EP_FUNCLIST *fp, // invoke all functions in list

                void *arg)              // second func arg</programlisting>

    </section>

  </section>

  <section>

    <title>CRYPTOGRAPHIC SUPPORT</title>

    <para>The current implementation wraps the OpenSSL library, but it could

    be retargeted.</para>

    <para>Before any cryptographic functions can be used, the library must be

    initialized:</para>

    <programlisting>void         ep_crypto_init(uint32_t flags)</programlisting>

    <para>At the moment flags is unused (just pass zero). There are also

    several general purpose definitions, useful for declaring buffers without

    memory allocation:</para>

    <informaltable border="1">

      <col width="4*"/>