Project

General

Profile

Statistics
| Branch: | Tag: | Revision:

gdp / doc / developer / gdp-programmatic-api.html @ master

History | View | Annotate | Download (76.9 KB)

1 26fa111b Eric Allman
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
2
<html>
3
  <head>
4 10125206 Eric Allman
    <meta http-equiv="content-type" content="text/html; charset=windows-1252">
5 26fa111b Eric Allman
    <title>Global Data Plane Programmatic API</title>
6
    <style type="text/css">
7 a801a6ea Eric Allman
.warning {
8
  font-weight: bold;
9
  font-style: italic;
10
  color: red;
11 26fa111b Eric Allman
  background-color: #ffff99;
12
}
13 d9827038 Eric Allman
.meta {
14
  font-style:italic;
15
  color: blue;
16
}
17 19bf6370 Eric Allman
.command {
18
  font-family: monospace;
19
}
20
.variable {
21
  font-style: oblique;
22
}
23
.filename {
24
  font-family: monospace;
25
}
26 0b99b3fa Eric Allman
.admin-param {
27
  font-family: monospace;
28
}
29
.function {
30
  font-family: monospace;
31
}
32 808a66bf Eric Allman
.manifest {
33
  font-family: monospace;
34
}
35 7d83fa37 Eric Allman
.datatype {
36
  font-family: monospace;
37
}
38 25650d47 Eric Allman
39 26fa111b Eric Allman
</style></head>
40
  <body>
41 a2d7923a Eric Allman
    <h1>Global Data Plane Programmatic API </h1>
42 26fa111b Eric Allman
    <h4>Editor: Eric Allman, U.C. Berkeley Swarm Lab, eric@cs.berkeley.edu<br>
43 80da996e Eric Allman
      Version 2.1.0, 2018-10-20</h4>
44 26fa111b Eric Allman
    <p>This document describes the procedural programmatic interface to the
45
      Global Data Plane.&nbsp; The native code is written in C for maximum
46
      flexibility and performance, but it is expected that most applications
47
      will be written in higher level languages, and hence there will be
48
      multiple language bindings for the GDP library.&nbsp; There is also a REST
49 a2d7923a Eric Allman
      interface that is not described in this document. </p>
50 26fa111b Eric Allman
    <p>The GDP library uses the EP portability library, and applications are
51
      free to use that library as well; in particular, since the GDP libraries
52
      makes extensive use of the EP library some efficiencies may result from
53
      using them both.&nbsp; However, this document does not attempt to define
54
      the EP library and describes it only as necessary.&nbsp; However, one EP
55
      concept that appears commonly is the <code>EP_STAT</code> data type,
56
      which represents a status code that can be returned by a function to
57
      indicate completion status that includes a "severity" (e.g. OK, ERROR,
58
      SEVERE), a "registry" (in our case always UC Berkeley), a "module" (e.g.,
59
      GDP or the EP library itself), and detail information.&nbsp; An OK status
60 a2d7923a Eric Allman
      can return a positive integer as extra information. </p>
61 26fa111b Eric Allman
    <p>The code distribution includes an "apps" directory with two example
62 d9827038 Eric Allman
      programs: <code>gdp-writer.c</code> and <code>gdp-reader.c</code>, that
63
      show simple cases of how to append to a GOB and read from a GOB (including
64
      subscriptions). </p>
65 80da996e Eric Allman
    <p>This document corresponds to version 2.1 of the GDP library and
66 eae1d3ec Eric Allman
      associated applications.&nbsp; Note that this interface is massively
67
      incompatible with version 0.9.</p>
68 80da996e Eric Allman
    <h2 class="meta">0&nbsp; To Be Done</h2>
69
    <ul>
70
      <li>Document metadata interfaces, e.g., <code>gdp_gin_getmetadata</code>,
71
        <code>gdp_md_get</code>.</li>
72
      <li>Convert this documentation into man pages.</li>
73
    </ul>
74 26fa111b Eric Allman
    <h2>1&nbsp; Terminology</h2>
75
    <dl>
76
      <dt>Datum</dt>
77
      <dd>A unit of data in the GDP; essentially a record.&nbsp; Each datum has
78
        associated with it a record number (numbered sequentially from one), a
79
        commit timestamp (the time the record was committed into the GDP, as
80 808a66bf Eric Allman
        distinct from the time that the data originated), an associated blob
81
        containing the data itself, which we expect to be encrypted, and in most
82 d9827038 Eric Allman
        cases a signature.&nbsp; Beware: because of our unique single-writer
83
        commit protocol, record numbers are not guaranteed to be unique nor
84
        continuous; for example, there can be no record 3 but two record 4s in
85
        some circumstances.&nbsp; This should be rare.</dd>
86
      <dt>GDP Object (GOB)</dt>
87 eae1d3ec Eric Allman
      <dd>This represents an addressable entity in the Global Data Plane, which
88
        may be a log or a service.&nbsp; It is always addressed with a
89 d9827038 Eric Allman
        location-independent 256-bit name (a "GDPName").&nbsp; Applications
90
        interact with GOBs via GDP Instances (GINs).</dd>
91 eae1d3ec Eric Allman
      <dt>GOB Instance (GIN)</dt>
92
      <dd>An instance of a Global Data Plane log.&nbsp; An application can have
93
        multiple instances of a given GDP Object.&nbsp; It is the equivalent of
94
        a Unix File Descriptor (as distinct from a Unix file) that is created by
95
        <code>gdp_gin_open</code>.</dd>
96
      <dt>GDP Name</dt>
97
      <dd>The name of a Global Data Plane Object.&nbsp; This is a 256-bit,
98
        opaque, globally unique number created as the SHA-256 hash of the
99 80da996e Eric Allman
        metadata of the object.&nbsp; The usual representation for printing is a
100
        43 character base64 encoded string.</dd>
101
      <dt>Human-Oriented Name</dt>
102
      <dd>A name that human beings like to use, such as a conventional file
103
        name.&nbsp; These are unrelated to the internal name.&nbsp; A
104
        Human-Oriented Name to GDPname Directory Service (HONGDS) maintains a
105
        database mapping from one to the other.</dd>
106 26fa111b Eric Allman
    </dl>
107 eae1d3ec Eric Allman
    <p> In earlier versions of the GDP, both GINs and GOBs were referred to as
108 d9827038 Eric Allman
      GCLs (GDP Channel-Logs). These names are now broken apart for clarity. For
109
      historic reasons, some of the interfaces and much of the documentation
110
      still refer to GCLs, but that name is officially deprecated.</p>
111 eae1d3ec Eric Allman
    <p><i>A Note on Naming:</i> Function and type names are mostly intended to
112
      map cleanly to class and method names, which occasionally leads to
113 d9827038 Eric Allman
      somewhat tortured names.&nbsp; For example, function names beginning with
114
      <code>gdp_gin_</code> operate on objects of type <code>gdp_gin_t</code>
115
      (with a few exceptions such as <code>gdp_gin_new</code>) and will take a
116
      pointer to a <code>gdp_gin_t</code> as the first ("self") argument.&nbsp;
117
      In most cases, everything that has a type (identified by a name ending <code>_t</code>)
118
      is probably a class.&nbsp; More concretely, <code>gdp_gin_read_by_recno(gin,
119
        ...)</code> where <code>gin</code> is type <code>gdp_gin_t</code> is
120
      intended to map to a method "<code>gin.read_by_recno(...)</code>" (i.e,
121
      the operation <code>read_by_recno</code> on a variable of class <code>gdp_gin</code>),
122 e95d2be4 Eric Allman
      <code>gdp_buf_getlength(buf, ...)</code> maps to <code><em>buf</em>.getlength(...)</code>,
123 d9827038 Eric Allman
      etc.&nbsp; In some cases these also apply to what would be class methods;
124
      for example <code>gdp_gin_create(...)</code> would map to a class method
125
      <code>gdp_gin.create(...)</code>.</p>
126
    <br>
127 eae1d3ec Eric Allman
    <h2> 2&nbsp; Theory of Operation </h2>
128 d9827038 Eric Allman
    <p> GDP-based applications rely on three major pieces: an in-process GDP
129
      library, a GDP Log Daemon, and the Routing Layer.&nbsp; In the future
130
      there will also be a service layer, but to date that is ad hoc at
131
      best.&nbsp; This document describes the GDP library API. </p>
132 26fa111b Eric Allman
    <p> The primary point of the GDP library is to speak the network protocol
133
      between the application and the GDP Daemon. The library is threaded, with
134 e95d2be4 Eric Allman
      (at the moment) at least two threads: one to process events (data arriving
135
      from the daemon, although others can be added), and the other to run the
136 26fa111b Eric Allman
      application itself. This allows the application to pretend it is a
137
      sequential program while still allowing asynchronous input from the GDP
138
      Daemon (e.g., processing results from subscription requests).&nbsp;
139
      Applications are free to create other threads as desired.&nbsp; The code
140
      has been written to do the locking as necessary, but stress tests have not
141 a2d7923a Eric Allman
      been run, so you may find unhappy results. </p>
142 eae1d3ec Eric Allman
    <p> The primary abstraction is the GDP Object (GOB). A GOB represents the
143
      rendezvous point for communication in the data plane. It is not directly
144
      tied to either a file or a network connection. On creation, a GOB is
145
      assigned a 256-bit opaque name. A GOB is append-only to writers.&nbsp; For
146
      the moment you can access the dataplane in one of two modes: synchronous
147 e95d2be4 Eric Allman
      mode (e.g., using <code>gdp_gin_read_by_*</code> for reading) or an
148
      asynchronous mode (e.g., using <code>gdp_gin_read_by_*_async</code> or <code>gdp_gin_subscribe</code>
149
      for reading). In asynchronous mode the original call return status applies
150
      to the sending of the command, while final results (including any read
151
      data) will be returned using callbacks or an event interface.&nbsp; These
152
      are described in more detail below. </p>
153 eae1d3ec Eric Allman
    <p>All GOBs are named with an opaque, location independent, 256-bit number
154 26fa111b Eric Allman
      from a flat namespace.&nbsp; When printed it is shown as a base64-encoded
155
      value that is tweaked to be valid in a URI (that is, "+" and "/" are
156
      replaced with "&ndash;" and "_").&nbsp; Applications may choose to overlay
157 d9827038 Eric Allman
      these unsightly names with some sort of directory service.&nbsp; <span class="meta">Such
158
        a directory service is planned but not yet implemented.</span></p>
159 26fa111b Eric Allman
    <p>Applications using the GDP library should <code>#include
160
        &lt;gdp/gdp.h&gt;</code> for all the essential definitions.</p>
161 d9827038 Eric Allman
    <br>
162
    <br>
163 eae1d3ec Eric Allman
    <h2>3&nbsp; GDP Operations</h2>
164
    <h3>3.1 Data Types and Initialization</h3>
165
    <hr>
166 26fa111b Eric Allman
    <h4>Name</h4>
167 d63b8179 Eric Allman
    <p>GDP data types and basic utilities</p>
168 26fa111b Eric Allman
    <h4>Synopsis</h4>
169 d9827038 Eric Allman
    <pre>#include &lt;gdp/gdp.h&gt;<br><br>// names: internal and printable<br>gdp_name_t        InternalGdpName;        // 256-bit number<br>gdp_pname_t        PrintableGdpName;        // base-64 encoded string<br>bool                GDP_NAME_SAME(gdp_name_t a, gdp_name_t b);<br>bool                gdp_name_is_valid(gdp_name_t gdpname);<br><br>// convert between printable and internal names<br>char                *gdp_printable_name(const gdp_name_t InternalFormat,<br>                        gdp_pname_t PrintableFormat);<br>EP_STAT                gdp_internal_name(const gdp_pname_t PrintableFormat,<br>                        gdp_name_t Internal);<br><br>// this will do human-friendly name lookup (may be expensive)<br>EP_STAT                gdp_parse_name(const char *external,<br>                        gdp_name_t Internal);<br><br>gdp_gin_t        *GdpLogInstance;<br><br>gdp_datum_t        *GdpDatum;                // data storage unit<br>gdp_hash_t      *GdpHash;                // hash function over a gdp_datum_t<br>gdp_sig_t        *GdpSignature;                // signature (multiple usage)<br>gdp_recno_t        RecordNumber;                // datum record number<br>EP_TIME_SPEC        TimeStampSpec;                // timestamp: see libep</pre>
170 26fa111b Eric Allman
    <h4>Notes</h4>
171
    <ul>
172
      <li>Most of these are described in more detail below.</li>
173
    </ul>
174 d9827038 Eric Allman
    <br>
175 eae1d3ec Eric Allman
    <hr width="100%" size="2">
176 26fa111b Eric Allman
    <h4> Name</h4>
177 b1d346a7 Eric Allman
    gdp_init &mdash; Initialize the GDP library
178 26fa111b Eric Allman
    <h4> Synopsis</h4>
179
    <pre>#include &lt;gdp/gdp.h&gt;<br><br>EP_STAT gdp_init(const char *gdpd_addr)
180
</pre>
181
    <h4> Notes</h4>
182
    <ul>
183
      <li> Initializes the GDP library.&nbsp; <em><strong>Must</strong></em> be
184
        called before any other GDP functions are invoked.</li>
185
      <li>The <code>gdpd_addr</code> parameter is the address to use to contact
186
        the GDP routing layer in the format "host:port".&nbsp; If <code>NULL</code>
187 a2d7923a Eric Allman
        a system default is used. </li>
188 26fa111b Eric Allman
      <li>If the status is not <code>EP_STAT_OK</code> then the library failed
189
        to initialize (for example, by being unable to acquire resources.&nbsp;
190 80da996e Eric Allman
        Failure to check this status (generally using the <code class="function">EP_STAT_ISOK</code>
191
        predicate) may result in mysterious failures later.</li>
192 b1d346a7 Eric Allman
      <li>All source files using the GDP must include <code>&lt;gdp/gdp.h&gt;</code>.
193 26fa111b Eric Allman
      </li>
194
    </ul>
195 10125206 Eric Allman
    <hr width="100%" size="2">
196 26fa111b Eric Allman
    <h4>Name</h4>
197
    <p>GDP_LIB_VERSION &mdash; GDP library version</p>
198
    <h4>Synopsis</h4>
199
    <pre>#include &lt;gdp/gdp_version.h&gt;</pre>
200
    <h4>Notes</h4>
201
    <ul>
202 937851e5 Eric Allman
      <li>The <code>gdp_version.h</code> file defines the integer constant <code>GDP_LIB_VERSION</code>
203
        as the major, minor, and patch level of this version of the GDP library,
204
        for example, 0x010203 for version 1.2.3.&nbsp; It can be used during
205
        compilation.&nbsp; There is also a string <code>GdpVersion</code> that
206
        is suitable for printing.</li>
207 26fa111b Eric Allman
    </ul>
208
    <hr>
209 eae1d3ec Eric Allman
    <h3>3.2&nbsp; GOB Management</h3>
210
    <p>This section covers operations on GOBs as a whole, notably creating,
211
      opening, and closing.</p>
212
    <br>
213 10125206 Eric Allman
    <hr width="100%" size="2">
214 26fa111b Eric Allman
    <h4> Name</h4>
215 80da996e Eric Allman
    gdp_gin_create &mdash; Create an append-only GOB, returning a GIN<span class="warning"></span>
216 26fa111b Eric Allman
    <h4> Synopsis</h4>
217 80da996e Eric Allman
    <pre>EP_STAT gdp_gin_create(gdp_create_info_t *gci,<br>                const char *human_name,<br>                gdp_gin_t **ginp)
218 26fa111b Eric Allman
</pre>
219
    <h4> Notes</h4>
220
    <ul>
221 80da996e Eric Allman
      <li> Creates a GOB with the given human-oriented name.&nbsp; The
222
        human-oriented name is a text string which is stored in the metadata and
223
        inserted into the Human-Oriented Name to GDPname Directory by the
224
        creation service.</li>
225
      <li>Metadata and other details are provided by the indicated creation
226
        information, which may be <code>NULL</code>.</li>
227
      <li>Arguably should be called <code>gdp_gob_create</code> since it
228
        creates a GOB, but it returns a GIN.</li>
229
      <li>Unless otherwise arranged via the <code>gci</code> parameter, a
230
        keypair will be created to be associated with this log.</li>
231 eae1d3ec Eric Allman
      <li> Returns a GOB instance (indirectly through <code>*ginp</code>).</li>
232
      <li> The returned GIN handle is an append-only object.&nbsp; <i>(Actually,
233 124e7f49 Eric Allman
234
235 fe034bdb Eric Allman
236 2b29bb77 Eric Allman
237 16dcce04 Eric Allman
238
239 cf71c5ea Eric Allman
240 26fa111b Eric Allman
          this is not enforced at this time).</i></li>
241
    </ul>
242
    <br>
243 80da996e Eric Allman
    <hr>
244
    <h4>Name</h4>
245
    <p>gdp_create_info_new &mdash; allocate a new creation information data
246
      structure</p>
247
    <h4>Synopsis</h4>
248
    <pre>gdp_create_info_t *gdp_create_info_new(void)</pre>
249
    <h4>Notes</h4>
250
    <ul>
251
      <li>Returns a new creation information data structure to be used in the
252
        following routines.</li>
253
    </ul>
254
    <hr>
255
    <h4>Name</h4>
256
    gdp_create_info_free &mdash; Free the creation information structure<br>
257
    <h4>Synopsis</h4>
258
    <pre>void gdp_create_info_free(gdp_create_info_t *info)</pre>
259
    <h4>Notes</h4>
260
    <ul>
261
      <li>Frees the data structure pointed to by <code>info</code>.</li>
262
      <li>This can be freed as soon as it has been used in <code>gdp_create</code>
263
        or re-used to create multiple logs.</li>
264
    </ul>
265
    <hr>
266
    <h4>Name</h4>
267
    gdp_create_info_set_owner_key, gdp_create_info_set_writer_key &mdash; Set
268
    owner/writer key for a new GOB<br>
269
    <h4>Synopsis</h4>
270
    <pre>EP_STAT gdp_create_info_set_owner_key(<br>                gdp_create_info_t *info,
271
                EP_CRYPTO_KEY *skey,
272
                const char *dig_alg_name)
273
274
275
EP_STAT gdp_create_info_set_writer_key(
276
                gdp_create_info_t *info,
277
                EP_CRYPTO_KEY *skey,
278
                const char *dig_alg_name)</pre>
279
    <h4>Notes</h4>
280
    <ul>
281
      <li>Sets the owner or writer key to be used for a new GOB.</li>
282
      <li>The <code>dig_alg_name</code> parameter specifies the message digest
283
        algorithm to be used when signing.</li>
284
    </ul>
285
    <hr>
286
    <p></p>
287
    <ul>
288
    </ul>
289
    <h4>Name</h4>
290
    gdp_create_info_new_owner_key, gdp_create_info_new_writer_key &mdash; Create
291
    new owner/writer key for a new GOB<br>
292
    <h4>Synopsis</h4>
293
    <pre>EP_STAT gdp_create_info_new_owner_key(<br>&nbsp;&nbsp;&nbsp;             gdp_create_info_t *info,<br>&nbsp;&nbsp;&nbsp;             const char *dig_alg_name,<br>                const char *key_alg_name,<br>                int key_bits,<br>                const char *curve_name,<br>                const char *key_enc_alg_name)
294
295
296
EP_STAT gdp_create_info_new_writer_key(
297
&nbsp;&nbsp;&nbsp;             gdp_create_info_t *info,<br>&nbsp;&nbsp;&nbsp;             const char *dig_alg_name,<br>                const char *key_alg_name,<br>                int key_bits,<br>                const char *curve_name,<br>                const char *key_enc_alg_name)
298
</pre>
299
    <h4>Notes</h4>
300
    <ul>
301
      <li>Creates a new owner or writer key to be used for a new GOB.</li>
302
      <li>The <code>dig_alg_name</code> parameter specifies the message digest
303
        algorithm to be used when signing.</li>
304
      <li>The <code>key_alg_name</code>, <code>key_bits</code>, and <code>curve_name</code>
305
        parameters specify the signing algorithm.</li>
306
      <li>The <code>key_enc_alg_name</code> parameter specifies the symmetric
307
        encryption algorithm to be used when the key is written to disk.</li>
308
      <li>If any of the parameters are <code>NULL</code> (or zero, in the case
309
        of <code>key_bits</code>), a default is used; these defaults can be set
310
        using administrative parameters.</li>
311
    </ul>
312
    <hr>
313
    <h4>Name</h4>
314
    <p>gdp_create_info_set_creator &mdash; set name and domain of entity
315
      creating this GOB</p>
316
    <h4>Synopsis</h4>
317
    <pre>EP_STAT gdp_create_info_set_creator(
318
&nbsp;&nbsp;&nbsp;             gdp_create_info_t *info,<br>&nbsp;&nbsp;&nbsp;             const char *user,<br>                const char *domain)</pre>
319
    <p></p>
320
    <h4>Notes</h4>
321
    <ul>
322
      <li>Sets the name and/or domain of the creator of this GOB.</li>
323
      <li>The full creator name is set to <code>user</code>@<code>domain</code>.</li>
324
      <li>If only the <code>user</code> is specified, the <code>domain</code>
325
        is set to the DNS domain name of the current host.</li>
326
      <li>If only the <code>domain</code> is specified, the <code>user</code>
327
        is set to the currently logged in user.</li>
328
      <li>If user contains an "<code>@</code>" sign and no <code>domain</code>
329
        is specified, the <code>user</code> is used directly as the creator.</li>
330
      <li>The creator name is included in the log metadata for informational
331
        purposes; it has no direct impact on the semantics of the GOB.</li>
332
    </ul>
333
    <hr>
334
    <h4>Name</h4>
335
    <p>gdp_create_info_set_creation_service &mdash; set the name of the GDP
336
      creation service</p>
337
    <h4>Synopsis</h4>
338
    <pre>EP_STAT gdp_create_info_set_creation_service(
339
&nbsp;&nbsp;&nbsp;             gdp_create_info_t *info,<br>&nbsp;&nbsp;&nbsp;             const char *creation_service_name)</pre>
340
    <p></p>
341
    <h4>Notes</h4>
342
    <ul>
343
      <li>Sets the name of the creation service.&nbsp; Use of this should be
344
        rare.</li>
345
      <li>Currently defaults to a system-wide default.</li>
346
      <li>In the future it will probably try with Zeroconf.</li>
347
    </ul>
348
    <p></p>
349
    <ul>
350
    </ul>
351
    <hr>
352
    <h4>Name</h4>
353
    <p>gdp_create_info_set_expiration &mdash; set GOB expiration parameters</p>
354
    <h4>Synopsis</h4>
355
    <pre>EP_STAT gdp_create_info_add_expiration(<br>&nbsp;&nbsp;&nbsp;             gdp_create_info_t *info,<br>                &lt;to be determined&gt;)</pre>
356
    <p></p>
357
    <h4>Notes</h4>
358
    <ul>
359
      <li class="warning">It still isn't clear how this is going to work.&nbsp;
360
        For the time being it should not be used.&nbsp; The parameters will
361
        probably change.</li>
362
    </ul>
363
    <ul>
364
    </ul>
365
    <hr>
366
    <h4>Name</h4>
367
    <p>gdp_create_info_add_metadata &mdash; add user-defined metadata to the GOB
368
      metadata</p>
369
    <h4>Synopsis</h4>
370
    <pre>EP_STAT gdp_create_info_add_metadata(
371
&nbsp;&nbsp;&nbsp;             gdp_create_info_t *info,<br>&nbsp;&nbsp;&nbsp;             uint32_t md_name,<br>                size_t md_len,<br>                const char *md_val)</pre>
372
    <p></p>
373
    <h4>Notes</h4>
374
    <ul>
375
      <li>Adds the indicated value to the log metadata.</li>
376
      <li>Metadata names are 32-bit unsigned numbers.&nbsp; Values where the
377
        high-order byte is zero are reserved for system-defined use.</li>
378
      <li>Conventionally metadata names are actually ASCII characters encoded
379
        into a four-byte integer.&nbsp; For example, the name <code>0x44454d4f</code>
380
        could also be read as "<code>DEMO</code>".</li>
381
      <li>Metadata values are arbitrary binary strings up to 255 bytes in
382
        length.</li>
383
    </ul>
384
    <ul>
385
    </ul>
386 10125206 Eric Allman
    <hr width="100%" size="2">
387 26fa111b Eric Allman
    <h4> Name</h4>
388 eae1d3ec Eric Allman
    gdp_gin_open &mdash; Open an existing GOB, returning a GIN<br>
389 26fa111b Eric Allman
    <h4> Synopsis</h4>
390 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_gin_open(gdp_name_t name,
391
                gdp_iomode_t rw,
392
                gdp_open_info_t *info,
393
                gdp_gin_t **ginp)
394 26fa111b Eric Allman
</pre>
395
    <h4> Notes</h4>
396
    <ul>
397 eae1d3ec Eric Allman
      <li> Opens the GOB with the indicated name for the mode indicated by <code>rw</code>,
398 b1d346a7 Eric Allman
        which may be <code>GDP_MODE_RO</code> (read only), <code>GDP_MODE_AO</code>
399 eae1d3ec Eric Allman
        (append only), or <code>GDP_MODE_RA</code> (read and append).&nbsp; It
400
        returns an instance of that object.</li>
401 10125206 Eric Allman
      <li>Open information is to pass in detailed information needed for the
402
        open as described below.&nbsp; In most cases it can be passed as
403
        NULL.&nbsp; Eventually it will be used to convey signing keys, quality
404
        of service requests, payment information, authorization tokens, etc.</li>
405 eae1d3ec Eric Allman
      <li>The handle itself is returned indirectly through <code>*ginp</code>.</li>
406
      <li>If a signing key associated with the GOB is found, all writes to this
407
        GOB will automatically be signed.&nbsp; See the section on Signing and
408 10125206 Eric Allman
        Encryption below for details.</li>
409
    </ul>
410
    <hr>
411
    <h4>Name</h4>
412 eae1d3ec Eric Allman
    gdp_open_info_new &mdash; Create a new open information data structure<br>
413 10125206 Eric Allman
    <h4>Synopsis</h4>
414 eae1d3ec Eric Allman
    <pre>gdp_open_info_t *gdp_open_info_new(void)</pre>
415 10125206 Eric Allman
    <h4>Notes</h4>
416
    <ul>
417
      <li>Creates a new data structure to be used in the following routines.</li>
418
    </ul>
419 80da996e Eric Allman
    <hr>
420 10125206 Eric Allman
    <h4>Name</h4>
421 eae1d3ec Eric Allman
    gdp_open_info_free &mdash; Free the open information structure<br>
422 10125206 Eric Allman
    <h4>Synopsis</h4>
423 eae1d3ec Eric Allman
    <pre>void gdp_open_info_free(gdp_open_info_t *info)</pre>
424 10125206 Eric Allman
    <h4>Notes</h4>
425
    <ul>
426
      <li>Frees the data structure pointed to by <code>info</code>.</li>
427 eae1d3ec Eric Allman
      <li>This can be freed as soon as it has been used in <code>gdp_open</code>.</li>
428 10125206 Eric Allman
    </ul>
429
    <hr>
430
    <h4>Name</h4>
431 eae1d3ec Eric Allman
    gdp_open_info_set_signing_key &mdash; Set signing key for an open GOB<br>
432 10125206 Eric Allman
    <h4>Synopsis</h4>
433 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_open_info_set_signing_key(<br>&nbsp;&nbsp;&nbsp;             gdp_open_info_t *info,<br>&nbsp;&nbsp;&nbsp;             EP_CRYPTO_KEY *skey)</pre>
434 10125206 Eric Allman
    <h4>Notes</h4>
435
    <ul>
436
      <li>Sets the signing key to be used when appending data to a log.</li>
437 eae1d3ec Eric Allman
      <li>If no signing key is specified, the library attempts to find the key
438
        using a search path.</li>
439
      <li>If the open <code>info</code> passed to <code>gdp_open</code> for a
440
        GOB that has already been opened using a different key (that is, on a
441
        different instance), the result is undefined.</li>
442 10125206 Eric Allman
    </ul>
443
    <hr>
444
    <h4>Name</h4>
445 eae1d3ec Eric Allman
    <p>gdp_open_info_set_signkey_cb &mdash; Set a callback function to read a
446
      signing key</p>
447 10125206 Eric Allman
    <h4>Synopsis</h4>
448 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_open_info_set_signkey_cb(<br>                gdp_open_info_t *info,<br>                EP_STAT (*signkey_cb)(<br>                        gdp_name_t gname,<br>                        void *signkey_udata,<br>                        EP_CRYPTO_KEY **skey),<br>                void *signkey_udata)</pre>
449 10125206 Eric Allman
    <h4>Notes</h4>
450
    <ul>
451 eae1d3ec Eric Allman
      <li>If the <code>gdp_open</code> call requires a secret key, that that
452
        key was not passed in using <code>gdp_open_info_set_signing_key</code>,
453 10125206 Eric Allman
        the callback function <code>signkey_cb</code> is invoked to get a
454
        key.&nbsp; It will only be invoked if the key is required (notably
455
        because it isn't already cached).</li>
456
      <li>The arbitrary data pointer <code>signkey_udata</code> is passed
457
        through to <code>signkey_cb</code> if it is invoked.</li>
458
    </ul>
459
    <hr>
460
    <h4>Name</h4>
461 eae1d3ec Eric Allman
    gdp_open_info_set_caching &mdash; set caching behavior<br>
462 10125206 Eric Allman
    <h4>Synopsis</h4>
463 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_open_info_set_caching(<br>&nbsp;&nbsp;            &nbsp; gdp_open_info_t *info,<br>&nbsp;&nbsp;            &nbsp; bool keep_in_cache)</pre>
464 10125206 Eric Allman
    <h4>Notes</h4>
465
    <ul>
466 eae1d3ec Eric Allman
      <li>Sets whether this GOB should be kept in the cache after <code>gdp_gin_close</code>
467 10125206 Eric Allman
        is called.</li>
468
      <li>If set, cached information will be reclaimed based on the last usage
469 eae1d3ec Eric Allman
        time of the GOB.</li>
470 10125206 Eric Allman
      <li>Use of this call with <code>keep_in_cache</code> set to <code>TRUE</code>
471
        may cause a cleanup thread to be spawned.</li>
472
    </ul>
473 80da996e Eric Allman
    <hr>
474
    <h4>Name</h4>
475 49d76189 Eric Allman
    <pre>EP_STAT gdp_open_info_set_no_skey_nonfatal(<br>
476
        &nbsp;&nbsp;            &nbsp; gdp_open_info_t *info,<br>
477
        &nbsp;&nbsp;            &nbsp; bool no_skey_nonfatal)</pre>
478
    <p></p>
479
    <h4>Notes</h4>
480
    <ul>
481
      <li>Sets whether the failure to find a secret key for a writable log
482
        is a non-fatal error.</li>
483
      <li>Normally only used for testing or when the server that hosts the
484
              log is configured to not check signatures.</li>
485
      <li>Defaults to <code>false</code> (i.e., lack of a secret key is
486
              a fatal error).</li>
487
    </ul>
488
    <hr>
489
    <h4>Name</h4>
490 80da996e Eric Allman
    <p>gdp_open_info_set_vrfy &mdash; set log verification behavior</p>
491
    <h4>Synopsis</h4>
492
    <pre>EP_STAT gdp_open_info_set_vrfy(<br>&nbsp;&nbsp;            &nbsp; gdp_open_info_t *info,<br>&nbsp;&nbsp;            &nbsp; bool do_verification)</pre>
493
    <p></p>
494
    <h4>Notes</h4>
495
    <ul>
496
      <li>Sets whether data retrieved from a log server should be verified.</li>
497
      <li>The computational cost of verifying additional signatures may
498
        negatively affect performance, but it gives you important security
499
        protection.</li>
500
      <li>Defaults to <code>false</code>.</li>
501
    </ul>
502 10125206 Eric Allman
    <hr width="100%" size="2">
503
    <h4> Name</h4>
504 eae1d3ec Eric Allman
    gdp_gin_close &mdash; Close a GDP Instance (GIN) and release resources
505 26fa111b Eric Allman
    <h4> Synopsis</h4>
506 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_gin_close(gdp_gin_t *gin)
507 26fa111b Eric Allman
</pre>
508
    <h4> Notes</h4>
509
    <ul>
510 eae1d3ec Eric Allman
      <li>Closes an open GDP Instance.</li>
511
      <li>If this is the last open instance for a GOB in this process, sends a
512
        hint to the log daemon that the associated resources are no longer
513
        referenced.</li>
514
      <li>Releases the client-side resources (that is, memory) for this GIN.</li>
515
      <li><i>Should this interface say whether to preserve or drop the GIN
516 26fa111b Eric Allman
          (i.e., is it persistent, or for how long)?</i></li>
517
    </ul>
518 10125206 Eric Allman
    <hr width="100%" size="2">
519 eae1d3ec Eric Allman
    <h3>3.3&nbsp; Synchronous Operations</h3>
520
    <p>Synchronous operations block until the operation is complete.&nbsp; They
521
      are the easiest interface for simple programs, but may not perform as well
522
      as the asynchronous versions.&nbsp; The synchronous calls only read or
523
      write single records at a time; to operate on many records in one call,
524
      use the asynchronous versions.</p>
525
    <p>If synchronous operations do not receive an acknowledgement, they will
526
      attempt to re-send the request after a timeout.</p>
527
    <hr>
528
    <h4>Name</h4>
529
    gdp_gin_append &mdash; Append a record to a GOB
530 26fa111b Eric Allman
    <h4> Synopsis</h4>
531 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_gin_append(gdp_gin_t *gin,
532
                gdp_datum_t *datum,<br>                gdp_hash_t *prevhash)
533 26fa111b Eric Allman
</pre>
534
    <h4> Notes</h4>
535
    <ul>
536 eae1d3ec Eric Allman
      <li> Appends the indicated datum to the GOB.</li>
537 26fa111b Eric Allman
      <li>Any subscribers get immediate updates about the new datum.</li>
538 eae1d3ec Eric Allman
      <li>If a secret key is available for this GOB, the appends will be signed.</li>
539
      <li>The <code>prevhash</code> parameter should be the hash of the
540
        previous record.&nbsp; It is required to provide GDP security
541
        guarantees.&nbsp; If <code>prevhash</code> is NULL, the GDP library
542 d9827038 Eric Allman
        will attempt to fill in the appropriate value.&nbsp; <span class="meta">Someday.</span></li>
543 5ced6d34 Eric Allman
      <li>Note that when this returns the datum will still be filled in
544
        (including the signature).&nbsp; To re-use the same datum, the
545
        application <strong>must</strong> use <code>gdp_datum_reset</code> to
546
        clear the old information before adding new data.</li>
547 26fa111b Eric Allman
    </ul>
548
    <br>
549 10125206 Eric Allman
    <hr width="100%" size="2">
550 26fa111b Eric Allman
    <h4> Name</h4>
551 eae1d3ec Eric Allman
    gdp_gin_read_by_recno, gdp_gin_read_by_ts, gdp_gin_read_by_hash &mdash; Read
552
    from a readable GIN
553 26fa111b Eric Allman
    <h4> Synopsis</h4>
554 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_gin_read_by_recno(gdp_gin_t *gin,
555 26fa111b Eric Allman
                gdp_recno_t recno,
556 eae1d3ec Eric Allman
                gdp_datum_t *datum)<br>EP_STAT gdp_gin_read_by_ts(gdp_gin_t *gin,
557 d2647d1f Eric Allman
                EP_TIME_SPEC *ts,
558 eae1d3ec Eric Allman
                gdp_datum_t *datum)<br>EP_STAT gdp_gin_read_by_hash(gdp_gin_t *gin,<br>                gdp_hash_t *hash,<br>                gdp_datum_t *datum)<br> </pre>
559 26fa111b Eric Allman
    <h4> Notes</h4>
560
    <ul>
561 e95d2be4 Eric Allman
      <li> These present a message-oriented interface.</li>
562
      <li>The user provides the space in which to store the result (see section
563
        4).</li>
564
      <li> An OK stat includes the number of octets actually read. (This is
565
        passed back in datum, so unneeded here.)</li>
566
      <li><code>gdp_gin_read_by_recno</code> reads the specified record number
567
        (sequential, starting from 1).&nbsp; Negative <code>recno</code>s are
568
        interpreted relative to the end of the log (so the value &ndash;1
569
        indicates the last message in the log).</li>
570 eae1d3ec Eric Allman
      <li><code>gdp_gin_read_by_ts</code> reads the record dated on or
571 10125206 Eric Allman
        immediately after the indicated timestamp.</li>
572 eae1d3ec Eric Allman
      <li><code>gdp_gin_read_by_hash</code> returns the record with the
573
        indicated hash.</li>
574 26fa111b Eric Allman
    </ul>
575
    <ul>
576
    </ul>
577 10125206 Eric Allman
    <hr width="100%" size="2">
578 eae1d3ec Eric Allman
    <h3>3.4&nbsp; Asynchronous Operations (Asynchronous I/O, Subscriptions, and
579
      Events) </h3>
580 26fa111b Eric Allman
    <p>Asynchronous operations allow an application to subscribe to one or more
581 eae1d3ec Eric Allman
      GOBs and receive events as those GOBs see activity.&nbsp; The event
582 a2d7923a Eric Allman
      mechanism is intended to be extensible for possible future expansion. </p>
583 eae1d3ec Eric Allman
    <p>Every event has a type, a pointer to the GIN handle, and a pointer to a
584 26fa111b Eric Allman
      datum.&nbsp; Applications could in principle define their own event types,
585 e2682ba9 Eric Allman
      but at the moment this functionality is not exposed.</p>
586
    <p>All asynchronous operations return status and/or data via either a
587
      callback function or the event interface.&nbsp; Callback functions may not
588
      be called in the same thread as the operation initiation.&nbsp; If no
589
      callback function is given then the event interface is used; this has the
590
      effect of serializing the event stream.&nbsp; In either case, it is the
591
      responsibility of the caller to free the event after use using <code>gdp_event_free</code>.</p>
592
    <p>Note that asynchronous calls do not do retransmissions.</p>
593 a2d7923a Eric Allman
    <br>
594
    <hr>
595 e2682ba9 Eric Allman
    <h4>Name</h4>
596
    <p>gdp_event_t &mdash; event structure</p>
597
    <h4>Synopsis</h4>
598
    <p><code>typedef struct _gdp_event&nbsp;&nbsp;&nbsp; gdp_event_t;</code></p>
599
    <h4>Notes</h4>
600
    <ul>
601
      <li>This is an opaque type.</li>
602
    </ul>
603
    <hr>
604
    <h4>Name</h4>
605 728e4394 Eric Allman
    <p>gdp_event_cbfunc_t &mdash; event callback function type</p>
606 e2682ba9 Eric Allman
    <h4>Synopsis</h4>
607 728e4394 Eric Allman
    <p><code>typedef void (*gdp_event_cbfunc_t)(gdp_event_t *gev);</code></p>
608 e2682ba9 Eric Allman
    <h4>Notes</h4>
609
    <ul>
610
      <li>This is the type of callback function as passed into the asynchronous
611
        interfaces.</li>
612
      <li>All the interesting data is encoded into the event.&nbsp; This is
613
        exactly the same data structure as returned by <code>gdp_event_next</code>.</li>
614
      <li>In all cases, if the callback function is not specified the
615
        information will be returned through the event interface.&nbsp; The two
616 10125206 Eric Allman
        interfaces are related but mutually exclusive.</li>
617 e2682ba9 Eric Allman
    </ul>
618 d9827038 Eric Allman
    <br>
619 10125206 Eric Allman
    <hr>
620
    <h4>Name</h4>
621 eae1d3ec Eric Allman
    gdp_gin_read_by_recno_async, gdp_gin_read_by_ts_async,
622
    gdp_gin_read_by_hash_async &mdash; Asynchronously read records from a
623
    readable GOB
624
    <h4> Synopsis</h4>
625
    <pre>typedef void (*gdp_event_cbfunc_t)(gdp_event_t *gev)
626
<br>EP_STAT gdp_gin_read_by_recno_async(<br>                gdp_gin_t *gin,
627
                gdp_recno_t start,<br>                int32_t numrecs,<br>                gdp_event_cbfunc_t cbfunc,<br>                void *udata)<br>EP_STAT gdp_gin_read_by_ts_async(<br>                gdp_gin_t *gin,
628 4217d271 Eric Allman
                EP_TIME_SPEC *start,<br>                int32_t numrecs,<br>                gdp_event_cbfunc_t cbfunc,<br>                void *udata)<br>EP_STAT gdp_gin_read_by_hash_async(<br>                gdp_gin_t *gin,<br>                int32_t n_hashes,<br>                gdp_hash_t **hashes,<br>                gdp_event_cbfunc_t cbfunc,<br>                void *udata)
629 eae1d3ec Eric Allman
</pre>
630
    <h4> Notes</h4>
631 10125206 Eric Allman
    <ul>
632 eae1d3ec Eric Allman
      <li>Initializes an asynchronous read of data.</li>
633
      <li>These functions return before any result is read and thus do not
634 10125206 Eric Allman
        include the final status information.</li>
635
      <li>The return status will be <code>GDP_STAT_OK</code> if the read
636
        command is successfully sent, and a later callback or event will give
637
        the actual status; otherwise no callback or event will occur.</li>
638
      <li>Status is returned through the event interface (if <code>cbfunc</code>
639
        is <code>NULL</code>) or through <code>cbfunc</code>.</li>
640 eae1d3ec Eric Allman
      <li>Similar to a subscription, except data in the future is never read
641
        (i.e., this is only for reading historic data).&nbsp; This is the
642
        interface to use for asynchronous reads.</li>
643 26fa111b Eric Allman
      <li>If a <code>cbfunc</code> is specified, arranges to call callback when
644 eae1d3ec Eric Allman
        a message is generated on the <code>gin</code>.&nbsp; See below for the
645
        definition of <code>gdp_event_t</code>. </li>
646 26fa111b Eric Allman
      <li> The callback is not necessarily invoked instantly, and may or may not
647
        be called in a separate thread.</li>
648 eae1d3ec Eric Allman
      <li>It is the responsibility of the callback function to call <code>gdp_event_free(gev)</code>.<br>
649 26fa111b Eric Allman
      </li>
650
      <li>If no <code>cbfunc</code> is specified, subscription information is
651 eae1d3ec Eric Allman
        available through the <code>gdp_event</code> interface (see below).</li>
652 26fa111b Eric Allman
      <li> The <code>udata</code> is passed through untouched in generated
653
        events.&nbsp; See below for the definition of <code>gdp_event_t</code>.</li>
654
      <li>At most <code>numrecs</code> records are returned, after which the
655 eae1d3ec Eric Allman
        read is terminated.&nbsp; If <code>numrecs</code> is 0 it reads to the
656
        end of the data.&nbsp; Since hash values must be unique and are
657
        unordered, <code>gdp_gin_read_by_hash_async</code> always returns at
658
        most one record.</li>
659 26fa111b Eric Allman
      <li> The <code>start</code> parameter tells when to start the
660 eae1d3ec Eric Allman
        subscription (that is, the starting record number or timestamp).</li>
661
      <li>For <code>gdp_gin_read_by_recno_async</code>, if <code>start</code>
662
        is negative, returns the most recent &ndash;<code>start</code>
663 26fa111b Eric Allman
        records.&nbsp; If a negative <code>start</code> indicates going back
664 eae1d3ec Eric Allman
        more records than are available, it starts from the first record.&nbsp;
665
        If <code>start</code> is zero, an error is returned. </li>
666 26fa111b Eric Allman
      <li>If <code>start</code> specifies an existing record but there are
667 eae1d3ec Eric Allman
        fewer than <code>numrecs</code> records available, only the existing
668
        records are returned.</li>
669
      <li>If there are multiple records matching a <code>recno</code> or <code>timestamp</code>,
670
        all of them are returned.</li>
671 26fa111b Eric Allman
      <li><em>Callbacks make binding to languages like Java particularly
672 eae1d3ec Eric Allman
          difficult</em><em>, but fit more naturally in with languages such as
673
          Javascript.</em><em>&nbsp; Note also that callbacks will generally run
674
          in the GDP I/O thread (so no further GDP operations will run until the
675 26fa111b Eric Allman
          callback completes) or in a separate thread (in which case several
676 eae1d3ec Eric Allman
          instances of the callback may be run at once).</em></li>
677
      <li>Example for <code>gdp_gin_read_by_recno_async</code>:&nbsp; suppose
678
        there are 20 records already in a GOB.&nbsp; Then:</li>
679 26fa111b Eric Allman
    </ul>
680 80da996e Eric Allman
    <table cellspacing="2" cellpadding="2" border="1" align="center" width="80%">
681 26fa111b Eric Allman
      <tbody>
682
        <tr>
683 a2d7923a Eric Allman
          <td valign="top">start </td>
684
          <td valign="top">numrecs </td>
685
          <td valign="top">Behavior </td>
686 26fa111b Eric Allman
        </tr>
687
        <tr>
688 a2d7923a Eric Allman
          <td valign="top">1 </td>
689
          <td valign="top">10 </td>
690 26fa111b Eric Allman
          <td valign="top">Returns records 1&ndash;10 immediately and terminates
691 eae1d3ec Eric Allman
            the read. </td>
692 26fa111b Eric Allman
        </tr>
693
        <tr>
694 a2d7923a Eric Allman
          <td valign="top">&ndash;10 </td>
695
          <td valign="top">10 </td>
696 26fa111b Eric Allman
          <td valign="top">Returns records 11&ndash;20 immediately and
697 eae1d3ec Eric Allman
            terminates the read. </td>
698 26fa111b Eric Allman
        </tr>
699
        <tr>
700 a2d7923a Eric Allman
          <td valign="top">0 </td>
701 eae1d3ec Eric Allman
          <td valign="top">any </td>
702
          <td valign="top">Returns "4.02 bad option" failure. </td>
703 26fa111b Eric Allman
        </tr>
704
        <tr>
705 a2d7923a Eric Allman
          <td valign="top">&ndash;10 </td>
706
          <td valign="top">20 </td>
707 eae1d3ec Eric Allman
          <td valign="top">Returns records 11&ndash;20. </td>
708 26fa111b Eric Allman
        </tr>
709
        <tr>
710 a2d7923a Eric Allman
          <td valign="top">1 </td>
711 26fa111b Eric Allman
          <td valign="top">0</td>
712 eae1d3ec Eric Allman
          <td valign="top">Returns records 1&ndash;20 immediately. </td>
713 26fa111b Eric Allman
        </tr>
714
        <tr>
715 a2d7923a Eric Allman
          <td valign="top">&ndash;30 </td>
716
          <td valign="top">30 </td>
717 eae1d3ec Eric Allman
          <td valign="top">Returns records 1&ndash;20 immediately. </td>
718 26fa111b Eric Allman
        </tr>
719
        <tr>
720 a2d7923a Eric Allman
          <td valign="top">30 </td>
721
          <td valign="top">10 </td>
722 26fa111b Eric Allman
          <td valign="top"><i>Currently undefined.&nbsp; Should probably wait
723
              until 10 more records are added before starting to return the
724 a2d7923a Eric Allman
              data.</i> </td>
725 26fa111b Eric Allman
        </tr>
726
        <tr>
727 a2d7923a Eric Allman
          <td valign="top">any </td>
728
          <td valign="top">&ndash;1 </td>
729
          <td valign="top">Returns "4.02 bad option" failure. </td>
730 26fa111b Eric Allman
        </tr>
731
      </tbody>
732
    </table>
733
    <br>
734 eae1d3ec Eric Allman
    <br>
735 d9827038 Eric Allman
    <br>
736 10125206 Eric Allman
    <hr width="100%" size="2">
737 26fa111b Eric Allman
    <h4> Name</h4>
738 eae1d3ec Eric Allman
    <p>gdp_gin_append_async &mdash; Asynchronously append one or more records to
739
      a writable GOB</p>
740 10125206 Eric Allman
    <h4>Synopsis</h4>
741 d9827038 Eric Allman
    <pre>EP_STAT gdp_gin_append_async(<br>                gdp_gin_t *gin,<br>                int32_t n_datums<br>                gdp_datum_t **datums,<br>                gdp_hash_t *prevhash,<br>                gdp_event_cbfunc_t *cbfunc,<br>                void *udata)</pre>
742 10125206 Eric Allman
    <h4>Notes</h4>
743 eae1d3ec Eric Allman
    <ul>
744
      <li>Appends the indicated <code>datum</code> to the GOB.</li>
745
      <li>This function returns before any result is read and thus does not
746
        include the final status information.</li>
747
      <li>The return status will be <code>GDP_STAT_OK</code> if the append
748
        command is successfully sent, and a later callback or event will give
749
        the actual status; otherwise no callback or event will occur.</li>
750
      <li><em>Should a warning status be returned to make it clear that a status
751
          will be returned later?</em></li>
752
      <li>Status is returned through the event interface (if <code>cbfunc</code>
753
        is <code>NULL</code>) or through <code>cbfunc</code> with an event
754
        type of <code>GDP_EVENT_ASTAT</code>.</li>
755
    </ul>
756 10125206 Eric Allman
    <hr>
757
    <h4>Name</h4>
758 eae1d3ec Eric Allman
    gdp_gin_subscribe_by_* &mdash; Subscribe to a readable GOB
759 26fa111b Eric Allman
    <h4> Synopsis</h4>
760 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_gin_subscribe_by_recno(<br>                gdp_gin_t *gin,
761
                gdp_recno_t start,<br>                int32_t numrecs,<br>                gdp_sub_qos_t *qos;<br>                gdp_event_cbfunc_t *cbfunc,<br>                void *udata)<br>EP_STAT gdp_gin_subscribe_by_ts(<br>                gdp_gin_t *gin,
762
                EP_TIME_SPEC *start,<br>                int32_t numrecs,<br>                gdp_sub_qos_t *qos;<br>                gdp_event_cbfunc_t *cbfunc,<br>                void *udata) </pre>
763 26fa111b Eric Allman
    <h4> Notes</h4>
764
    <ul>
765
      <li>If a <code>cbfunc</code> is specified, arranges to call callback when
766 eae1d3ec Eric Allman
        a message is generated on the <code>gin</code>.</li>
767 26fa111b Eric Allman
      <li> The callback is not necessarily invoked instantly, and may or may not
768
        be called in a separate thread.</li>
769 eae1d3ec Eric Allman
      <li>It is the responsibility of the callback function to call <code>gdp_event_free(gev)</code>.
770 26fa111b Eric Allman
      </li>
771 eae1d3ec Eric Allman
      <li>If qos is specified, it contains quality of service information about
772
        the subscription &mdash; for example, whether subscriptions should be
773
        reliable or best effort (default).</li>
774 26fa111b Eric Allman
      <li>If no <code>cbfunc</code> is specified, subscription information is
775 eae1d3ec Eric Allman
        available through the <code>gdp_event</code> interface (see below).</li>
776 26fa111b Eric Allman
      <li> The <code>udata</code> is passed through untouched in generated
777
        events.&nbsp; See below for the definition of <code>gdp_event_t</code>.</li>
778
      <li>At most <code>numrecs</code> records are returned, after which the
779 eae1d3ec Eric Allman
        subscription is terminated.&nbsp; If <code>numrecs</code> is 0 it waits
780
        for data forever. </li>
781 26fa111b Eric Allman
      <li> The <code>start</code> parameter tells when to start the
782 eae1d3ec Eric Allman
        subscription (that is, the starting record number for <code>gdp_gin_subscribe_by_recno</code>
783
        or the earliest time of interest for <code>gdp_gin_subscribe_by_ts</code>).</li>
784
      <li>In <code>gdp_gin_subscribe_by_recno</code>, if <code>start</code> is
785
        negative, returns the most recent &ndash;<code>start</code>
786 26fa111b Eric Allman
        records.&nbsp; If a negative <code>start</code> indicates going back
787 a2d7923a Eric Allman
        more records than are available, it starts from the first record. </li>
788 26fa111b Eric Allman
      <li>If <code>start</code> specifies an existing record but there are
789 eae1d3ec Eric Allman
        fewer than <code>numrecs</code> records available, this returns the
790
        available records and then waits for the additional data to appear as it
791
        is published.</li>
792
      <li>In <code>gdp_gin_subscribe_by_recno</code>, if <code>start</code> is
793
        zero, or in <code>gdp_gin_subscribe_by_ts</code>, it points past the
794
        last record already in the log, no current records are returned (i.e.,
795
        it returns new records as they are published).</li>
796 26fa111b Eric Allman
      <li><em>Callbacks make binding to languages like Java particularly
797 eae1d3ec Eric Allman
          difficult, but fit more naturally in with languages such as
798
          Javascript.&nbsp; Note also that callbacks will generally run in the
799
          GDP I/O thread (so no further GDP operations will run until the
800 26fa111b Eric Allman
          callback completes) or in a separate thread (in which case several
801 eae1d3ec Eric Allman
          instances of the callback may be run at once). </em></li>
802
      <li>Example:&nbsp; suppose there are 20 records already in a GOB.&nbsp;
803 26fa111b Eric Allman
        Then:</li>
804
    </ul>
805 80da996e Eric Allman
    <table cellspacing="2" cellpadding="2" border="1" align="center" width="80%">
806 26fa111b Eric Allman
      <tbody>
807
        <tr>
808 a2d7923a Eric Allman
          <td valign="top">start </td>
809
          <td valign="top">numrecs </td>
810
          <td valign="top">Behavior </td>
811 26fa111b Eric Allman
        </tr>
812
        <tr>
813 a2d7923a Eric Allman
          <td valign="top">1 </td>
814
          <td valign="top">10 </td>
815 26fa111b Eric Allman
          <td valign="top">Returns records 1&ndash;10 immediately and terminates
816 eae1d3ec Eric Allman
            the subscription. </td>
817 26fa111b Eric Allman
        </tr>
818
        <tr>
819 a2d7923a Eric Allman
          <td valign="top">&ndash;10 </td>
820
          <td valign="top">10 </td>
821 26fa111b Eric Allman
          <td valign="top">Returns records 11&ndash;20 immediately and
822 eae1d3ec Eric Allman
            terminates the subscription. </td>
823 26fa111b Eric Allman
        </tr>
824
        <tr>
825 a2d7923a Eric Allman
          <td valign="top">0 </td>
826 eae1d3ec Eric Allman
          <td valign="top">0 </td>
827
          <td valign="top">Starts returning data when record 21 is published and
828
            continues forever. </td>
829 26fa111b Eric Allman
        </tr>
830
        <tr>
831 a2d7923a Eric Allman
          <td valign="top">&ndash;10 </td>
832
          <td valign="top">20 </td>
833 eae1d3ec Eric Allman
          <td valign="top">Returns records 11&ndash;20 immediately, then returns
834
            records 21&ndash;30 as they are published.&nbsp; The subscription
835
            then terminates. </td>
836 26fa111b Eric Allman
        </tr>
837
        <tr>
838 a2d7923a Eric Allman
          <td valign="top">1 </td>
839 26fa111b Eric Allman
          <td valign="top">0</td>
840 eae1d3ec Eric Allman
          <td valign="top">Returns records 1&ndash;20 immediately, then returns
841
            any new records published in perpetuity. </td>
842 26fa111b Eric Allman
        </tr>
843
        <tr>
844 a2d7923a Eric Allman
          <td valign="top">&ndash;30 </td>
845
          <td valign="top">30 </td>
846 eae1d3ec Eric Allman
          <td valign="top">Returns records 1&ndash;20 immediately, then returns
847
            records 21&ndash;30 as they are published. </td>
848 26fa111b Eric Allman
        </tr>
849
        <tr>
850 a2d7923a Eric Allman
          <td valign="top">30 </td>
851
          <td valign="top">10 </td>
852 26fa111b Eric Allman
          <td valign="top"><i>Currently undefined.&nbsp; Should probably wait
853
              until 10 more records are added before starting to return the
854 a2d7923a Eric Allman
              data.</i> </td>
855 26fa111b Eric Allman
        </tr>
856
        <tr>
857 a2d7923a Eric Allman
          <td valign="top">any </td>
858
          <td valign="top">&ndash;1 </td>
859
          <td valign="top">Returns "4.02 bad option" failure. </td>
860 26fa111b Eric Allman
        </tr>
861
      </tbody>
862
    </table>
863
    <br>
864 10125206 Eric Allman
    <hr width="100%" size="2">
865 26fa111b Eric Allman
    <h4> Name</h4>
866 eae1d3ec Eric Allman
    <p>gdp_sub_qos_new, gdp_sub_qos_free &mdash; allocate/free subscription
867
      quality of service information</p>
868
    <h4>Synopsis</h4>
869
    <pre>gdp_sub_qos_t *gdp_sub_qos_new(void)</pre>
870
    <pre>void *gdp_sub_qos_free(<br>                gdp_sub_qos_t *qos)</pre>
871
    <h4>Notes</h4>
872 80da996e Eric Allman
    <ul>
873
      <li class="warning">Tentative &mdash; do not use.</li>
874
    </ul>
875 eae1d3ec Eric Allman
    <hr>
876
    <h4>Name</h4>
877
    <p>gdp_sub_qos_set_<i>xyzzy</i> &mdash; set <i>xyzzy</i> qos</p>
878
    <h4>Synopsis</h4>
879
    <pre>gdp_sub_qos_set_<code>xyzzy</code>(<br>                gdp_sub_qos_t *qos,<br>                xxx yyy)</pre>
880
    <h4>Notes</h4>
881 80da996e Eric Allman
    <ul>
882
      <li class="warning">Tentative &mdash; do not use.<br>
883
      </li>
884
    </ul>
885
    <p></p>
886 eae1d3ec Eric Allman
    <hr>Name gdp_gin_unsubscribe &mdash; Unsubscribe GIN from an associated GOB<span
887
      class="warning"></span>
888 26fa111b Eric Allman
    <h4> Synopsis</h4>
889 eae1d3ec Eric Allman
    <pre>EP_STAT gdp_gin_unsubscribe(gdp_gin_t *gin,
890 728e4394 Eric Allman
                gdp_event_cbfunc_t *cbfunc,
891 e2682ba9 Eric Allman
                void *udata)
892 26fa111b Eric Allman
</pre>
893
    <h4> Notes</h4>
894
    <ul>
895 eae1d3ec Eric Allman
      <li> Deletes all subscriptions matching the given {gin, cbfunc, udata}
896 10125206 Eric Allman
        tuple.</li>
897
      <li>If <code>cbfunc</code> and or <code>udata</code> is <code>NULL</code>
898 eae1d3ec Eric Allman
        they are treated as wildcards.&nbsp; For example, "<code>gdp_gin_unsubscribe(gin,
899
          NULL, NULL)</code>" deletes all subscriptions for the given GIN.</li>
900 10125206 Eric Allman
      <li>If there are multiple subscriptions matching this tuple, they will all
901
        be terminated.&nbsp; For example, this might happen in a multi-threaded
902
        application.</li>
903
      <li><em>(Should subscribe/multiread return a handle to be passed to this
904 26fa111b Eric Allman
          function rather than this interface?)</em></li>
905
    </ul>
906
    <br>
907 10125206 Eric Allman
    <hr width="100%" size="2">
908 26fa111b Eric Allman
    <h4> Name</h4>
909 b1d346a7 Eric Allman
    gdp_event_next &mdash; get next asynchronous event
910 26fa111b Eric Allman
    <h4> Synopsis</h4>
911 eae1d3ec Eric Allman
    <pre>gdp_event_t *gdp_event_next(<br>                gdp_gin_t *gin,<br>                EP_TIME_SPEC *timeout)</pre>
912 26fa111b Eric Allman
    <h4> Notes</h4>
913
    <ul>
914 eae1d3ec Eric Allman
      <li> Returns the next asynchronous event on the specified GIN.&nbsp; If
915
        gin is NULL, returns the next event from any GIN.</li>
916 26fa111b Eric Allman
      <li>Returns <code>NULL</code> if the <code>timeout</code> expires.&nbsp;
917 a2d7923a Eric Allman
        If <code>timeout</code> is <code>NULL</code>, it waits forever. </li>
918 26fa111b Eric Allman
      <li>Currently the only asynchronous events are data arriving as the result
919
        of a subscription.</li>
920
      <li><em>(Should timeout be an absolute time or a delta on the current
921
          time?&nbsp; Currently it is implemented as a delta.)</em></li>
922
    </ul>
923 10125206 Eric Allman
    <hr width="100%" size="2">
924 26fa111b Eric Allman
    <h4>Name</h4>
925 b1d346a7 Eric Allman
    gdp_event_gettype &mdash; extract the type from the event
926 26fa111b Eric Allman
    <h4>Synopsis</h4>
927
    <pre>int gdp_event_gettype(gdp_event_t *gev)</pre>
928
    <h4>Notes</h4>
929
    <ul>
930
      <li>The event types are as follows:</li>
931
    </ul>
932 80da996e Eric Allman
    <table cellspacing="2" cellpadding="2" border="1" align="center" width="80%">
933 26fa111b Eric Allman
      <tbody>
934
        <tr>
935 8cc28925 Eric Allman
          <td style="width: 162.2px;" valign="top">Event Name </td>
936
          <td style="width: 486.6px;" valign="top">Meaning </td>
937 26fa111b Eric Allman
        </tr>
938
        <tr>
939 8cc28925 Eric Allman
          <td style="text-align: left; vertical-align: top;" valign="top"><code>GDP_EVENT_DATA</code>
940
          </td>
941
          <td style="text-align: left; vertical-align: top;" valign="top">Data
942
            is returned in the event from a previous subscription or
943
            asynchronous read. </td>
944 26fa111b Eric Allman
        </tr>
945
        <tr>
946 5ced6d34 Eric Allman
          <td style="text-align: left; vertical-align: top;" valign="top"><code>GDP_EVENT_DONE</code>
947 8cc28925 Eric Allman
          </td>
948 5ced6d34 Eric Allman
          <td style="text-align: left; vertical-align: top;" valign="top">Indicates
949
            the end of a subscription or asynchronous read. </td>
950 26fa111b Eric Allman
        </tr>
951
        <tr>
952 8cc28925 Eric Allman
          <td style="text-align: left; vertical-align: top;" valign="top"><tt>GDP_EVENT_SHUTDOWN</tt>
953
          </td>
954
          <td style="text-align: left; vertical-align: top;" valign="top">Subscription
955
            is terminated because the log daemon has shut down. </td>
956 26fa111b Eric Allman
        </tr>
957 e2682ba9 Eric Allman
        <tr>
958 8cc28925 Eric Allman
          <td style="text-align: left; vertical-align: top;"><code>GDP_EVENT_CREATED</code></td>
959
          <td style="text-align: left; vertical-align: top;">Status is returned
960
            from an asynchronous append, create, or other similar operation.</td>
961
        </tr>
962 10125206 Eric Allman
        <tr>
963
          <td style="vertical-align: top; background-color: white;"><code>GDP_EVENT_MISSING</code></td>
964
          <td style="vertical-align: top; background-color: white;">The
965
            requested data was not available at this time, but more data may be
966
            available.</td>
967
        </tr>
968
        <tr>
969 8cc28925 Eric Allman
          <td style="text-align: left; vertical-align: top;"><span style="font-family: monospace;">GDP_EVENT_SUCCESS<br>
970
            </span></td>
971
          <td style="text-align: left; vertical-align: top;">Generic
972
            asynchronous success status. See the detailed status using <span style="font-family: monospace;">gdp_event_getstat</span>.</td>
973
        </tr>
974
        <tr>
975
          <td style="text-align: left; vertical-align: top;"><span style="font-family: monospace;">GDP_EVENT_FAILURE</span></td>
976 10125206 Eric Allman
          <td style="text-align: left; vertical-align: top;">Generic
977
            asynchronous failure status.&nbsp; See the detailed status using <span
978
              style="font-family: monospace;">gdp_event_getstat</span>.</td>
979 e2682ba9 Eric Allman
        </tr>
980 26fa111b Eric Allman
      </tbody>
981
    </table>
982
    <ul>
983 a2d7923a Eric Allman
      <li>Unrecognized events should be ignored. </li>
984 26fa111b Eric Allman
    </ul>
985 10125206 Eric Allman
    <hr width="100%" size="2">
986 26fa111b Eric Allman
    <h4>Name</h4>
987 e2682ba9 Eric Allman
    <p>gdp_event_getstat &mdash; extract the detailed result status from the
988
      event</p>
989
    <h4>Synopsis</h4>
990
    <p><code>EP_STAT gdp_event_getstat(gdp_event_t *gev)</code></p>
991
    <h4>Notes</h4>
992
    <ul>
993
      <li>Returns the status code from the event.&nbsp; In most cases this will
994
        be <code>EP_STAT_OK</code>, but may be otherwise in some event types
995
        such as <code>GDP_EVENT_ASTAT</code>.</li>
996
    </ul>
997
    <hr>
998
    <h4>Name</h4>
999 eae1d3ec Eric Allman
    gdp_event_getgin &mdash; extract the GIN handle from the event
1000 26fa111b Eric Allman
    <h4>Synopsis</h4>
1001 eae1d3ec Eric Allman
    <pre>gdp_gin_t *gdp_event_getgin(gdp_event_t *gev)</pre>
1002 26fa111b Eric Allman
    <h4>Notes</h4>
1003
    <ul>
1004 eae1d3ec Eric Allman
      <li>Returns the GIN handle that triggered this event.</li>
1005 26fa111b Eric Allman
    </ul>
1006 10125206 Eric Allman
    <hr width="100%" size="2">
1007 26fa111b Eric Allman
    <h4>Name</h4>
1008 b1d346a7 Eric Allman
    gdp_event_getdatum &mdash; get the datum associated with this event
1009 26fa111b Eric Allman
    <h4>Synopsis</h4>
1010
    <pre>gdp_datum_t *gdp_event_getdatum(gdp_event_t *gev)</pre>
1011
    <h4>Notes</h4>
1012
    <ul>
1013
      <li>Returns the data associated with the event.</li>
1014
    </ul>
1015 10125206 Eric Allman
    <hr width="100%" size="2">
1016 26fa111b Eric Allman
    <h4>Name</h4>
1017 a2d7923a Eric Allman
    <p>gdp_event_getudata &mdash; get user data associated with this event </p>
1018 26fa111b Eric Allman
    <h4>Synopsis</h4>
1019 b1d346a7 Eric Allman
    <pre>void *gdp_event_getudata(gdp_event_t *gev)
1020
    </pre>
1021 26fa111b Eric Allman
    <h4>Notes</h4>
1022
    <ul>
1023
      <li>Returns user data associated with the event.&nbsp; The user data is
1024 a2d7923a Eric Allman
        from the call that initiated the asynchronous operation. </li>
1025 26fa111b Eric Allman
    </ul>
1026 7d83fa37 Eric Allman
    <hr>
1027 eae1d3ec Eric Allman
    <h3> 3.6&nbsp; Utilities </h3>
1028
    <hr width="100%" size="2">
1029 7d83fa37 Eric Allman
    <h4>Name</h4>
1030 80da996e Eric Allman
    gdp_name_parse, gdp_printable_name &mdash; parse an external representation
1031 eae1d3ec Eric Allman
    to internal, create printable version of name<br>
1032 7d83fa37 Eric Allman
    <h4>Synopsis</h4>
1033 80da996e Eric Allman
    <pre>EP_STAT gdp_name_parse(<br>                const char *external_name,<br>               &nbsp;gdp_name_t gob_name,<br>                char **extended_name)<br><br>char *gdp_printable_name(const gdp_name_t gob_name,<br>                gdp_pname_t printable)</pre>
1034 7d83fa37 Eric Allman
    <h4>Notes</h4>
1035
    <ul>
1036 eae1d3ec Eric Allman
      <li>The <code>gdp_parse_name</code> function converts an external string
1037
        representation of a GOB name to an internal 256-bit encoding.</li>
1038 80da996e Eric Allman
      <li>If <code>external_name</code> is a URI-base64-encoded string of the
1039
        correct length, it is converted directly to 256 bits.</li>
1040
      <li>Otherwise, it is looked up in the Human-Oriented Name to GDPname
1041
        Directory.</li>
1042
      <li>If the <code>GDP_NAME_ROOT</code> environment variable is set, it can
1043
        be used when looking up the name.&nbsp; If the input name has no dots
1044
        then the root name will be prepended to the name before lookup.&nbsp;
1045
        For example, if <code>GDP_NAME_ROOT=edu.berkeley.eecs.eric</code> and
1046
        the <code>external_name</code> is "<code>test</code>" then <code>gdp_name_parse</code>
1047
        will search for "<code>edu.berkeley.eecs.eric.test</code>" in the
1048
        directory.</li>
1049
      <li>If <code>extended_name</code> is non-<code>NULL</code>, <code>*extended_name</code>
1050
        will be used to store a pointer to the name actually used.&nbsp; This
1051
        will be the same as <code>external_name</code> unless name extension is
1052
        performed.&nbsp; If the status is not <code>EP_STAT_OK</code>, <code>*extended_name</code>
1053
        will be set to <code>NULL</code>.&nbsp; This memory is dynamically
1054
        allocated; the application is responsible for freeing the memory when it
1055
        is done with it.</li>
1056 eae1d3ec Eric Allman
      <li>The <code>gdp_printable_name</code> function converts an internal
1057 80da996e Eric Allman
        name to a URI-base64-encoded string &mdash; note, not the original
1058
        human-oriented name.</li>
1059
    </ul>
1060
    <hr>
1061
    <h4>Name</h4>
1062
    <p>gdp_name_root_set, gdp_name_root_get &mdash; set/get the root name used
1063
      by gdp_name_parse</p>
1064
    <h4>Synopsis</h4>
1065
    <pre>EP_STAT gdp_name_root_set(<br>                const char *root_name)<br>const char *gdp_name_root_get(void)</pre>
1066
    <h4>Notes</h4>
1067
    <ul>
1068
      <li>The root name used by <code>gdp_name_parse</code> can be set or
1069
        queried using these routines.</li>
1070
      <li class="warning">It's possible that the name root may be converted to a
1071
        colon-separated path in the future.&nbsp; Caution is advised.</li>
1072
    </ul>
1073
    <ul>
1074 7d83fa37 Eric Allman
    </ul>
1075
    <hr>
1076
    <h4>Name</h4>
1077 eae1d3ec Eric Allman
    gdp_gin_getname &mdash; Return the name of a GOB from a GIN
1078
    <h4> Synopsis</h4>
1079
    <pre>EP_STAT gdp_gin_getname(gdp_gin_t *gin,
1080
                gdp_name_t namebuf)
1081
</pre>
1082
    <h4> Notes</h4>
1083 7d83fa37 Eric Allman
    <ul>
1084 eae1d3ec Eric Allman
      <li> Returns the internal name of the GOB referenced by <code>gin</code>
1085
        into <code> namebuf</code>. </li>
1086
      <li> May not be necessary if the creator provides the name; really only
1087
        intended after <code>gdp_gin_create</code> so that the name can be
1088
        shared to other nodes that want to <code>gdp_gin_open</code> it.</li>
1089 7d83fa37 Eric Allman
    </ul>
1090 eae1d3ec Eric Allman
    <br>
1091
    <hr width="100%" size="2">
1092
    <h4> Name</h4>
1093 b364b308 Eric Allman
    gdp_getstat &mdash; Return information about a GIN&nbsp; <span class="warning">[NOT
1094 eae1d3ec Eric Allman
      YET IMPLEMENTED] </span>
1095
    <h4> Synopsis</h4>
1096
    <pre>EP_STAT gdp_getstat(gdp_name_t gobname,
1097
                gob_stat_t *statbuf)
1098
</pre>
1099
    <h4> Notes</h4>
1100
    <ul>
1101
      <li> <i>What status is included? Size (or number of records/messages),
1102
          last access, &hellip;</i></li>
1103
    </ul>
1104
    <br>
1105 e95d2be4 Eric Allman
    <hr>
1106
    <h4>Name</h4>
1107
    <p>gdp_gin_getnrecs &mdash; return the number of records in an existing GOB</p>
1108
    <h4>Synopsis</h4>
1109
    <p><code>gdp_recno_t gdp_gin_getnrecs(gdp_gin_t *gin)</code></p>
1110
    <h4>Notes</h4>
1111
    <ul>
1112
      <li>Returns the number of records in a GOB.</li>
1113
      <li>This does not check with the log server, i.e., the return value is
1114
        cached in the local process (and is updated each time the GOB is read or
1115
        written).&nbsp; This shouldn't be a problem for writers (since there
1116
        should only be one writer), but may be for readers.&nbsp; Readers
1117
        wanting to get the latest number of records can read the last record in
1118
        the GOB (using <code>gdp_gin_read_by_recno</code>) and check the record
1119
        number in the returned datum.</li>
1120
    </ul>
1121
    <hr>
1122
    <h4>Name</h4>
1123 9f6be413 Eric Allman
    gdp_gin_gethashalg &mdash; get the hash algorithm used by a GOB
1124
    <h4>Synopsis</h4>
1125
    <pre>    int gdp_gin_gethashalg(const gdp_gin_t *gin)</pre>
1126
    <h4>Notes</h4>
1127
    <ul>
1128
      <li>Returns the type of the hash (a.k.a. message digest) algorithm.</li>
1129
      <li>The values are defined by the libep.&nbsp; They consist of the major
1130
        SHA algorithms in various sizes, e.g., <span class="manifest">EP_CRYPTO_MD_SHA1</span>,
1131
        <span class="manifest">EP_CRYPTO_MD_SHA224</span>, etc. through <span class="manifest">SHA512</span>.</li>
1132
    </ul>
1133
    <hr>
1134
    <h4>Name</h4>
1135
    gdp_gin_getsigalg &mdash; get the signature algorithm used by a GOB
1136
    <h4>Synopsis</h4>
1137
    <pre>    int gdp_gin_getsigalg(const gdp_gin_t *gin)</pre>
1138
    <h4>Notes</h4>
1139
    <ul>
1140
      <li>Returns the type of the signature algorithm.</li>
1141
      <li>The values are defined by the libep.</li>
1142
    </ul>
1143
    <hr>
1144
    <h4>Name</h4>
1145 e95d2be4 Eric Allman
    <p>gdp_gin_set_append_filter &mdash; filter appended data</p>
1146
    <h4>Synopsis</h4>
1147
    <code>void gdp_gin_set_append_filter(gdp_gin_t *gin,<br>
1148
      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
1149
      EP_STAT (*filter(gdp_datum_t *, void *),<br>
1150
      &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; void *filterdata)</code><br>
1151
    <h4>Notes</h4>
1152
    <ul>
1153
      <li>All data appended to the named <span class="variable">gin</span> are
1154
        first run through the <span class="variable">filter</span>.&nbsp; The
1155
        filter may modify the <span class="variable">datum</span> before it is
1156
        sent.</li>
1157
      <li>The <span class="variable">filterdata</span> parameter is passed as
1158
        the second argument to <span class="variable">filter</span>.</li>
1159
    </ul>
1160 80da996e Eric Allman
    <hr>
1161 e95d2be4 Eric Allman
    <h4>Name</h4>
1162
    <p>gdp_gin_set_read_filter &mdash; filter read data</p>
1163
    <h4>Synopsis</h4>
1164
    <p><code>void gdp_gin_set_read_filter(gdp_gin_t *gin,<br>
1165
        &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
1166
        EP_STAT (*filter(gdp_datum_t *, void *),<br>
1167
        &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; void *filterdata)</code></p>
1168
    <h4>Notes</h4>
1169
    <ul>
1170
      <li>All data read from the named <span class="variable">gin</span> are
1171
        passed through this filter before being returned.&nbsp; The filter may
1172
        modify the <span class="variable">datum</span>.</li>
1173
      <li>The <span class="variable">filterdata</span> parameter is passed as
1174
        the second argument to <span class="variable">filter</span>.</li>
1175
    </ul>
1176 d9827038 Eric Allman
    <br>
1177 eae1d3ec Eric Allman
    <hr width="100%" size="2">
1178
    <h4> Name</h4>
1179
    gdp_gin_print &mdash; print a GIN and associated GOB (for debugging)
1180 7d83fa37 Eric Allman
    <h4>Synopsis</h4>
1181 eae1d3ec Eric Allman
    <pre>void gdp_gin_print(const gdp_gin_t *gin, FILE *fp)</pre>
1182 7d83fa37 Eric Allman
    <h4>Notes</h4>
1183
    <ul>
1184 eae1d3ec Eric Allman
      <li>Prints the GIN and GOB on the indicated file.</li>
1185
      <li>The output will contain internal information; it is not intended to
1186
        display information to end users.</li>
1187 7d83fa37 Eric Allman
    </ul>
1188 80da996e Eric Allman
    <hr>
1189 7d83fa37 Eric Allman
    <h2>4&nbsp; Datums (Records)</h2>
1190 eae1d3ec Eric Allman
    <p>GOBs are represented as a series of records of type <code>gdp_datum_t</code>.&nbsp;
1191 c8e97446 Eric Allman
      Each record has a record number, a commit timestamp, associated data, and
1192
      possible signature information if the record was signed.&nbsp; Record
1193
      numbers are of type <code>gdp_recno_t</code> and count up by one as
1194 b364b308 Eric Allman
      records are added (i.e., record numbers are unique within a GOB and
1195 b1d346a7 Eric Allman
      dense).&nbsp; Data is represented in dynamic buffers, as described below.
1196 26fa111b Eric Allman
    </p>
1197 7d83fa37 Eric Allman
    <h3>4.1&nbsp; Datum Headers</h3>
1198 10125206 Eric Allman
    <hr width="100%" size="2">
1199 26fa111b Eric Allman
    <h4>Name</h4>
1200 5ced6d34 Eric Allman
    <p>gdp_datum_new, gdp_datum_reset, gdp_datum_free, gdp_datum_print &mdash;
1201 a2d7923a Eric Allman
      allocate/free/print a datum structure </p>
1202 26fa111b Eric Allman
    <h4>Synopsis</h4>
1203 5ced6d34 Eric Allman
    <pre>gdp_datum_t *gdp_datum_new(void)<br>void gdp_datum_reset(gdp_datum_t *datum)<br>void gdp_datum_free(gdp_datum_t *datum)
1204 b1d346a7 Eric Allman
void gdp_datum_print(const gdp_datum_t *datum,
1205
                FILE *fp,
1206
                uint32_t flags)</pre>
1207 26fa111b Eric Allman
    <h4>Notes</h4>
1208
    <ul>
1209
      <li><code>gdp_datum_new</code> allocates a new empty datum.</li>
1210 5ced6d34 Eric Allman
      <li><code>gdp_datum_reset</code> resets a datum to the same state as a
1211
        newly created datum.</li>
1212 26fa111b Eric Allman
      <li><code>gdp_datum_free</code> frees a datum.</li>
1213
      <li><code>gdp_datum_print</code> writes a description of the datum
1214 d63b8179 Eric Allman
        (including the data contents) to the given file.&nbsp; If flags includes
1215 808a66bf Eric Allman
        the <span class="manifest">GDP_DATUM_PRTEXT</span> bit, it shows the
1216
        datum as plain text (the default shows it as a hex dump).&nbsp; It is up
1217
        to the caller to determine that the datum is printable.&nbsp; If the <span
1218 10125206 Eric Allman
          class="manifest">GDP_DATUM_PRSIG</span> bit is set, signature
1219 808a66bf Eric Allman
        information is included.&nbsp; If the <span class="manifest">GDP_DATUM_PRDEBUG</span>
1220
        flag is set, additional information about the datum is printed.</li>
1221 26fa111b Eric Allman
    </ul>
1222 10125206 Eric Allman
    <hr width="100%" size="2">
1223 26fa111b Eric Allman
    <h4>Name</h4>
1224 b1d346a7 Eric Allman
    gdp_datum_getrecno &mdash; get the record number from a datum
1225 26fa111b Eric Allman
    <h4>Synopsis</h4>
1226
    <pre>    <tt>gdp_recno_t gdp_datum_getrecno(const gdp_datum_t *datum)
1227
</tt></pre>
1228 a2d7923a Eric Allman
    <h4>Notes </h4>
1229 10125206 Eric Allman
    <hr width="100%" size="2">
1230 26fa111b Eric Allman
    <h4>Name</h4>
1231 b1d346a7 Eric Allman
    gdp_datum_getts &mdash; get the timestamp from a datum
1232 26fa111b Eric Allman
    <h4>Synopsis</h4>
1233
    <pre>    void gdp_datum_getts(const gdp_datum_t *datum, EP_TIME_SPEC *ts)
1234
</pre>
1235
    <h4>Notes</h4>
1236 10125206 Eric Allman
    <hr width="100%" size="2">
1237 26fa111b Eric Allman
    <h4>Name</h4>
1238 b1d346a7 Eric Allman
    gdp_datum_getdlen &mdash; get the data length from a datum
1239 808a66bf Eric Allman
    <h4>Synopsis</h4>
1240
    <pre>    size_t gdp_datum_getdlen(const gdp_datum_t *datum)</pre>
1241
    <h4>Notes</h4>
1242 10125206 Eric Allman
    <hr width="100%" size="2">
1243 808a66bf Eric Allman
    <h4>Name</h4>
1244 b1d346a7 Eric Allman
    gdp_datum_getdbuf &mdash; get the data buffer from a datum
1245 26fa111b Eric Allman
    <h4>Synopsis</h4>
1246
    <pre>    gdp_buf_t *gdp_datum_getdbuf(const gdp_datum_t *datum)</pre>
1247
    <h4>Notes</h4>
1248 10125206 Eric Allman
    <hr width="100%" size="2">
1249 808a66bf Eric Allman
    <h4>Name</h4>
1250 b1d346a7 Eric Allman
    gdp_datum_getsig &mdash; get the signature from a datum
1251 808a66bf Eric Allman
    <h4>Synopsis</h4>
1252 e95d2be4 Eric Allman
    <pre>    gdp_sig_t *gdp_datum_getsig(const gdp_datum_t *datum)</pre>
1253 808a66bf Eric Allman
    <h4>Notes</h4>
1254
    <ul>
1255
      <li>Can return <span class="manifest">NULL</span> or an empty buffer if
1256
        there is no signature.</li>
1257
    </ul>
1258 10125206 Eric Allman
    <hr width="100%" size="2">
1259 7d83fa37 Eric Allman
    <h3>4.2&nbsp; Data Buffers</h3>
1260 a2d7923a Eric Allman
    <p>Data buffers grow dynamically as needed. </p>
1261 10125206 Eric Allman
    <hr width="100%" size="2">
1262 26fa111b Eric Allman
    <h4>Name</h4>
1263
    gdp_buf_new, gdp_buf_reset, gdp_buf_free &mdash; allocate, reset, or free a
1264 b1d346a7 Eric Allman
    buffer
1265 26fa111b Eric Allman
    <h4>Synopsis</h4>
1266
    <pre>gdp_buf_t *gdp_buf_new(void)<br>void gdp_buf_reset(gdp_buf_t *b)<br>void gdp_buf_free(gdp_buf_t *b)</pre>
1267 b1d346a7 Eric Allman
    <h4>Notes</h4>
1268 26fa111b Eric Allman
    <ul>
1269
      <li><code>gdp_buf_new</code> creates a new, empty buffer.</li>
1270
      <li><code>gdp_buf_reset</code> clears the buffer, leaving it in the same
1271
        condition as when it was first created.</li>
1272
      <li><code>gdp_buf_free</code> frees the buffer.&nbsp; It must not be used
1273
        again after being freed.</li>
1274
    </ul>
1275 10125206 Eric Allman
    <hr width="100%" size="2">
1276 26fa111b Eric Allman
    <h4>Name</h4>
1277 b1d346a7 Eric Allman
    gdp_buf_getlength &mdash; return the length of the data in the buffer
1278 26fa111b Eric Allman
    <h4>Synopsis</h4>
1279
    <pre>size_t gdp_buf_getlength(gdp_buf_t *b)</pre>
1280
    <h4>Notes</h4>
1281
    <ul>
1282
      <li>Returns the number of bytes of data currently in the buffer.</li>
1283
    </ul>
1284 10125206 Eric Allman
    <hr width="100%" size="2">
1285 26fa111b Eric Allman
    <h4>Name</h4>
1286
    gdp_buf_read, gdp_buf_peek, gdp_buf_drain &mdash; remove or peek at data in
1287 b1d346a7 Eric Allman
    a buffer
1288 26fa111b Eric Allman
    <h4>Synopsis</h4>
1289
    <pre>size_t gdp_buf_read(gdp_buf_t *b, void *out, size_t sz)<br>size_t gdp_buf_peek(gdp_buf_t *b, void *out, size_t sz)<br>int gdp_buf_drain(gdp_buf_t *b, size_t sz)</pre>
1290
    <h4>Notes</h4>
1291
    <ul>
1292
      <li>Data can be consumed from the buffer by calling <code>gdp_buf_read</code>;
1293 d9827038 Eric Allman
        data is copied into a memory area and removed from the buffer.</li>
1294 26fa111b Eric Allman
      <li>Applications can "peek" at the buffer using <code> gdp_buf_peek</code>.&nbsp;
1295
        This is identical to <code>gdp_buf_read</code> except that the data
1296
        remains in the buffer.</li>
1297
      <li>Applications can discard data from the buffer using <code>
1298
          gdp_buf_drain</code>.</li>
1299
      <li>In all cases, <code>sz</code> is the number of bytes to copy out
1300
        and/or discard.</li>
1301
    </ul>
1302 10125206 Eric Allman
    <hr width="100%" size="2">
1303 26fa111b Eric Allman
    <h4>Name</h4>
1304 b1d346a7 Eric Allman
    gdp_buf_write, gdp_buf_printf &mdash; copy data into a buffer
1305 26fa111b Eric Allman
    <h4>Synopsis</h4>
1306
    <pre>int gdp_buf_write(gdp_buf_t *b, void *in, size_t sz)<br>int gdp_buf_printf(gdp_buf_t *b, const char *fmt, ...)</pre>
1307
    <h4>Notes</h4>
1308
    <ul>
1309 5ced6d34 Eric Allman
      <li>These routines append bytes into the named buffer.</li>
1310 26fa111b Eric Allman
      <li><code>gdp_buf_write</code> copies <code>sz</code> bytes into the
1311
        buffer from the memory area in and returns 0 on success or &ndash;1 on
1312 a2d7923a Eric Allman
        failure. </li>
1313 26fa111b Eric Allman
      <li><code>gdp_buf_printf</code> essentially does a "printf" into the
1314 a2d7923a Eric Allman
        buffer and returns the number of bytes appended. </li>
1315 26fa111b Eric Allman
    </ul>
1316 10125206 Eric Allman
    <hr width="100%" size="2">
1317 26fa111b Eric Allman
    <h4>Name</h4>
1318 e76d01a5 Eric Allman
    gdp_buf_move &mdash; move data from one buffer into another
1319 26fa111b Eric Allman
    <h4>Synopsis</h4>
1320 e76d01a5 Eric Allman
    <pre>int gdp_buf_move(gdp_buf_t *ob, gdp_buf_t *ib, size_t sz)</pre>
1321 26fa111b Eric Allman
    <h4>Notes</h4>
1322
    <ul>
1323 10125206 Eric Allman
      <li>Appends the first <code>sz</code> bytes of <code>ib</code> to the
1324
        end of <code>ob</code>.</li>
1325 26fa111b Eric Allman
      <li>This is more efficient than using <code>gdp_buf_read</code> and <code>gdp_buf_write</code>.</li>
1326
    </ul>
1327 10125206 Eric Allman
    <hr width="100%" size="2">
1328 26fa111b Eric Allman
    <h4>Name</h4>
1329 b1d346a7 Eric Allman
    gdp_buf_dump &mdash; print the contents of the buffer for debugging
1330 26fa111b Eric Allman
    <h4>Synopsis</h4>
1331
    <pre>void gdp_buf_dump(gdp_buf_t *b, FILE *fp)</pre>
1332
    <h4>Notes</h4>
1333
    <ul>
1334
      <li>Prints the contents of buffer b to the file fp.</li>
1335
      <li>This is not intended for end user presentation.</li>
1336
    </ul>
1337 10125206 Eric Allman
    <hr width="100%" size="2">
1338 7d83fa37 Eric Allman
    <h3>4.3&nbsp; Timestamps</h3>
1339 26fa111b Eric Allman
    <p>The time abstraction is imported directly from the ep library.&nbsp;
1340 a2d7923a Eric Allman
      Times are represented as follows: </p>
1341 26fa111b Eric Allman
    <blockquote>
1342 d9827038 Eric Allman
      <pre>#pragma pack(push, 1)
1343 26fa111b Eric Allman
typedef struct
1344
{
1345 b1d346a7 Eric Allman
     int64_t        tv_sec;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; // seconds since January 1, 1970
1346 d9827038 Eric Allman
    &nbsp;uint32_t&nbsp;&nbsp; tv_nsec;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; // nanoseconds<br>     float&nbsp;&nbsp;&nbsp;&nbsp;  tv_accuracy;&nbsp;&nbsp;&nbsp; // accuracy in seconds<br>} EP_TIME_SPEC;<br>#pragma pack(pop)</pre>
1347 26fa111b Eric Allman
    </blockquote>
1348 a2d7923a Eric Allman
    Note that the host system <code>struct timespec</code> may not match this
1349
    structure; some systems still represent the time with only four bytes for <code>tv_sec</code>,
1350
    which expires in 2038.&nbsp; The <code>tv_accuracy</code> field indicates
1351
    an estimate for how accurate the clock is; for example, if you are running
1352
    NTP this value is likely to be on the order of a few tens to a few hundreds
1353
    of milliseconds, but if you set your clock manually it is likely to be
1354
    several seconds or worse.
1355 7d83fa37 Eric Allman
    <h2>5&nbsp; Signing and Encryption</h2>
1356
    <h3> 5.1&nbsp; Signing</h3>
1357 19bf6370 Eric Allman
    <p>Each log should have a public key in the metadata which is used to verify
1358
      writes to the log.&nbsp; The library hides most of the details of this,
1359
      but some still appear.</p>
1360 d9827038 Eric Allman
    <p> The <span class="command">gdp-create</span> command automatically
1361 19bf6370 Eric Allman
      creates a public/secret keypair unless otherwise specified.&nbsp; See the
1362
      man page for details.&nbsp; The public part of the key is inserted into
1363
      the log metadata and stored with the log.&nbsp; The secret part is stored
1364
      somewhere on your local filesystem, typically <span class="filename">KEYS/<span
1365 b364b308 Eric Allman
          class="variable">gob-id</span>.pem</span>.&nbsp; Normally <span class="command">gdp-create</span>
1366 19bf6370 Eric Allman
      will encrypt the secret key with another key entered from the command
1367
      line, although this can also be turned off.</p>
1368 eae1d3ec Eric Allman
    <p>When a GDP application attempts to open a log using <span class="function">gdp_gin_open</span>,
1369 0b99b3fa Eric Allman
      the library will attempt to find a secret key by searching the directories
1370
      named in the <span class="admin-param">swarm.gdp.crypto.key.path</span>
1371
      administrative parameter for a file having the same name as the log (with
1372
      a <span class="filename">.pem</span> file suffix).&nbsp; If that secret
1373
      key is encrypted, the library will prompt the (human) user for the secret
1374
      key password.&nbsp; The default path is "<span class="filename">.</span>",
1375
      "<span class="filename">KEYS</span>", "<span class="filename">~/.swarm/gdp/keys</span>",
1376
      "<span class="filename">/usr/local/etc/swarm/gdp/keys</span>", and "<span
1377 10125206 Eric Allman
        class="filename">/etc/swarm/gdp/keys</span>".</p>
1378 19bf6370 Eric Allman
    <p>Once the secret key has been located and decrypted, all further append
1379
      requests will be signed using the secret key and verified by the log
1380
      daemon against the public key in the log metadata.</p>
1381 7d83fa37 Eric Allman
    <h3>5.2&nbsp; Encryption</h3>
1382 b1d346a7 Eric Allman
    <p>Encryption is explicitly not part of the GDP.&nbsp; Ideally the GDP will
1383 d9827038 Eric Allman
      never see unencrypted data.&nbsp; However, read and write filters (see <code>gdp_gin_set_append_filter</code>
1384
      and <code>gdp_gin_set_append_filter</code> for details) can be used to
1385
      set encryption and decryption hooks for externally implemented
1386
      encryption.&nbsp; <span class="meta">We need to make this easier.</span></p>
1387 808a66bf Eric Allman
    <br>
1388 9c59e638 Eric Allman
    <hr>
1389
    <h3>To be done</h3>
1390 26fa111b Eric Allman
    <p>Header files<br>
1391
      Version info<br>
1392
      PRIgdp_recno macro<br>
1393
    </p>
1394 7d83fa37 Eric Allman
    <h2>Appendix A:&nbsp; Examples</h2>
1395 d9827038 Eric Allman
    <p class="meta">The following examples have not been validated with the v2
1396
      API, so they may be inaccurate.</p>
1397 6e2cccae Eric Allman
    <p>The following pseudo-code example excerpts from <tt>apps/gdp-writer.c</tt>.
1398 26fa111b Eric Allman
    </p>
1399 eae1d3ec Eric Allman
    <pre>#include &lt;gdp/gdp.h&gt;<br>#include &lt;stdio.h&gt;<br>#include &lt;string.h&gt;<br><br>int main(int argc, char **argv)<br>{<br>        gdp_gin_t *gin;<br>        EP_STAT estat;<br>        gdp_name_t gobiname;        // internal name of GOB<br>        gdp_datum_t *d;<br><br>        // general startup and initialization<br>        if (argc &lt; 2)<br>                usage_error();<br>        estat = gdp_init();<br>        if (!EP_STAT_ISOK(estat))<br>                initialization_error(estat);<br>        d = gdp_datum_new();
1400
<br><br>        // parse command line name to internal format<br>        estat = gdp_parse_name(argv[1], gobiname);<br>        if (!EP_STAT_ISOK(estat))<br>                name_syntax_error();<br><br>        // attempt to create that name<br>        estat = gdp_gin_create(gobiname, &amp;gin);<br>        if (!EP_STAT_ISOK(estat))<br>                creation_error(estat);<br><br>        // read lines from standard input<br>        while (fgets(buf, sizeof buf, stdin) != NULL)<br>        {<br>                char *p = strchr(buf, '\n');<br>                if (p != NULL)<br>                        *p = '\0';<br><br>                // write them to the dataplane<br>                if (gdp_buf_write(gdp_datum_getbuf(d), buf, strlen(buf)) &lt; 0)<br>                        estat = GDP_STAT_BUFFER_FAILURE;<br>                else<br>                        estat = gdp_gin_append(gin, d);<br>                EP_STAT_CHECK(estat, break);<br>        }<br><br>        // cleanup and exit<br>        gdp_gin_close(gin);<br>        exit(!EP_STAT_ISOK(estat));<br>}</pre>
1401 10125206 Eric Allman
    <hr width="100%" size="2">
1402 6e2cccae Eric Allman
    <p>This example is a similar excerpt from apps/gdp-reader.c (without using
1403 26fa111b Eric Allman
      subscriptions):</p>
1404 eae1d3ec Eric Allman
    <pre>#include &lt;gdp/gdp.h&gt;<br><br>int main(int argc, char **argv)<br>{<br>        gdp_gin_t *gin;<br>        EP_STAT estat;<br>        gdp_name_t gobiname;        // internal name of GOB<br>        gdp_datum_t *d;<br>        gdp_recno_t recno;<br><br>        // general startup and initialization<br>        if (argc &lt; 2)<br>                usage_error();<br>        estat = gdp_init();<br>        if (!EP_STAT_ISOK(estat))<br>                initialization_error(estat);<br>        d = gdp_datum_new();
1405
<br><br>        // parse command line name to internal format<br>        estat = gdp_parse_name(argv[1], gobiname);<br>        if (!EP_STAT_ISOK(estat))<br>                name_syntax_error();
1406
<br>        // attempt to open the GOB<br>        estat = gdp_gin_open(gobiname, GDP_MODE_RO, &amp;gin);<br>        if (!EP_STAT_ISOK(estat))<br>                open_error(estat, argv[1]);<br>
1407
&nbsp;        recno = 1;<br>        for (;;)<br>        {<br>                estat = gdp_gin_read_by_recno(gin, recno++, d);<br>                EP_STAT_CHECK(estat, break);<br>                gdp_datum_print(d, stdout);<br>        }<br>        exit(0);<br>}</pre>
1408 26fa111b Eric Allman
    <br>
1409 10125206 Eric Allman
    <hr width="100%" size="2">
1410 26fa111b Eric Allman
    <p>If you want to use subscriptions, the recno variable can be removed and
1411 a2d7923a Eric Allman
      the for loop replaced with: </p>
1412 eae1d3ec Eric Allman
    <pre>        // enable the subscription<br>        estat = gdp_gin_subscribe_by_recno(gin, 1, -1, NULL, NULL);<br>        if (!EP_STAT_ISOK(estat))<br>                subscribe_error(estat, argv[1]);<br><br>        for (;;)<br>        {<br>                gdp_event_t *gev = gdp_event_next(true);<br>                if (gdp_event_gettype(gev) != GDP_EVENT_DATA)<br>                        continue;<br>                gdp_datum_print(gdp_event_getdatum(gev), stdout);<br>                gdp_event_free(gev);<br>        }</pre>
1413 26fa111b Eric Allman
    <br>
1414 10125206 Eric Allman
    <hr width="100%" size="2">
1415 7d83fa37 Eric Allman
    <h2>Appendix B:&nbsp; Compiling and Linking</h2>
1416 26fa111b Eric Allman
    <p>The GDP library uses a reduced version of libep and also uses the
1417
      libevent library version 2.1. These will need to be included both during
1418
      compilation and linking.</p>
1419
    <p> At compile time you must use:</p>
1420
    <p><code>-I</code><code><i>libevent_includes_parent</i></code><code> -I</code><code><i>libep_includes_parent</i></code></p>
1421
    <p> Note that these take the parent of the directory containing the include
1422
      files. For example, if the include files for libevent are in <tt>/usr/local/include/event2</tt>
1423
      and the include files for libep are in <tt>/usr/local/include/ep</tt> you
1424
      only need to specify the one flag "<code>-I/usr/local/include</code>".</p>
1425
    For linking you must use:<br>
1426
    <pre>-L<i>libevent_libraries</i> -levent -levent_pthreads -L<i>libep_libraries</i> -lep</pre>
1427
    As before, if the libraries for libevent and libep are in the same directory
1428
    you only need a single <tt>-L</tt> flag.<br>
1429
    Libep is a library that I produced several years ago intended for use in
1430
    sendmail. This uses a stripped down version of that library that excludes
1431
    several things that would not be helpful here. For more details of the
1432 b1d346a7 Eric Allman
    original (full) library, see <a href="http://www.neophilic.com/blogs/eric.php/2014/05/12/libep-portable-c-runtime">http://www.neophilic.com/blogs/eric.php/2014/05/12/libep-portable-c-runtime</a>.
1433 26fa111b Eric Allman
    <p>For additional information, see the <tt>README</tt> file in the
1434 a2d7923a Eric Allman
      distribution directory. </p>
1435 7d83fa37 Eric Allman
    <h2> Appendix C: Open Questions</h2>
1436 d9827038 Eric Allman
    <p class="meta">This section is really an addendum to the document &mdash; a
1437
      "scratch area" to keep track of issues that we still need to
1438
      consider.&nbsp; It may not be up to date. </p>
1439 7d83fa37 Eric Allman
    <h3>C.1 Access Control</h3>
1440 26fa111b Eric Allman
    <p> Do this using Access Control Lists (so each user/app has a keypair) or
1441 eae1d3ec Eric Allman
      by passing public/secret keys around (so each GOB has a secret keypair).
1442 26fa111b Eric Allman
      The latter makes revocation impossible (even for write access), so I
1443
      prefer the ACL approach. Third way?</p>
1444
    <p> Revocation? Deep vs. Shallow. Deep = take away permissions that have
1445
      already been given. Shallow = you can only prevent an accessor from
1446
      getting to new versions. Argument: deep revocation is hard to do from a
1447
      technical perspective and ultimately futile (someone might have taken a
1448
      photo of a screen while they still had access), but is still what people
1449
      are used to (Unix and SQL permissions work this way). Shallow is all that
1450
      can really be guaranteed. Also, anything involving Certificate Revocation
1451
      Lists (CRLs) is doomed to failure. This implies that ACLs are the correct
1452
      direction.</p>
1453
    <p> ACLs get us into the question of identity. Pretending that a keypair
1454
      represents an identity doesn't work in the real world where bad players
1455
      simply create new "identities" (keypairs) when an old identity has become
1456
      untrusted. See the extensive work in email sender reputation. However,
1457
      when a bad player creates a new identity/keypair they do not get access to
1458
      any previous grants, so this may be sufficient.</p>
1459 7d83fa37 Eric Allman
    <h3> C.2 Naming</h3>
1460 eae1d3ec Eric Allman
    <p> If each GOB has a secret keypair, then the public key is sufficient to
1461
      name the entity. If not, then assigning a GOB a GUID on creation seems
1462 26fa111b Eric Allman
      like the best approach. Having the user assign a name seems like a
1463
      non-starter, if only because of the possibility of conflicts.</p>
1464
    <p> There will probably be some need for external naming, e.g., some overlay
1465 b364b308 Eric Allman
      directory structure. That might be a different gob_type.</p>
1466 26fa111b Eric Allman
    <p> This seems like an open research topic.</p>
1467 7d83fa37 Eric Allman
    <h3> C.3 Orphans, Expiration, Charging, and Accounting</h3>
1468 eae1d3ec Eric Allman
    <p> If a GOB isn't linked into a directory structure and everyone forgets
1469 26fa111b Eric Allman
      its name then it will live forever (or until it expires). This could be
1470 eae1d3ec Eric Allman
      quite common if a GOB is temporary, that is, not a candidate for long-term
1471 26fa111b Eric Allman
      archival.</p>
1472
    <p> Expiration could be an issue without some sort of charging, which
1473
      implies accounting.</p>
1474 eae1d3ec Eric Allman
    <p> Charging and accounting will affect the API. It seems like on GOB
1475 26fa111b Eric Allman
      creation the creator needs to offer payment for both carrying and storing
1476
      the data. This payment would presumably accrue to the actors providing the
1477
      actual service. Payment for storage might be limited time or indefinite
1478
      time (i.e., it would be an endowment).</p>
1479
    <p> The creator could also specify a cost for any potential consumer in
1480 eae1d3ec Eric Allman
      order to access the GOB. Such payments would accrue to the creator of the
1481
      GOB, and might be used to fund continued access, i.e. it could be rolled
1482 26fa111b Eric Allman
      back into the endowment. This would lean toward making less-used data
1483
      disappear: appealing in some ways, but anathema to librarians and
1484
      historians.</p>
1485 eae1d3ec Eric Allman
    <p> As for API effects, it seems that GOB creation needs to include a
1486 26fa111b Eric Allman
      payment for initial service, a cost for access, and an account into which
1487 eae1d3ec Eric Allman
      to deposit any consumer payments. Accessing a GOB only requires an offered
1488 26fa111b Eric Allman
      payment (which is probably best viewed as a bid rather than a payment,
1489
      thus allowing multiple providers to compete for access).</p>
1490
    <p> Note that none of this is dependent on the form of payment. It does
1491
      however assume that there is a mutually agreed upon form of payment, i.e.,
1492
      a universal currency.</p>
1493 7d83fa37 Eric Allman
    <h3> C.4 Quality of Service</h3>
1494 eae1d3ec Eric Allman
    <p> Is Quality of Service specified on a particular GOB, a particular open
1495
      instance of a GOB, or between a pair of endpoints?</p>
1496 26fa111b Eric Allman
    <p> What does QoS actually mean? For example, in a live media stream it
1497
      probably means the resolution of the data stream (which determines
1498
      real-time bandwidth), latency, and possibly jitter, but after that stream
1499
      is stored the QoS will be resolution (as before), delivery bandwidth (how
1500
      quickly you can download the video, for example), and possibly jitter of
1501
      the network connection (that is, how even the data flow will be). Delivery
1502
      bandwidth depends on the entire path between the data source and the data
1503
      sync, and may be higher or lower than the bandwidth required to send a
1504
      real-time version of the stream &mdash; for example, over a slow network
1505
      link.</p>
1506 7d83fa37 Eric Allman
    <h2> Appendix D: References</h2>
1507 80da996e Eric Allman
    <table cellspacing="2" cellpadding="2" border="1" width="100%">
1508 26fa111b Eric Allman
      <tbody>
1509
        <tr>
1510 a2d7923a Eric Allman
          <td valign="top">[Dab13a] </td>
1511 26fa111b Eric Allman
          <td valign="top">Palmer Dabbelt, Swarm OS Universal Dataplane, August
1512
            22, 2013</td>
1513
        </tr>
1514
        <tr>
1515
          <td valign="top">[Dab13b]</td>
1516
          <td valign="top">Palmer Dabbelt, What is the Universal Dataplane,
1517
            Anyway?, September 17, 2013</td>
1518
        </tr>
1519
      </tbody>
1520
    </table>
1521 10125206 Eric Allman
  </body>
1522
</html>