| Branch: | Tag: | Revision:

gdp / gdp / gdp_api.c @ master

History | View | Annotate | Download (24.8 KB)

# Date Author Comment
49d76189 01/14/2020 01:06 PM Eric Allman

Version 2.2.3: Make failure to read a secret key for a writable log an
error instead of waiting for the first append to fail (this can be
turned off using gdp_open_info_set_no_skey_nonfatal, which causes
gdp_gin_open to return a warning instead of an error). Add a -S flag to...

a4c503f0 04/29/2019 12:02 PM Eric Allman

Fix bug caused when passing a NULL open_info to gdp_gin_open.

866c2ca2 01/23/2019 01:17 PM Eric Allman

Pass flags to GDP library initialization routines. Add flags to disable
Zeroconf and suppress initialization of HONGDS, which is unused in some

c87dd166 01/23/2019 07:56 AM Eric Allman

Update license date on all files. No functional changes.

80da996e 10/24/2018 12:53 PM Eric Allman

Mega-change: new object creation API with updated metadata; internal
name is hash of metadata, not hash of human name; query and maintain
human-oriented name to GDPname directory (HONGD).

75f25465 09/13/2018 03:35 PM Eric Allman

Minor fix for GIN flag debug printing.

ee0cddb9 08/24/2018 02:16 PM Eric Allman

Add optional signature (proof) checking when reading records. Lightly
tested. Seems to do the right thing if the signature does verify, but
fails incorrectly if it does not. Does not do any hash chain checking

b2910e9d 08/15/2018 02:06 PM Eric Allman

Additional debugging for events and PDU processing for debugging out of
order PDU problems.

caef25cd 07/31/2018 11:08 AM Eric Allman

Restructure signature verification code to move it from gdplogd into
libgdp in preparation for verifying signatures on reads. Version number
bumped to 2.0.4.

c9f4745b 07/27/2018 10:52 AM Eric Allman

Begin implementation of signature checking by readers.

1a1a88c7 06/25/2018 01:55 PM Eric Allman

Avoid problems with the name of gdplogd during upgrades.

4aa32996 06/22/2018 02:46 PM Eric Allman

Pass GIN through to requests so that events can be properly correlated.

4217d271 06/14/2018 05:28 PM Eric Allman

Better take on gdp_gin_read_by_hash_async for multiple records.

587a05de 06/09/2018 01:32 PM Eric Allman

Restore semantics leaving datums filled in after an append command.
This requires that apps that re-use datums must use gdp_datum_reset to
clear the old data before adding more.

bd278f2e 06/08/2018 11:45 AM Eric Allman

Snapshot: change physical interface to make more efficient use of SQLite
when retrieving sets. Use this for asynchronous reads.

eae1d3ec 05/25/2018 01:22 PM Eric Allman

The grand renaming: GCL is (mostly) gone in favor of GOB and GIN.

31d6906f 05/25/2018 01:22 PM Eric Allman

Get basic subscriptions working again.

fa7cfdb9 05/25/2018 01:22 PM Eric Allman

Snapshot: compiles cleanly with updated protocol and crypto
support, but completely untested.

92243bf4 05/25/2018 01:22 PM Eric Allman

Snapshot in on-disk format conversion from ad hoc files to SQLite
databases. Compiles but is completely untested.

48236b26 04/17/2018 10:29 AM Eric Allman

Break out GDP name utility functions (parse, print, etc.) into a
separate file so Rick can use them in the router without dragging a lot
of cruft along.

10125206 02/27/2018 06:02 PM Eric Allman

First step at adding QoS information to subscriptions.

8bebf0e3 02/26/2018 11:11 AM Eric Allman

Get basic subscriptions working again.

c9242838 01/24/2018 09:53 AM Eric Allman

Fix a minor (but legitimate) bug with reversed arguments to strcpy.
Only happened for certain unusual errors, hence why it hadn't been found

f67d0170 01/24/2018 09:45 AM Eric Allman

Be more careful with enums, which are not just aliases for integers in
C++. Most importantly, use gdp_cmd_t instead of various int types for
commands and acks.

328ab6a8 01/23/2018 05:34 PM Eric Allman

Changes for signed/unsigned conflicts and other type nits. C++ is much
more picky about these than C, but this is probably justified. Sadly,
some of the changes are pretty ugly.

fc0cbc23 01/23/2018 04:09 PM Eric Allman

Changes necessary to compile under C++ because it doesn't understand
that (void *) is compatible with other pointer types. This adds a lot
of explicit casts that are not needed in C.

d65c01e2 01/09/2018 11:18 AM Eric Allman

First cut at running Layer 5 network protocol version 4 over old
(version 3) Layer 4 stack. Substantial parts are unimplemented,
including signature generation and checking, read or subscribe by
timestamp or hash, and others. No asynchronous operations including...

08bc143f 11/18/2017 05:11 AM Eric Allman

Simplified version of gdp_gcl_append that avoids a temporary copy of the
datum. I can't figure out why that was necessary.

2d4227cb 11/18/2017 05:10 AM Eric Allman

Return datum metadata on append.

5870de58 11/16/2017 03:00 PM Eric Allman

Additional debugging during initialization.

d92b6c0d 11/08/2017 12:56 PM Eric Allman

Merge branch 'eric/net4' of into eric/net4

f2bf61a4 11/08/2017 12:38 PM Eric Allman

Snapshot. Not tested.

5a73c6ad 11/08/2017 12:36 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

4a6db795 10/27/2017 07:35 PM Eric Allman

First cut at "delete log" command (lightly tested).

6188de9e 10/25/2017 11:59 AM Eric Allman

Fix reference counting problem in gdp_gcl_open and gdp_gcl_create.

c9b75ebc 10/25/2017 11:47 AM Eric Allman

Better debugging for GDP instances (GINs).

fec93aac 10/24/2017 02:38 PM Eric Allman

First cut at separating GCL instances (GINs) from GDP Objects (GOBs).
This is intended to fix some semantic problems with subscriptions. End
users always use GINs (aliased to gdp_gcl_t for now for compatibility),
and there is one GIN for each call to gdp_gcl_open (i.e., no returning...

1c492c37 10/02/2017 08:17 AM Eric Allman

Minor comment cleanup for clarity.

556652f8 09/07/2017 02:31 PM Eric Allman

Put some extra checking in to make sure GCLs returned from
gdp_gcl_create are good.

5e5bd9c4 09/05/2017 03:45 PM Eric Allman

Update iomode handling to avoid insisting on a secret key when you
aren't writing the log; clean up secret lookup to be less convoluted.

95547c3d 09/04/2017 01:46 PM Eric Allman

Force all GCLs into GDP_MODE_ANY (a.k.a. GDP_MODE_RA) to work around
application issues.

1ca7751e 09/01/2017 03:45 PM Eric Allman

More work on making helgrind happy. This seems to be pretty close,
maybe worth a try in production.

7f174460 09/01/2017 10:15 AM Eric Allman


f2bf1027 08/31/2017 03:55 PM Eric Allman

Snapshot. Basic things still seem OK, create still seems iffy.

11029b9c 08/29/2017 01:59 PM Eric Allman

Snapshot on reliability work.

42443eab 08/09/2017 07:46 AM Eric Allman

SNAPSHOT: still with some locking problems.

1b7c480e 08/05/2017 02:17 PM Eric Allman

SNAPSHOT: fewer race conditions; still need more testing.

9c515e64 07/26/2017 06:21 PM Eric Allman

Initial implementation of unsubscribe.

b28f9bd6 07/14/2017 01:33 PM Eric Allman

Snapshot. Not tested.

3bc20136 07/14/2017 01:33 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

f678e274 07/07/2017 02:03 PM Eric Allman

Generalize statistics printing on exit (and save gdp-reader and
gdp-writer from using internal interfaces).

2a8cda22 06/24/2017 02:50 PM Eric Allman

Cleanup (and hopefully simplification) of assertion subsystem. Lots of
changes, but nothing particularly obtuse.

ddb857b7 06/20/2017 12:52 PM Eric Allman

Cache freed GCL handles so we can avoid double frees.

284a8bac 05/11/2017 01:50 PM Eric Allman

Add gdp_gcl_open_info_set_signkey_cb to set a callback to get the secret
key. This is to avoid asking for a decryption password if it isn't
going to be used (e.g., if you are opening a GCL that is already in the
cache). This is particularly important for gdp-rest.

ee6b5440 05/10/2017 01:17 PM Eric Allman

Attempt to allow GCLs to be cached in applications. This is
specifically for gdp-rest, but there may be other cases where it is

f759690d 04/18/2017 04:23 PM Eric Allman

Don't dereference null pointer if gdp_gcl_create fails.

00cf2f7b 04/11/2017 12:18 PM Eric Allman

Need to unlock GCL after creation.

6dfc87d5 04/11/2017 10:31 AM Eric Allman

Significant changes to locking to avoid problems with threaded clients.

511a7b77 03/31/2017 06:08 PM Eric Allman

Avoid bogus locking errors when gdp_gcl_open fails.

c6ab3a19 03/30/2017 04:26 PM Eric Allman

More locking fixes. This actually simplifies some code.

7882a3b0 03/23/2017 10:13 PM Eric Allman

Add order specification for some leaf locks.

25a3c2d2 03/14/2017 03:35 PM Eric Allman

Fix some locking issues.

1c226959 03/06/2017 01:17 PM Eric Allman

Fairly extensive locking changes to avoid races/deadlocks.

897accec 02/06/2017 10:41 AM Eric Allman

Update licenses to 2017.

4d3ca9a1 02/06/2017 10:36 AM Eric Allman

Refactor code so "unnamed" GCLs without metadata are still unique.

a104a10f 02/06/2017 10:36 AM Eric Allman

Split req->pdu into two: cpdu holds the PDU for the command and rpdu
holds the PDU for the response. This makes it easier to make restart
work and the code to understand. However, it also means more copying of
data and a general lack of elegance. It also means that the datum in...

083cb865 12/12/2016 12:15 PM Eric Allman

Make sure datum is completely reset on read (including signature).

a977e6aa 12/11/2016 09:56 AM Eric Allman

Snapshot: basic operations seem to be working (no stress tests yet).

e7b2a80c 11/22/2016 02:22 PM Eric Allman

Extensive work on reliability, notably sharing a datum between threads.
Increases reliability, but still not fully tested. Basic things seem to
work though. Note: this may disrupt the replication code.

8701c522 11/03/2016 11:34 AM Eric Allman

Further attempts to be more forgiving about errors, particularly in
terms of inappropriate values in API calls (which are no longer

a749564b 10/28/2016 12:18 PM Eric Allman

Check result of auto-initialization; library cannot continue if
initialization fails.

2bd3be0e 10/28/2016 11:24 AM Eric Allman

First cut at using kinder, gentler assertions (mostly in gdp library).

195530a2 10/08/2016 08:04 AM Eric Allman

Default swarm.gdplogd.sequencing.allowdups to true. Improved debugging.

3348ff2a 10/05/2016 10:36 AM Eric Allman

Fix bogus assertion failures or other errors when gdp_gcl_open can't
find a route to the log.

8204d4b2 09/28/2016 10:45 AM Eric Allman

Implement gdp_gcl_read_async. This also fixes some obscure problems
with other asynchronous operations, and tweaks debugging flags to make
it easier to pick out the operations you want to see.

8f08ae4f 09/22/2016 06:15 AM Eric Allman

If a gdp_gcl_open fails, be sure to remove the useless GCL from the
cache so that future attempts will retry rather than incorrectly
return success. The alternative would be to cache the failure, but it
might be useful to try again on a retry in the off chance that the log...

1aa4fc2c 09/04/2016 03:14 PM Eric Allman

Fix problem raised by issue #16 (threaded applications failed in
unpredicatable ways). The problem was that calls to gdp_gcl_open need to
be single threaded so that there will be a unique name => handle mapping
for GCLs. This puts a mutex around the entire open call; not ideal...

d2647d1f 07/26/2016 07:34 PM Eric Allman

Next take on read-by-timestamp. This version works for read, multiread,
and subscribe. There are still some glitches (e.g., you can't have a
starting timestamp in the future) but fixing these would require major
code restructuring, and besides, you can't specify a starting record...

65309098 07/26/2016 08:36 AM Eric Allman

First cut at read-by-timestamp. Adds Berkeley DB as a dependency (two
versions supported). The interface still isn't quite right; in
particular, subscribe and multiread don't take timestamps. This should
be remedied before this code is released.

0fbf9115 07/13/2016 12:55 PM Eric Allman

Rename "extents" to be "segments" because of different potential
semantics and enable them by default.

7fcb8dfd 04/13/2016 04:58 PM Eric Allman

Change name of GCL in cache after creation so that the log can be used
immediately in the same process that created it. Patch initially
created by Sam Kumar.

e8097b09 04/13/2016 03:11 PM Eric Allman

Allow clients to open more than one instance of a single GCL. Second
and subsequent opens increment the reference count and return the
already instance of the GCL. gdp_gcl_close just decrements the
reference count unless it has the last reference.

f9b45862 01/28/2016 01:08 PM Eric Allman

Implement ability to create new extents. Currently there is no attempt
made to include authorization.

e89296c0 01/27/2016 12:11 PM Eric Allman

Improved debug output for gdp_internal_name

6bd5476b 12/01/2015 01:22 PM Eric Allman

Update license. No code changes.

b23b6dc2 11/24/2015 02:03 PM Eric Allman

Code cleanup: pull out dead code (#ifdefed out) and old SVN version

055d3009 11/21/2015 01:20 PM Eric Allman

Include copyright in all files. There are scripts to do bulk
replacement of the copyright if desired.

2cdd45bd 11/04/2015 03:27 PM Eric Allman

Don't mark GDP as initialized until after it succeeds.

02de7463 10/27/2015 04:53 PM Eric Allman

Implement subscription renewals. This causes clients who have open
subscriptions to re-issue the subscription every
swarm.gdp.subscr.pokeintvl seconds. It's not complete yet: gdplogd
still needs to expire subscriptions after a timeout.

b8c5e7c8 10/15/2015 05:17 PM Eric Allman

Avoid assertion failures if applications forget to call gdp_init().

a14a218e 10/09/2015 11:01 AM Eric Allman

Improve GCL safety checks (asserts on function entry).

7d83fa37 10/05/2015 06:38 PM Eric Allman

Add new gdp_gcl_open_info interface to allow tweaking of GCL parameters
on open. The only thing included now is the setting of the signing key.

8ce5b4db 10/05/2015 11:50 AM Eric Allman

Use the third parameter to gdp_gcl_open for general open information,
not just QoS.

34612550 09/24/2015 01:53 PM Eric Allman

Implement read and append filtering so people can add hooks to do (for
example) encryption and decryption. Adds two new APIs:

EP_STAT (*filter)(gdp_datum_t *datum, void *closure)
gdp_gcl_set_read_filter(gcl, filter, closure)
gdp_gcl_set_append_filter(gcl, filter, closure)
9c59e638 09/18/2015 10:54 AM Eric Allman

Add log-mirror command for trivial replication. Required also adding
gdp_gcl_getnrecs to API.

b48f4199 08/28/2015 10:37 AM Eric Allman

Fix GDP_MODE_RA (only half implemented). Also adds a hidden "-A" flag
to reader-test to use this mode; it's only for debugging.

f43e276d 08/20/2015 03:58 PM Eric Allman

Allow logs to opened for both read and append (GDP_MODE_RA).

b8591e8b 08/06/2015 01:59 PM Eric Allman

Fix some initialization bugs that caused mutex failures.

e24c9b55 08/01/2015 04:38 PM Eric Allman

Fix creation of anonymous (no external name) logs

f7307b63 07/27/2015 04:30 PM Eric Allman

Signature checking working. No action taken on bad signature yet.
Still considerable cleanup left to be done.