| Branch: | Tag: | Revision:

gdp / gdp / gdp_proto.c @ master

History | View | Annotate | Download (24.2 KB)

# Date Author Comment
bd472f67 03/20/2019 03:32 PM Eric Allman

Add some scaffolding to allow multiple advertisements in a single PDU.

c87dd166 01/23/2019 07:56 AM Eric Allman

Update license date on all files. No functional changes.

4412cb93 01/03/2019 12:28 PM Eric Allman

Return an error if retransmissions fail (this caused a SEGV).

34e502ee 09/12/2018 01:41 PM Eric Allman

Second cut at sorting asynchronous data records. Currently all done with
simulation (-Dtest.gdp.chan.seqno=102 randomizes the incoming sequence
numbers), and so missing or duplicated incoming records have not been

e6319882 09/04/2018 09:09 AM Eric Allman

First pass at ordering results of asynchronous reads. Works if the
reads come in pre-sorted, but so far untested for out-of-order delivery.

5657a4ed 08/28/2018 03:28 PM Eric Allman

Better logic about command retries; makes signature checking work. Now

ee0cddb9 08/24/2018 02:16 PM Eric Allman

Add optional signature (proof) checking when reading records. Lightly
tested. Seems to do the right thing if the signature does verify, but
fails incorrectly if it does not. Does not do any hash chain checking

b2910e9d 08/15/2018 02:06 PM Eric Allman

Additional debugging for events and PDU processing for debugging out of
order PDU problems.

825689f9 08/10/2018 05:07 PM Eric Allman

Tweaks to hopefully do a better job of handling out-of-order results for
asynchronous reads.

6ccfea97 08/10/2018 05:05 PM Eric Allman

Debug tweaks.

5e33d674 08/09/2018 03:00 PM Eric Allman

Tweak debugging output to print "end of results" more preferentially
than "results".

d3d990e4 07/31/2018 11:03 AM Eric Allman

Produce more debug output in _gdp_req_dispatch.

e95fef13 07/20/2018 11:59 AM Eric Allman

First cut at fixing bug that sometimes caused an "End of Results" PDU to
be delivered before some of the results themselves. This involves
passing a "number of results sent" field in the "End of Results" PDU and
saving that PDU until the indicated number of results are processed....

2bd9199f 06/27/2018 11:28 AM Eric Allman

Don't throw an assertion failure if an unimplemented command comes in.

2da682ef 06/18/2018 02:51 PM Eric Allman

Get signatures on APPEND working again. Still incomplete, but handles
the single datum in a request case.

d4782b0c 06/09/2018 04:56 AM Eric Allman

Compilation and comment glitches.

bd278f2e 06/08/2018 11:45 AM Eric Allman

Snapshot: change physical interface to make more efficient use of SQLite
when retrieving sets. Use this for asynchronous reads.

b27895fb 05/25/2018 01:22 PM Eric Allman

Checkpoint, mostly locking issues.

eae1d3ec 05/25/2018 01:22 PM Eric Allman

The grand renaming: GCL is (mostly) gone in favor of GOB and GIN.

72871cfc 05/25/2018 01:22 PM Eric Allman

Checkpoint: basic create, write, read all work. Subscriptions, async
commands, and multireads untested.

fa7cfdb9 05/25/2018 01:22 PM Eric Allman

Snapshot: compiles cleanly with updated protocol and crypto
support, but completely untested.

44a5b0ef 01/24/2018 09:57 AM Eric Allman

Other fixes necessary to compile with a C++ compiler:
* Subtlety different semantics for extern keyword.
* Inability to do a forward declaration of a static variable.
* Structs must be initialized in the same order as declared.
* Cannot initialize a fixed size character array with a quoted string....

328ab6a8 01/23/2018 05:34 PM Eric Allman

Changes for signed/unsigned conflicts and other type nits. C++ is much
more picky about these than C, but this is probably justified. Sadly,
some of the changes are pretty ugly.

4a89fdda 01/17/2018 11:34 AM Eric Allman

First cut at flattening out protobuf definition. Some changes to data
structure printing (e.g., _gdp_pdu_dump) to get indenting more rational
so as to make reading the output easier.

d65c01e2 01/09/2018 11:18 AM Eric Allman

First cut at running Layer 5 network protocol version 4 over old
(version 3) Layer 4 stack. Substantial parts are unimplemented,
including signature generation and checking, read or subscribe by
timestamp or hash, and others. No asynchronous operations including...

f2bf61a4 11/08/2017 12:38 PM Eric Allman

Snapshot. Not tested.

5a73c6ad 11/08/2017 12:36 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

4a6db795 10/27/2017 07:35 PM Eric Allman

First cut at "delete log" command (lightly tested).

fec93aac 10/24/2017 02:38 PM Eric Allman

First cut at separating GCL instances (GINs) from GDP Objects (GOBs).
This is intended to fix some semantic problems with subscriptions. End
users always use GINs (aliased to gdp_gcl_t for now for compatibility),
and there is one GIN for each call to gdp_gcl_open (i.e., no returning...

7fdfaa3c 09/19/2017 01:23 AM Eric Allman

Generalize ack/nak to EP_STAT mapping to be more flexible and better
allow for improved error messages. Still more to be done, but the
basics are in.

7bfc0663 09/04/2017 04:34 PM Eric Allman

Changes to handle certain error conditions with log creation. This
still isn't right: see the comments in gdp/gdp_main.c.

11029b9c 08/29/2017 01:59 PM Eric Allman

Snapshot on reliability work.

ee75004c 08/10/2017 09:47 AM Eric Allman

Delete some debugging cruft.

42443eab 08/09/2017 07:46 AM Eric Allman

SNAPSHOT: still with some locking problems.

1b7c480e 08/05/2017 02:17 PM Eric Allman

SNAPSHOT: fewer race conditions; still need more testing.

9c515e64 07/26/2017 06:21 PM Eric Allman

Initial implementation of unsubscribe.

6b31c578 07/07/2017 08:43 AM Eric Allman

Print better debugging info for command tracing.

d583182d 07/06/2017 11:01 AM Eric Allman

Tweak debugging to make command traces easier.

7fe80bc9 07/02/2017 09:36 AM Eric Allman

Code cleanup for GDP command retry.

66ba1423 07/01/2017 05:17 PM Eric Allman

Make sure _gdp_invoke returns an error status if a command times out.

2a8cda22 06/24/2017 02:50 PM Eric Allman

Cleanup (and hopefully simplification) of assertion subsystem. Lots of
changes, but nothing particularly obtuse.

e7726047 04/25/2017 04:56 PM Eric Allman

Don't reset gcl->nrecs on reads unless the datum shows more records than
the GCL structure; this avoids problems with subscriptions in the
presence of asynchronous appends that would move the number of records

729cc8a3 04/25/2017 04:56 PM Eric Allman

Attempt to avoid assertion error if the server fails on a command.

6f52b2b7 04/14/2017 03:16 PM Eric Allman

Implement a way to deliver "record missing" back to client.

6dfc87d5 04/11/2017 10:31 AM Eric Allman

Significant changes to locking to avoid problems with threaded clients.

951b6fb6 03/10/2017 05:05 PM Eric Allman

for consistency with other routine names.

09048d92 03/07/2017 04:48 PM Eric Allman

Make sure number of records is updated when getting subscription

1c226959 03/06/2017 01:17 PM Eric Allman

Fairly extensive locking changes to avoid races/deadlocks.

f8766968 02/17/2017 03:09 PM Eric Allman

Add special debug flag to do command tracing.

897accec 02/06/2017 10:41 AM Eric Allman

Update licenses to 2017.

a104a10f 02/06/2017 10:36 AM Eric Allman

Split req->pdu into two: cpdu holds the PDU for the command and rpdu
holds the PDU for the response. This makes it easier to make restart
work and the code to understand. However, it also means more copying of
data and a general lack of elegance. It also means that the datum in...

e7b2a80c 11/22/2016 02:22 PM Eric Allman

Extensive work on reliability, notably sharing a datum between threads.
Increases reliability, but still not fully tested. Basic things seem to
work though. Note: this may disrupt the replication code.

5efb12f3 11/08/2016 01:57 PM Eric Allman

Avoid interspersed debugging messages when threaded.

2bd3be0e 10/28/2016 11:24 AM Eric Allman

First cut at using kinder, gentler assertions (mostly in gdp library).

195530a2 10/08/2016 08:04 AM Eric Allman

Default swarm.gdplogd.sequencing.allowdups to true. Improved debugging.

60e57723 09/28/2016 12:19 PM Eric Allman

Make sure that the ASYNCIO flag is off whenever we are waiting in
_gdp_invoke to avoid misinterpretation of responses. This makes the
req->state = WAITING very similar to !ASYNCIO; perhaps the flag should
be eliminated.

4d6ff457 09/27/2016 04:52 PM Eric Allman

Fix problem that might occur if an initial response PDU to a command
such as MULTIREAD is delayed; follow-on data was treated as a response to
the initial command rather than as asynchronous events.

354ee62a 09/23/2016 02:17 AM Eric Allman

Fix signature buffer reuse and ensure that datum->siglen ==

1aa4fc2c 09/04/2016 03:14 PM Eric Allman

Fix problem raised by issue #16 (threaded applications failed in
unpredicatable ways). The problem was that calls to gdp_gcl_open need to
be single threaded so that there will be a unique name => handle mapping
for GCLs. This puts a mutex around the entire open call; not ideal...

cbcff1f6 08/26/2016 04:39 PM Eric Allman

Fix a problem associating "NOROUTE" responses with the correct command.

ca209725 08/25/2016 12:36 PM Eric Allman

When swapping PDUs on a response, be sure signatures are properly

75e1eff8 08/24/2016 02:20 PM Eric Allman

Don't keep retrying command if response is a server NAK.

4dc7a0b2 08/24/2016 12:08 PM Eric Allman

Fix Assertion Failure after timeout (related to GCLF_INUSE).

88929fff 08/22/2016 08:41 AM Eric Allman

Make sure that gdp_gcl_append returns the same dbuf that it was passed,
since the application conceivably could have kept a pointer to it.

49ae7ef9 08/18/2016 04:47 PM Eric Allman

More tweaking on re-synchronizing recno conflicts.

6ea74122 08/18/2016 11:56 AM Eric Allman

Add swarm.gdp.sequencing.allow(dups|gaps) options. Fix
resynchronization mechanism (hopefully).

4718c9a7 08/01/2016 10:47 AM Eric Allman

Add some debugging hooks in GDP library.

d2647d1f 07/26/2016 07:34 PM Eric Allman

Next take on read-by-timestamp. This version works for read, multiread,
and subscribe. There are still some glitches (e.g., you can't have a
starting timestamp in the future) but fixing these would require major
code restructuring, and besides, you can't specify a starting record...

65309098 07/26/2016 08:36 AM Eric Allman

First cut at read-by-timestamp. Adds Berkeley DB as a dependency (two
versions supported). The interface still isn't quite right; in
particular, subscribe and multiread don't take timestamps. This should
be remedied before this code is released.

5a64048e 07/15/2016 10:17 AM Eric Allman

Fix likely race condition in GCL req list handling; improved debugging.

0fbf9115 07/13/2016 12:55 PM Eric Allman

Rename "extents" to be "segments" because of different potential
semantics and enable them by default.

a257266f 06/23/2016 10:56 AM Eric Allman

Cleanup for convenience macros for ep_time_nanosleep.
Add debugging to force parallelism in gdplogd.

003f9cf1 06/21/2016 05:26 PM Eric Allman

Fix bug that would cause clients to get out of sync if both appending
and subscribing to a single log.

02356366 06/18/2016 02:22 PM Eric Allman

Be more tolerant of missing records and duplicated records (preparation
for replication).

56de0327 06/15/2016 03:16 PM Eric Allman

Tweak some debugging levels to control verbosity.

8cc28925 05/26/2016 05:10 PM Eric Allman

First cut on implementing FWD_APPEND command. Still not working for all

7d33511d 05/17/2016 11:21 AM Eric Allman

More cleanup of reference counting for GCLs that have error conditions.

283110f2 05/15/2016 11:06 AM Eric Allman

Additional debugging for request locking. Change _gdp_req_free API so
it can NULL-out the pointer to the request.

e539f4d3 05/12/2016 12:28 PM Eric Allman

Fix a reference counting bug. Improve reference counting and locking

24a0a636 02/15/2016 03:19 PM Eric Allman

Add status code/NAK command corresponding to 410 Gone (for expired

f9b45862 01/28/2016 01:08 PM Eric Allman

Implement ability to create new extents. Currently there is no attempt
made to include authorization.

a6ce74aa 01/13/2016 07:51 AM Eric Allman

Fix a minor glitch in debugging output.

6bd5476b 12/01/2015 01:22 PM Eric Allman

Update license. No code changes.

1bfa6133 12/01/2015 12:51 PM Eric Allman

Adjust comment on Nitesh's patch. No code changes.

d5b58a67 11/29/2015 07:41 PM Nitesh Mor

Bug-fix for number of records in GCL

On receiving data from gdplogd after a read command, check before copying
record number to gcl->nrecs. Otherwise reading any old data sets gcl->nrecs
to a smaller value, which then makes any subsequent appends fail.

b23b6dc2 11/24/2015 02:03 PM Eric Allman

Code cleanup: pull out dead code (#ifdefed out) and old SVN version

055d3009 11/21/2015 01:20 PM Eric Allman

Include copyright in all files. There are scripts to do bulk
replacement of the copyright if desired.

08be4c9c 10/31/2015 11:34 AM Eric Allman

Reliability fixes: reinitialize subscriptions periodically and retry
commands on router or server errors (allows gdplogd restarts with open
connections). Subscriptions will catch up and commands such as APPEND
will complete when the daemon returns. Adds a new thread in clients to...

29f683eb 10/28/2015 02:53 PM Eric Allman

First cut at recovering from gdplogd failures (which will result in
NAK_R_NOROUTE errors). Still doesn't work, but no worse than before.

02de7463 10/27/2015 04:53 PM Eric Allman

Implement subscription renewals. This causes clients who have open
subscriptions to re-issue the subscription every
swarm.gdp.subscr.pokeintvl seconds. It's not complete yet: gdplogd
still needs to expire subscriptions after a timeout.

e87e0a1a 10/27/2015 10:46 AM Eric Allman

Work-in-Progress: automatic subscription renewals. Incomplete.

34612550 09/24/2015 01:53 PM Eric Allman

Implement read and append filtering so people can add hooks to do (for
example) encryption and decryption. Adds two new APIs:

EP_STAT (*filter)(gdp_datum_t *datum, void *closure)
gdp_gcl_set_read_filter(gcl, filter, closure)
gdp_gcl_set_append_filter(gcl, filter, closure)
0214bde3 08/24/2015 06:04 PM Eric Allman

Sample fix for Nitesh (Readable and Appendable logs)

f7307b63 07/27/2015 04:30 PM Eric Allman

Signature checking working. No action taken on bad signature yet.
Still considerable cleanup left to be done.

e2682ba9 05/28/2015 10:36 AM Eric Allman

Implement asynchronous writes. Adds new interfaces:
* gdp_gcl_append_async -- initiate an asynchronous write.
* gdp_event_getstat -- get the status code from an event.
Asynchronous reads can be done using multiread.
Note that asynchronous writes do not attempt retransmissions, unlike...

4102ad3a 03/05/2015 05:59 PM Eric Allman

Cleanup for code walkthrough:
* comments
* some function rearrangement
* make data structure detail printing more consistent

932f745e 03/02/2015 02:28 PM Eric Allman

Cleanup prior to code review: chiefly comments, a bit of code reordering
for better flow.

d05d61be 02/21/2015 05:14 PM Eric Allman

Eliminate troublesome GDP_REQ_LOCKED flag; it was really only for
debugging anyway.

7c79c6bc 02/20/2015 02:08 PM Eric Allman

Significant changes to locking and subscriptions:
* Locks on gdp_req_t are now more extensive to avoid data races.
* Care is given to prevent subscription data being processed before
the subscription itself is in place.
Minor changes:
* Move channel pointer into gdp_pdu_t purely as a convenience when...

81defd94 02/16/2015 08:46 AM Eric Allman

Back out broad request locks since that ended up passing locks between
threads; the request state does this now.