History | View | Annotate | Download (34.6 KB)
Fix subscription renewal to avoid having gaps. It isn't clear that req->act_ts is still useful, but we'll keep it around for a bit longer.
Improve some debugging in gdplogd commands.
First cut at getting signature checking working again. This is just a partial fix --- more to come.
Do a better job of communicating failure status from gdplogd to the client.
Update license date on all files. No functional changes.
First pass at ordering results of asynchronous reads. Works if the reads come in pre-sorted, but so far untested for out-of-order delivery.
Pass Layer 4 network sequence number to higher layers.
Remove vestiges of old signature implementation.
Restructure signature verification code to move it from gdplogd into libgdp in preparation for verifying signatures on reads. Version number bumped to 2.0.4.
Begin implementation of signature checking by readers.
First cut at fixing bug that sometimes caused an "End of Results" PDU to be delivered before some of the results themselves. This involves passing a "number of results sent" field in the "End of Results" PDU and saving that PDU until the indicated number of results are processed....
First cut at allowing gdp_gin_append_async take multiple datums in one call. It isn't clear the semantics are right yet.
Return EX_CANTCREAT from gdp-create when a log exists even when the -S flag is given; this also makes it easier for lower level functions in gdplogd to customize the NAK code.
Don't throw an assertion failure if an unimplemented command comes in.
Fix GOB reference counts on DELETE commands.
Add debugging flag gdplogd.sig.override which assumes all signatures validate if that flag is set to 101. Just a stop-gap until verify_req_signature is written.
Fix some compiler warnings.
Fix problems with hash chains on logs that did not have have crypto keys.
Fix or expand on FIXME comments.
Get signatures on APPEND working again. Still incomplete, but handles the single datum in a request case.
Cleanup signature and hash type handling. There's no point in storing the MD algorithm with each datum since it is fixed per GOB, nor in passing it over the wire since it is just a blob unless you have either the secret or public key, either of which contain that information...
Fix problem where re-subscriptions would re-send old data.
Snapshot: change physical interface to make more efficient use of SQLite when retrieving sets. Use this for asynchronous reads.
Checkpoint, mostly locking issues.
It turns out that including a GOB pointer in the datum causes too many locking problems. Just pass the gob in as needed.
The grand renaming: GCL is (mostly) gone in favor of GOB and GIN.
Simplify the datum interface back to what it was before. Basic subscriptions and subscription timeout seems to work now.
Snapshot: several problems with timestamps and metadata.
Checkpoint: basic create, write, read all work. Subscriptions, async commands, and multireads untested.
Snapshot: some things working, but more things (notably datum lists) need more work, including at the protocol level.
Snapshot: compiles cleanly with updated protocol and crypto support, but completely untested.
Snapshot in on-disk format conversion from ad hoc files to SQLite databases. Compiles but is completely untested.
Get basic subscriptions working again.
Snapshot of work on getting subscriptions working under Net4. Still has several problems.
Fix re-subscriptions, which were being treated as new subscriptions.
Fix re-subscriptions. Reference counting on subscriptions is still off, but the code does eventually recover.
Be more careful with enums, which are not just aliases for integers in C++. Most importantly, use gdp_cmd_t instead of various int types for commands and acks.
Changes for adding "const" qualifiers. These are mostly good things and not C++ specific.
Changes for signed/unsigned conflicts and other type nits. C++ is much more picky about these than C, but this is probably justified. Sadly, some of the changes are pretty ugly.
Change GdplogdForgive from a struct to a bitmask. Simplified C++ compilation.
Changes necessary to compile under C++ because it doesn't understand that (void *) is compatible with other pointer types. This adds a lot of explicit casts that are not needed in C.
Fix signature generation and checking.
First cut at flattening out protobuf definition. Some changes to data structure printing (e.g., _gdp_pdu_dump) to get indenting more rational so as to make reading the output easier.
Get basic subscriptions working.
First cut at running Layer 5 network protocol version 4 over old (version 3) Layer 4 stack. Substantial parts are unimplemented, including signature generation and checking, read or subscribe by timestamp or hash, and others. No asynchronous operations including...
Snapshot. Compiles but doesn't work.
First cut at "delete log" command (lightly tested).
First cut at separating GCL instances (GINs) from GDP Objects (GOBs). This is intended to fix some semantic problems with subscriptions. End users always use GINs (aliased to gdp_gcl_t for now for compatibility), and there is one GIN for each call to gdp_gcl_open (i.e., no returning...
Move req/gcl lock ordering fix into get_open_handle to reduce redundancy and assure consistency.
Changes to handle certain error conditions with log creation. This still isn't right: see the comments in gdp/gdp_main.c.
More lock ordering tweaks.
More work on making helgrind happy. This seems to be pretty close, maybe worth a try in production.
Snapshot.
Snapshot. Basic things still seem OK, create still seems iffy.
Snapshot on reliability work.
Delete some debugging cruft.
Basic things seems to work, including passing helgrind race condition tests. Still more testing needed.
SNAPSHOT: still with some locking problems.
SNAPSHOT: fewer race conditions; still need more testing.
Initial implementation of unsubscribe.
Tweak previous change to distinguish between 404 and 410 style errors.
Fix error status when attempting to read a record that doesn't exist. Previously returned an internal server error, now returns "410" client error.
Cleanup (and hopefully simplification) of assertion subsystem. Lots of changes, but nothing particularly obtuse.
Some code cleanup for previous commit.
Arrange for client to be able to see the timestamp after gdp_gcl_append returns.
Implement a way to deliver "record missing" back to client.
Avoid making log entries for innocuous errors.
Fix unnecessary locking issues in gdplogd.
Implement extreme list checking in an attempt to find circular loops in linked lists.
Change EP_ASSERT_MUTEX_IS(UN)?LOCKED to EP_THR_MUTEX_ASSERT_IS(UN)LOCKED for consistency with other routine names.
Subscriptions seem to get expired now, but only when another record is written (should happen spontaneously in order to mop up file descriptors).
Subscriptions work again.
Fix locking problem when creating a new GCL.
More work on locking issues.
Lock debug output file so messages can be read.
Add some tracing to gdplogd command processing.
Update licenses to 2017.
Debug level tweaks to make gdplogd log output more coherent. -D10 logs less than before (use -D11 to get more-or-less what existed previously). Some level 10s have been dropped to 9s (if they indicate error conditions).
Minor debug cleanups.
_gdp_gcl_incref should be called when locked, not when unlocked.
Split req->pdu into two: cpdu holds the PDU for the command and rpdu holds the PDU for the response. This makes it easier to make restart work and the code to understand. However, it also means more copying of data and a general lack of elegance. It also means that the datum in...
Snapshot: basic operations seem to be working (no stress tests yet).
Snapshot: get basic reading working again with new lock changes.
Delete unused code.
Use GDP_NAME_SAME macro consistently.
Extensive work on reliability, notably sharing a datum between threads. Increases reliability, but still not fully tested. Basic things seem to work though. Note: this may disrupt the replication code.
Include rid in duplicate comparison when processing a SUBSCRIBE command.
Bug-fix for stuck subscriptions after renewal
See #75
Attempt to deal more gracefully with records that are written out-of-order (as a result of asynchronous writes). The hard point is telling the difference between one writer with out-of-order appends versus two writers that are actually overwriting the same record. Only...
Yet more debug tweaking.
Default swarm.gdplogd.sequencing.allowdups to true. Improved debugging.
Default to allowing gaps on (necessary for asynchronous appends to work with a thread pool); change debug message to make out of order writes look less like an error condition.
Clarify error messages.
Try to match a NOROUTE error to a GCL in the case of FWD_APPEND. This requires a complete scan of the channel work list. Also, back out an "efficiency" change with buffering that broke FWD_APPEND.
Try to improve the "allow gaps" and "allow dups" functionality for replication research.
More tweaking on re-synchronizing recno conflicts.
Add swarm.gdp.sequencing.allow(dups|gaps) options. Fix resynchronization mechanism (hopefully).
Fix problem with inconsistent signatures. Might this be better done by just eliminating the datum->siglen field? It's implied by the size of the datum->sig buffer.
Avoid dereferencing a GCL that isn't cached. Only happens in gdp_gcl_create, and only on unusual errors.