Fix subscription renewal to avoid having gaps. It isn't clear that
req->act_ts is still useful, but we'll keep it around for a bit longer.
Improve some debugging in gdplogd commands.
First cut at getting signature checking working again. This is just a
partial fix --- more to come.
Do a better job of communicating failure status from gdplogd to the
Update license date on all files. No functional changes.
First pass at ordering results of asynchronous reads. Works if the
reads come in pre-sorted, but so far untested for out-of-order delivery.
Pass Layer 4 network sequence number to higher layers.
Remove vestiges of old signature implementation.
Restructure signature verification code to move it from gdplogd into
libgdp in preparation for verifying signatures on reads. Version number
bumped to 2.0.4.
Begin implementation of signature checking by readers.
First cut at fixing bug that sometimes caused an "End of Results" PDU to
be delivered before some of the results themselves. This involves
passing a "number of results sent" field in the "End of Results" PDU and
saving that PDU until the indicated number of results are processed....
First cut at allowing gdp_gin_append_async take multiple datums in one
call. It isn't clear the semantics are right yet.
Return EX_CANTCREAT from gdp-create when a log exists even when the -S
flag is given; this also makes it easier for lower level functions in
gdplogd to customize the NAK code.
Don't throw an assertion failure if an unimplemented command comes in.
Fix GOB reference counts on DELETE commands.
Add debugging flag gdplogd.sig.override which assumes all signatures
validate if that flag is set to 101. Just a stop-gap until
verify_req_signature is written.
Fix some compiler warnings.
Fix problems with hash chains on logs that did not have have crypto
Fix or expand on FIXME comments.
Get signatures on APPEND working again. Still incomplete, but handles
the single datum in a request case.
Cleanup signature and hash type handling. There's no point in storing
the MD algorithm with each datum since it is fixed per GOB, nor in
passing it over the wire since it is just a blob unless you have either
the secret or public key, either of which contain that information...
Fix problem where re-subscriptions would re-send old data.
Snapshot: change physical interface to make more efficient use of SQLite
when retrieving sets. Use this for asynchronous reads.
Checkpoint, mostly locking issues.
It turns out that including a GOB pointer in the datum causes too many
locking problems. Just pass the gob in as needed.
The grand renaming: GCL is (mostly) gone in favor of GOB and GIN.
Simplify the datum interface back to what it was before. Basic
subscriptions and subscription timeout seems to work now.
Snapshot: several problems with timestamps and metadata.
Checkpoint: basic create, write, read all work. Subscriptions, async
commands, and multireads untested.
Snapshot: some things working, but more things (notably datum lists)
need more work, including at the protocol level.
Snapshot: compiles cleanly with updated protocol and crypto
support, but completely untested.
Snapshot in on-disk format conversion from ad hoc files to SQLite
databases. Compiles but is completely untested.
Get basic subscriptions working again.
Snapshot of work on getting subscriptions working under Net4. Still
has several problems.
Fix re-subscriptions, which were being treated as new subscriptions.
Reference counting on subscriptions is still off, but the code does
Be more careful with enums, which are not just aliases for integers in
C++. Most importantly, use gdp_cmd_t instead of various int types for
commands and acks.
Changes for adding "const" qualifiers. These are mostly good things and
not C++ specific.
Changes for signed/unsigned conflicts and other type nits. C++ is much
more picky about these than C, but this is probably justified. Sadly,
some of the changes are pretty ugly.
Change GdplogdForgive from a struct to a bitmask. Simplified C++
Changes necessary to compile under C++ because it doesn't understand
that (void *) is compatible with other pointer types. This adds a lot
of explicit casts that are not needed in C.
Fix signature generation and checking.
First cut at flattening out protobuf definition. Some changes to data
structure printing (e.g., _gdp_pdu_dump) to get indenting more rational
so as to make reading the output easier.
Get basic subscriptions working.
First cut at running Layer 5 network protocol version 4 over old
(version 3) Layer 4 stack. Substantial parts are unimplemented,
including signature generation and checking, read or subscribe by
timestamp or hash, and others. No asynchronous operations including...
Snapshot. Compiles but doesn't work.
First cut at "delete log" command (lightly tested).
First cut at separating GCL instances (GINs) from GDP Objects (GOBs).
This is intended to fix some semantic problems with subscriptions. End
users always use GINs (aliased to gdp_gcl_t for now for compatibility),
and there is one GIN for each call to gdp_gcl_open (i.e., no returning...
Move req/gcl lock ordering fix into get_open_handle to reduce redundancy
and assure consistency.
Changes to handle certain error conditions with log creation. This
still isn't right: see the comments in gdp/gdp_main.c.
More lock ordering tweaks.
More work on making helgrind happy. This seems to be pretty close,
maybe worth a try in production.
Snapshot. Basic things still seem OK, create still seems iffy.
Snapshot on reliability work.
Delete some debugging cruft.
Basic things seems to work, including passing helgrind race condition
tests. Still more testing needed.
SNAPSHOT: still with some locking problems.
SNAPSHOT: fewer race conditions; still need more testing.
Initial implementation of unsubscribe.
Tweak previous change to distinguish between 404 and 410 style errors.
Fix error status when attempting to read a record that doesn't exist.
Previously returned an internal server error, now returns "410" client
Cleanup (and hopefully simplification) of assertion subsystem. Lots of
changes, but nothing particularly obtuse.
Some code cleanup for previous commit.
Arrange for client to be able to see the timestamp after gdp_gcl_append
Implement a way to deliver "record missing" back to client.
Avoid making log entries for innocuous errors.
Fix unnecessary locking issues in gdplogd.
Implement extreme list checking in an attempt to find circular loops in
Change EP_ASSERT_MUTEX_IS(UN)?LOCKED to EP_THR_MUTEX_ASSERT_IS(UN)LOCKED
for consistency with other routine names.
Subscriptions seem to get expired now, but only when another record is
written (should happen spontaneously in order to mop up file
Subscriptions work again.
Fix locking problem when creating a new GCL.
More work on locking issues.
Lock debug output file so messages can be read.
Add some tracing to gdplogd command processing.
Update licenses to 2017.
Debug level tweaks to make gdplogd log output more coherent. -D10 logs
less than before (use -D11 to get more-or-less what existed previously).
Some level 10s have been dropped to 9s (if they indicate error
Minor debug cleanups.
_gdp_gcl_incref should be called when locked, not when unlocked.
Split req->pdu into two: cpdu holds the PDU for the command and rpdu
holds the PDU for the response. This makes it easier to make restart
work and the code to understand. However, it also means more copying of
data and a general lack of elegance. It also means that the datum in...
Snapshot: basic operations seem to be working (no stress tests yet).
Snapshot: get basic reading working again with new lock changes.
Delete unused code.
Use GDP_NAME_SAME macro consistently.
Extensive work on reliability, notably sharing a datum between threads.
Increases reliability, but still not fully tested. Basic things seem to
work though. Note: this may disrupt the replication code.
Include rid in duplicate comparison when processing a SUBSCRIBE command.
Bug-fix for stuck subscriptions after renewal
Attempt to deal more gracefully with records that are written
out-of-order (as a result of asynchronous writes). The hard point is
telling the difference between one writer with out-of-order appends
versus two writers that are actually overwriting the same record. Only...
Yet more debug tweaking.
Default swarm.gdplogd.sequencing.allowdups to true. Improved debugging.
Default to allowing gaps on (necessary for asynchronous appends to work
with a thread pool); change debug message to make out of order writes
look less like an error condition.
Clarify error messages.
Try to match a NOROUTE error to a GCL in the case of FWD_APPEND. This
requires a complete scan of the channel work list. Also, back out an
"efficiency" change with buffering that broke FWD_APPEND.
Try to improve the "allow gaps" and "allow dups" functionality for
More tweaking on re-synchronizing recno conflicts.
Add swarm.gdp.sequencing.allow(dups|gaps) options. Fix
resynchronization mechanism (hopefully).
Fix problem with inconsistent signatures. Might this be better done by
just eliminating the datum->siglen field? It's implied by the size of
the datum->sig buffer.
Avoid dereferencing a GCL that isn't cached. Only happens in
gdp_gcl_create, and only on unusual errors.