Stop using deprecated EP_ASSERT_TEST macro (in favor of EP_ASSERT).
Minor debugging tweak.
Fix subscription renewal to avoid having gaps. It isn't clear that
req->act_ts is still useful, but we'll keep it around for a bit longer.
Update license date on all files. No functional changes.
Second cut at sorting asynchronous data records. Currently all done with
simulation (-Dtest.gdp.chan.seqno=102 randomizes the incoming sequence
numbers), and so missing or duplicated incoming records have not been
First pass at ordering results of asynchronous reads. Works if the
reads come in pre-sorted, but so far untested for out-of-order delivery.
Add optional signature (proof) checking when reading records. Lightly
tested. Seems to do the right thing if the signature does verify, but
fails incorrectly if it does not. Does not do any hash chain checking
Pass Layer 4 network sequence number to higher layers.
Tweaks to hopefully do a better job of handling out-of-order results for
Change debugging flags; flushing PDU isn't an unusual event, so that
shouldn't be on -D1.
Remove vestiges of old signature implementation.
Don't send normal NAKs to syslog --- just internal errors.
Make event queue a TAILQ instead of an STAILQ to avoid n2 algorithm
when results arrive in order.
First cut at fixing bug that sometimes caused an "End of Results" PDU to
be delivered before some of the results themselves. This involves
passing a "number of results sent" field in the "End of Results" PDU and
saving that PDU until the indicated number of results are processed....
Don't throw an assertion failure if an unimplemented command comes in.
Snapshot: change physical interface to make more efficient use of SQLite
when retrieving sets. Use this for asynchronous reads.
The grand renaming: GCL is (mostly) gone in favor of GOB and GIN.
Get basic subscriptions working again.
Snapshot of work on getting subscriptions working under Net4. Still
has several problems.
Be more careful with enums, which are not just aliases for integers in
C++. Most importantly, use gdp_cmd_t instead of various int types for
commands and acks.
Changes for signed/unsigned conflicts and other type nits. C++ is much
more picky about these than C, but this is probably justified. Sadly,
some of the changes are pretty ugly.
Changes necessary to compile under C++ because it doesn't understand
that (void *) is compatible with other pointer types. This adds a lot
of explicit casts that are not needed in C.
Change req->md to req->digest to mirror the name in the GOB and to avoid
confusion between "message digest" and "metadata".
First cut at flattening out protobuf definition. Some changes to data
structure printing (e.g., _gdp_pdu_dump) to get indenting more rational
so as to make reading the output easier.
First cut at running Layer 5 network protocol version 4 over old
(version 3) Layer 4 stack. Substantial parts are unimplemented,
including signature generation and checking, read or subscribe by
timestamp or hash, and others. No asynchronous operations including...
Don't leak reqs when doing async read/append.
Snapshot. Not tested.
First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).
First cut at separating GCL instances (GINs) from GDP Objects (GOBs).
This is intended to fix some semantic problems with subscriptions. End
users always use GINs (aliased to gdp_gcl_t for now for compatibility),
and there is one GIN for each call to gdp_gcl_open (i.e., no returning...
More work on making helgrind happy. This seems to be pretty close,
maybe worth a try in production.
Snapshot. Basic things still seem OK, create still seems iffy.
Helgrind fixes. Still kicks out errors, but so far they all look to be
on a trylock, and hence shouldn't actually cause any deadlocks.
Snapshot on reliability work.
SNAPSHOT: fewer race conditions; still need more testing.
Make Helgrind annotations work properly for (most?) race conditions.
Initial implementation of unsubscribe.
Better fix for problem with attempted fix in commit:a2d73fc0.
Temporary fix for bug introduced by previous commit.
Fix a resource exhaustion problem (GCL reference counting with
Cleanup (and hopefully simplification) of assertion subsystem. Lots of
changes, but nothing particularly obtuse.
Clear stale data out of old req before adding to free list.
Significant changes to locking to avoid problems with threaded clients.
Avoid diagnosing a non-error having to do with reqs on a gcl list.
Add request allocation statistics (for debugging).
Remove some commented out code lines that will not be revived.
Add order specification for some leaf locks.
Declare expected lock ordering for chan, datum, gcl, and req objects.
Add additional debugging on "locking free request" error.
Fix some locking issues.
Implement extreme list checking in an attempt to find circular loops in
Change EP_ASSERT_MUTEX_IS(UN)?LOCKED to EP_THR_MUTEX_ASSERT_IS(UN)LOCKED
for consistency with other routine names.
Subscriptions seem to get expired now, but only when another record is
written (should happen spontaneously in order to mop up file
Subscriptions work again.
More work on locking issues.
Fairly extensive locking changes to avoid races/deadlocks.
Move GDP_ASSERT_MUTEX_IS[UN]LOCKED into EP library; this implementation
is much less portable and relies on peeking into the secret mutex data,
but has the advantage that it seems to work. Also improves some of the
lock debugging output to add more information.
Fix a lock ordering problem; add Helgrind hints.
Update licenses to 2017.
Minor debug cleanups.
Split req->pdu into two: cpdu holds the PDU for the command and rpdu
holds the PDU for the response. This makes it easier to make restart
work and the code to understand. However, it also means more copying of
data and a general lack of elegance. It also means that the datum in...
Snapshot: get basic reading working again with new lock changes.
Extensive work on reliability, notably sharing a datum between threads.
Increases reliability, but still not fully tested. Basic things seem to
work though. Note: this may disrupt the replication code.
Fix bogus error message when flushing GCLs from cache.
Soften another assertion; this one is a true "can't happen", but
recovery is easy.
Attempt to recover a bit more gracefully from one "assertion failure"
like error. There is a memory leak in the recovery, but that can't be
First cut at using kinder, gentler assertions (mostly in gdp library).
Fix a couple of locking problems: one a doublely locked mutex and one a
thread that is spawned while holding a mutex.
Fix a potential infinite loop in _gdp_req_freeall.
Fix bogus assertion failures or other errors when gdp_gcl_open can't
find a route to the log.
Implement gdp_gcl_read_async. This also fixes some obscure problems
with other asynchronous operations, and tweaks debugging flags to make
it easier to pick out the operations you want to see.
Fix problem that might occur if an initial response PDU to a command
such as MULTIREAD is delayed; follow-on data was treated as a response to
the initial command rather than as asynchronous events.
Log a message if request id wraps around (it's only a uint32). This
does not prevent reuse! There should really be a _gdp_rid_free to make
Fix problem raised by issue #16 (threaded applications failed in
unpredicatable ways). The problem was that calls to gdp_gcl_open need to
be single threaded so that there will be a unique name => handle mapping
for GCLs. This puts a mutex around the entire open call; not ideal...
Make many parameters "const" where possible.
Fix Assertion Failure after timeout (related to GCLF_INUSE).
Fix likely race condition in GCL req list handling; improved debugging.
First cut on implementing FWD_APPEND command. Still not working for all
More cleanup of reference counting for GCLs that have error conditions.
Additional debugging for request locking. Change _gdp_req_free API so
it can NULL-out the pointer to the request.
Change _gdp_gcl_decref API to automatically null out the pointer ---
this makes it less prone to error.
Fix a race condition where _gdp_req_lock could return a pointer to a
free req. This could only happen if thread T1 got a pointer to the req,
then T2 locked the req, then T1 blocked on the lock, T2 freed the req
and unlocked the req, allowing T1 to continue with the now freed req....
Fix gdplogd cache flushing so that expired subscriptions don't prevent
reclaimation of a GCL. This caused long-running gdplogd processes to
run out of file descriptors.
Fix a problem that was causing occassional unlocks of unlocked mutexes
(only when a subscription timed out).
Update license. No code changes.
Include copyright in all files. There are scripts to do bulk
replacement of the copyright if desired.
Reliability fixes: reinitialize subscriptions periodically and retry
commands on router or server errors (allows gdplogd restarts with open
connections). Subscriptions will catch up and commands such as APPEND
will complete when the daemon returns. Adds a new thread in clients to...
First cut at recovering from gdplogd failures (which will result in
NAK_R_NOROUTE errors). Still doesn't work, but no worse than before.
Implement subscription renewals. This causes clients who have open
subscriptions to re-issue the subscription every
swarm.gdp.subscr.pokeintvl seconds. It's not complete yet: gdplogd
still needs to expire subscriptions after a timeout.
Work-in-Progress: automatic subscription renewals. Incomplete.
Improve GCL safety checks (asserts on function entry).
Signature checking working. No action taken on bad signature yet.
Still considerable cleanup left to be done.
Another signing snapshot: signatures are created and sent but not
checked. Metadata is returned when a GCL is opened. Still have a
problem with reference counting of GCL handles.
Snapshot adding signing. Signatures are computed but not sent or
Implement asynchronous writes. Adds new interfaces:
* gdp_gcl_append_async -- initiate an asynchronous write.
* gdp_event_getstat -- get the status code from an event.
Asynchronous reads can be done using multiread.
Note that asynchronous writes do not attempt retransmissions, unlike...
Cleanup for code walkthrough:
* some function rearrangement
* make data structure detail printing more consistent
Eliminate troublesome GDP_REQ_LOCKED flag; it was really only for
Fix a lock ordering problem (req/gcl). Unfortunately this fix may
introduce a race condition.
Significant changes to locking and subscriptions:
* Locks on gdp_req_t are now more extensive to avoid data races.
* Care is given to prevent subscription data being processed before
the subscription itself is in place.
* Move channel pointer into gdp_pdu_t purely as a convenience when...