It turns out gdplogd does need HONGD (in order to parse its own name).
Change default for swarm.gdplogd.advertise.interval to 300 seconds since
advertising large numbers of logs can take close to 150 seconds (the old
Try to make sure re-advertising doesn't occur too quickly after a
Use consistent naming when reading signature strictness parameter.
Fix advertising and withdrawal to actually delay between each
advertisement. Required changing handling of event loop startup and
Add swarm.gdplogd.advertise.delay parameter to delay between
advertisements to avoid flooding the network when there are many logs on
a server. Defaults to 100 (microseconds).
Pass flags to GDP library initialization routines. Add flags to disable
Zeroconf and suppress initialization of HONGDS, which is unused in some
Update license date on all files. No functional changes.
First cut at resurrecting log-view (now gdp-log-view) for v2 format.
Still needs some tweaking, but basics work.
First pass at ordering results of asynchronous reads. Works if the
reads come in pre-sorted, but so far untested for out-of-order delivery.
Pass Layer 4 network sequence number to higher layers.
Add new administrative parameters to ignore SIGPIPE in gdplogd
(swarm.gdplogd.ignore.sigpipe) and applications in general
(swarm.gdp.ignore.sigpipe). This allows the library to reconnect to the
router if the connection gets interrupted during a write. However,...
Avoid problems with the name of gdplogd during upgrades.
Snapshot: change physical interface to make more efficient use of SQLite
when retrieving sets. Use this for asynchronous reads.
Snapshot in on-disk format conversion from ad hoc files to SQLite
databases. Compiles but is completely untested.
Get basic subscriptions working again.
Snapshot of work on getting subscriptions working under Net4. Still
has several problems.
Print out version number on startup if debugging is turned on.
Fixes to prevent C++ compile errors for variables that are not
initialized and not used. This isn't a problem with C, and it's not a
bug, just an annoying C++-ism.
Change GdplogdForgive from a struct to a bitmask. Simplified C++
Changes necessary to compile under C++ because it doesn't understand
that (void *) is compatible with other pointer types. This adds a lot
of explicit casts that are not needed in C.
First cut at running Layer 5 network protocol version 4 over old
(version 3) Layer 4 stack. Substantial parts are unimplemented,
including signature generation and checking, read or subscribe by
timestamp or hash, and others. No asynchronous operations including...
Snapshot. Trying to get advertising working on startup. This may be a
larger problem if advertisements need to use challenge-response given
that the I/O thread has been eliminated in gdplogd.
Return to compilable state after rebase. No testing.
First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).
Snapshot. Not tested.
First cut at separating GCL instances (GINs) from GDP Objects (GOBs).
This is intended to fix some semantic problems with subscriptions. End
users always use GINs (aliased to gdp_gcl_t for now for compatibility),
and there is one GIN for each call to gdp_gcl_open (i.e., no returning...
Move some of the assertion recover cleverness from gdplogd into libep so
it is more generally available. Also updates the documentation to make
it clearer (I hope).
gdplogd should default to being more forgiving about assertions.
Attempt to allow GCLs to be cached in applications. This is
specifically for gdp-rest, but there may be other cases where it is
Add request allocation statistics (for debugging).
Lock stderr while printing state dump to make it readable.
Additional debug output on SIGABRT to try to diagnose Rico's problem.
Make it possible to reclaim resources in a worker thread rather than in
the main event loop.
Update licenses to 2017.
Add stack backtrace on assertion failures.
Force all assertions to be fatal if more than N of them happen in T
seconds. Set by swarm.gdplogd.assert.maxasserts and
First cut at using kinder, gentler assertions (mostly in gdp library).
Arrange to do systemd notifications if available. Note that the package
systemd-dev must be installed previously (this is not the default!).
Yet more debug tweaking.
Default swarm.gdplogd.sequencing.allowdups to true. Improved debugging.
Change swarm.gdplogd.advertise.interval default to 150 (2.5 minutes).
If the value is zero, advertisements are not refreshed.
Periodically refresh log advertisements (gdplogd => router). Adds
swarm.gdplogd.advertise.interval parameter (defaults to 30 (seconds)).
Default to allowing gaps on (necessary for asynchronous appends to work
with a thread pool); change debug message to make out of order writes
look less like an error condition.
Try to improve the "allow gaps" and "allow dups" functionality for
Fsync all files when a GCL is closed. By default, close all open GCLs
on shutdown (e.g., SIGTERM). This is an attempt to make corruption less
Accept either SIGINFO or SIGUSR1 to print out stats (to make scripts
easier to write).
Add "forgiveness" to gdplogd to allow it to self heal from various
problems. May be a security problem, but we need it for avoiding
catastrophes while we are debugging.
Make sure the log shows something when gdplogd aborts.
Allow state dump on SIGQUIT or SIGABRT as well as assertion failures.
Add GCL cache consistency checking.
Do detailed GCL cache dump on assertion failure (for debugging).
Arrange to dump gdplogd process state after assertion failure.
Additional debugging for request locking. Change _gdp_req_free API so
it can NULL-out the pointer to the request.
Take number of available file descriptors into account when reclaiming
the GCL cache.
Break out physical implementation to an explicit class so new
implementations (e.g., in memory) can be added easily.
Checkpoint: extent code closer, compiles, doesn't run.
Update license. No code changes.
Include copyright in all files. There are scripts to do bulk
replacement of the copyright if desired.
Don't inform subscribers if daemon shuts down --- the subscriber will
recover when it re-issues the subscription.
Don't log gdplogd shutdowns as LOG_ALERT messages (use LOG_INFO).
Fix some initialization bugs that caused mutex failures.
Improve command line usage messages for gcl-create and gdplogd.
Add gdplogd -s flag (or swarm.gdplogd.crypto.strictness parameter) to
set how strictly signatures are enforced.
Signature checking working. No action taken on bad signature yet.
Still considerable cleanup left to be done.
Add -N flag to gdplogd to allow setting of the routing name. Primarily
for testing; especially useful when starting up several containers from
a single image.
Cleanup for code walkthrough:
* some function rearrangement
* make data structure detail printing more consistent
Minor debugging/error output tweaks for clarity.
Try to at least reduce the number of compile warnings on gcc. It
compiles clean on the NUC (Debian 7, gcc 4.7.2) but still gives some
errors on Ubuntu 14.04.1 (gcc 4.8.2).
Always print our GDP routing name on startup.
Snapshot: working on timing out subscriptions.
Extend PING so it can check an individual subscription. Note: doesn't
work properly if one client has multiple subscriptions on the same GCL,
since it actually checks to see if there exists /any/ subscription from
Other random cleanups (naming and formatting).
Implement gdplogd shutdown actions: withdraw advertisements, explicitly
terminate all subscriptions.
Use ep_app_fatal instead of ep_app_abort in some cases.
Unify command and ack/nak processing; unify Associative and LRU GCL
caches; multiple other cleanups.
Restarting when the routing layer goes away now works. Cleanup needed.
Snapshot in getting reconnection going --- still not there.
Snapshot in getting fallbacks for connections to the routing plane.
Still doesn't work, but at least works for the case where the first
address in the list is available. It's also clear that _gdp_req_new
needs substantial work; as it is, gdplogd can handle commands and...
First steps toward allowing dynamic reconnection to the GDP if
connections are lost.
Allow for multiple log daemons; in particular, GCL creation is
now a separate program which requires the name of a log server.
First cut on the dual-layer model. The old GDP daemon is now the GDP
log daemon; instead of listening to connections it initiates them. This
commit does not include the routing layer.
First cut at new architecture for GDP (separate routing from log layer).
This compiles but does not run.
Move comment about size of thread pool to the correct place.
Rename gdplogd directory so it matches the name of the binary.