History | View | Annotate | Download (11.9 KB)
It turns out gdplogd does need HONGD (in order to parse its own name).
Change default for swarm.gdplogd.advertise.interval to 300 seconds since advertising large numbers of logs can take close to 150 seconds (the old default).
Try to make sure re-advertising doesn't occur too quickly after a reconnect.
Use consistent naming when reading signature strictness parameter.
Fix advertising and withdrawal to actually delay between each advertisement. Required changing handling of event loop startup and shutdown.
Add swarm.gdplogd.advertise.delay parameter to delay between advertisements to avoid flooding the network when there are many logs on a server. Defaults to 100 (microseconds).
Pass flags to GDP library initialization routines. Add flags to disable Zeroconf and suppress initialization of HONGDS, which is unused in some programs.
Update license date on all files. No functional changes.
First cut at resurrecting log-view (now gdp-log-view) for v2 format. Still needs some tweaking, but basics work.
First pass at ordering results of asynchronous reads. Works if the reads come in pre-sorted, but so far untested for out-of-order delivery.
Pass Layer 4 network sequence number to higher layers.
Add new administrative parameters to ignore SIGPIPE in gdplogd (swarm.gdplogd.ignore.sigpipe) and applications in general (swarm.gdp.ignore.sigpipe). This allows the library to reconnect to the router if the connection gets interrupted during a write. However,...
Avoid problems with the name of gdplogd during upgrades.
Snapshot: change physical interface to make more efficient use of SQLite when retrieving sets. Use this for asynchronous reads.
Snapshot in on-disk format conversion from ad hoc files to SQLite databases. Compiles but is completely untested.
Get basic subscriptions working again.
Snapshot of work on getting subscriptions working under Net4. Still has several problems.
Print out version number on startup if debugging is turned on.
Fixes to prevent C++ compile errors for variables that are not initialized and not used. This isn't a problem with C, and it's not a bug, just an annoying C++-ism.
Change GdplogdForgive from a struct to a bitmask. Simplified C++ compilation.
Changes necessary to compile under C++ because it doesn't understand that (void *) is compatible with other pointer types. This adds a lot of explicit casts that are not needed in C.
First cut at running Layer 5 network protocol version 4 over old (version 3) Layer 4 stack. Substantial parts are unimplemented, including signature generation and checking, read or subscribe by timestamp or hash, and others. No asynchronous operations including...
Snapshot. Trying to get advertising working on startup. This may be a larger problem if advertisements need to use challenge-response given that the I/O thread has been eliminated in gdplogd.
Return to compilable state after rebase. No testing.
First cut at new net protocol. Compiles, but not yet tested (not even once --- do not try to run this!).
Snapshot. Not tested.
First cut at separating GCL instances (GINs) from GDP Objects (GOBs). This is intended to fix some semantic problems with subscriptions. End users always use GINs (aliased to gdp_gcl_t for now for compatibility), and there is one GIN for each call to gdp_gcl_open (i.e., no returning...
Move some of the assertion recover cleverness from gdplogd into libep so it is more generally available. Also updates the documentation to make it clearer (I hope).
gdplogd should default to being more forgiving about assertions.
Attempt to allow GCLs to be cached in applications. This is specifically for gdp-rest, but there may be other cases where it is useful.
Add request allocation statistics (for debugging).
Lock stderr while printing state dump to make it readable.
Additional debug output on SIGABRT to try to diagnose Rico's problem.
Make it possible to reclaim resources in a worker thread rather than in the main event loop.
Update licenses to 2017.
Add stack backtrace on assertion failures.
Force all assertions to be fatal if more than N of them happen in T seconds. Set by swarm.gdplogd.assert.maxasserts and swarm.gdplogd.assert.resetinterval respectively.
First cut at using kinder, gentler assertions (mostly in gdp library).
Arrange to do systemd notifications if available. Note that the package systemd-dev must be installed previously (this is not the default!).
Yet more debug tweaking.
Default swarm.gdplogd.sequencing.allowdups to true. Improved debugging.
Change swarm.gdplogd.advertise.interval default to 150 (2.5 minutes). If the value is zero, advertisements are not refreshed.
Periodically refresh log advertisements (gdplogd => router). Adds swarm.gdplogd.advertise.interval parameter (defaults to 30 (seconds)).
Default to allowing gaps on (necessary for asynchronous appends to work with a thread pool); change debug message to make out of order writes look less like an error condition.
Try to improve the "allow gaps" and "allow dups" functionality for replication research.
Fsync all files when a GCL is closed. By default, close all open GCLs on shutdown (e.g., SIGTERM). This is an attempt to make corruption less likely.
Accept either SIGINFO or SIGUSR1 to print out stats (to make scripts easier to write).
Add "forgiveness" to gdplogd to allow it to self heal from various problems. May be a security problem, but we need it for avoiding catastrophes while we are debugging.
Make sure the log shows something when gdplogd aborts.
Allow state dump on SIGQUIT or SIGABRT as well as assertion failures.
Add GCL cache consistency checking.
Do detailed GCL cache dump on assertion failure (for debugging).
Arrange to dump gdplogd process state after assertion failure.
Additional debugging for request locking. Change _gdp_req_free API so it can NULL-out the pointer to the request.
Take number of available file descriptors into account when reclaiming the GCL cache.
Break out physical implementation to an explicit class so new implementations (e.g., in memory) can be added easily.
Checkpoint: extent code closer, compiles, doesn't run.
Update license. No code changes.
Include copyright in all files. There are scripts to do bulk replacement of the copyright if desired.
Don't inform subscribers if daemon shuts down --- the subscriber will recover when it re-issues the subscription.
Don't log gdplogd shutdowns as LOG_ALERT messages (use LOG_INFO).
Fix some initialization bugs that caused mutex failures.
Improve command line usage messages for gcl-create and gdplogd.
Add gdplogd -s flag (or swarm.gdplogd.crypto.strictness parameter) to set how strictly signatures are enforced.
Signature checking working. No action taken on bad signature yet. Still considerable cleanup left to be done.
Add -N flag to gdplogd to allow setting of the routing name. Primarily for testing; especially useful when starting up several containers from a single image.
Cleanup for code walkthrough: * comments * some function rearrangement * make data structure detail printing more consistent
Minor debugging/error output tweaks for clarity.
Try to at least reduce the number of compile warnings on gcc. It compiles clean on the NUC (Debian 7, gcc 4.7.2) but still gives some errors on Ubuntu 14.04.1 (gcc 4.8.2).
Always print our GDP routing name on startup.
Snapshot: working on timing out subscriptions.
Extend PING so it can check an individual subscription. Note: doesn't work properly if one client has multiple subscriptions on the same GCL, since it actually checks to see if there exists /any/ subscription from that client. Other random cleanups (naming and formatting).
Implement gdplogd shutdown actions: withdraw advertisements, explicitly terminate all subscriptions.
Use ep_app_fatal instead of ep_app_abort in some cases.
Unify command and ack/nak processing; unify Associative and LRU GCL caches; multiple other cleanups.
Restarting when the routing layer goes away now works. Cleanup needed.
Snapshot in getting reconnection going --- still not there.
Snapshot in getting fallbacks for connections to the routing plane. Still doesn't work, but at least works for the case where the first address in the list is available. It's also clear that _gdp_req_new needs substantial work; as it is, gdplogd can handle commands and...
First steps toward allowing dynamic reconnection to the GDP if connections are lost.
Allow for multiple log daemons; in particular, GCL creation is now a separate program which requires the name of a log server.
First cut on the dual-layer model. The old GDP daemon is now the GDP log daemon; instead of listening to connections it initiates them. This commit does not include the routing layer.
First cut at new architecture for GDP (separate routing from log layer). This compiles but does not run.
Move comment about size of thread pool to the correct place.
Rename gdplogd directory so it matches the name of the binary.