Project

General

Profile

Statistics
| Branch: | Tag: | Revision:

gdp / gdp / gdp_req.c @ master

History | View | Annotate | Download (20.7 KB)

# Date Author Comment
c6d41a3e 07/10/2020 03:37 PM Eric Allman

Debug tweaks.

7874f265 01/08/2020 01:29 PM Eric Allman

Stop using deprecated EP_ASSERT_TEST macro (in favor of EP_ASSERT).

fa872838 06/24/2019 12:39 PM Eric Allman

Minor debugging tweak.

94e663bc 05/07/2019 01:40 PM Eric Allman

Fix subscription renewal to avoid having gaps. It isn't clear that
req->act_ts is still useful, but we'll keep it around for a bit longer.

c87dd166 01/23/2019 07:56 AM Eric Allman

Update license date on all files. No functional changes.

34e502ee 09/12/2018 01:41 PM Eric Allman

Second cut at sorting asynchronous data records. Currently all done with
simulation (-Dtest.gdp.chan.seqno=102 randomizes the incoming sequence
numbers), and so missing or duplicated incoming records have not been
tested.

e6319882 09/04/2018 09:09 AM Eric Allman

First pass at ordering results of asynchronous reads. Works if the
reads come in pre-sorted, but so far untested for out-of-order delivery.

ee0cddb9 08/24/2018 02:16 PM Eric Allman

Add optional signature (proof) checking when reading records. Lightly
tested. Seems to do the right thing if the signature does verify, but
fails incorrectly if it does not. Does not do any hash chain checking
yet.

5edf4ad2 08/23/2018 12:43 PM Eric Allman

Pass Layer 4 network sequence number to higher layers.

825689f9 08/10/2018 05:07 PM Eric Allman

Tweaks to hopefully do a better job of handling out-of-order results for
asynchronous reads.

be93e2ed 08/09/2018 02:42 PM Eric Allman

Change debugging flags; flushing PDU isn't an unusual event, so that
shouldn't be on -D1.

80b79281 07/31/2018 01:07 PM Eric Allman

Remove vestiges of old signature implementation.

3b11c044 07/31/2018 11:04 AM Eric Allman

Don't send normal NAKs to syslog --- just internal errors.

caa69652 07/20/2018 01:02 PM Eric Allman

Make event queue a TAILQ instead of an STAILQ to avoid n2 algorithm
when results arrive in order.

e95fef13 07/20/2018 11:59 AM Eric Allman

First cut at fixing bug that sometimes caused an "End of Results" PDU to
be delivered before some of the results themselves. This involves
passing a "number of results sent" field in the "End of Results" PDU and
saving that PDU until the indicated number of results are processed....

2bd9199f 06/27/2018 11:28 AM Eric Allman

Don't throw an assertion failure if an unimplemented command comes in.

bd278f2e 06/08/2018 11:45 AM Eric Allman

Snapshot: change physical interface to make more efficient use of SQLite
when retrieving sets. Use this for asynchronous reads.

eae1d3ec 05/25/2018 01:22 PM Eric Allman

The grand renaming: GCL is (mostly) gone in favor of GOB and GIN.

8bebf0e3 02/26/2018 11:11 AM Eric Allman

Get basic subscriptions working again.

53393e9c 02/20/2018 03:26 PM Eric Allman

Snapshot of work on getting subscriptions working under Net4. Still
has several problems.

f67d0170 01/24/2018 09:45 AM Eric Allman

Be more careful with enums, which are not just aliases for integers in
C++. Most importantly, use gdp_cmd_t instead of various int types for
commands and acks.

328ab6a8 01/23/2018 05:34 PM Eric Allman

Changes for signed/unsigned conflicts and other type nits. C++ is much
more picky about these than C, but this is probably justified. Sadly,
some of the changes are pretty ugly.

fc0cbc23 01/23/2018 04:09 PM Eric Allman

Changes necessary to compile under C++ because it doesn't understand
that (void *) is compatible with other pointer types. This adds a lot
of explicit casts that are not needed in C.

63dc8142 01/17/2018 12:02 PM Eric Allman

Change req->md to req->digest to mirror the name in the GOB and to avoid
confusion between "message digest" and "metadata".

4a89fdda 01/17/2018 11:34 AM Eric Allman

First cut at flattening out protobuf definition. Some changes to data
structure printing (e.g., _gdp_pdu_dump) to get indenting more rational
so as to make reading the output easier.

d65c01e2 01/09/2018 11:18 AM Eric Allman

First cut at running Layer 5 network protocol version 4 over old
(version 3) Layer 4 stack. Substantial parts are unimplemented,
including signature generation and checking, read or subscribe by
timestamp or hash, and others. No asynchronous operations including...

194715b4 11/27/2017 10:12 AM Eric Allman

Don't leak reqs when doing async read/append.

f2bf61a4 11/08/2017 12:38 PM Eric Allman

Snapshot. Not tested.

5a73c6ad 11/08/2017 12:36 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

fec93aac 10/24/2017 02:38 PM Eric Allman

First cut at separating GCL instances (GINs) from GDP Objects (GOBs).
This is intended to fix some semantic problems with subscriptions. End
users always use GINs (aliased to gdp_gcl_t for now for compatibility),
and there is one GIN for each call to gdp_gcl_open (i.e., no returning...

1ca7751e 09/01/2017 03:45 PM Eric Allman

More work on making helgrind happy. This seems to be pretty close,
maybe worth a try in production.

f2bf1027 08/31/2017 03:55 PM Eric Allman

Snapshot. Basic things still seem OK, create still seems iffy.

21d0bb38 08/30/2017 11:02 AM Eric Allman

Helgrind fixes. Still kicks out errors, but so far they all look to be
on a trylock, and hence shouldn't actually cause any deadlocks.

9cacb7ea 08/29/2017 04:43 PM Eric Allman
  • Fix resource leak on some subscriptions.
  • Fix bogus assertion failure on gdplogd shutdown.
11029b9c 08/29/2017 01:59 PM Eric Allman

Snapshot on reliability work.

1b7c480e 08/05/2017 02:17 PM Eric Allman

SNAPSHOT: fewer race conditions; still need more testing.

66f6c374 08/02/2017 09:11 AM Eric Allman

Make Helgrind annotations work properly for (most?) race conditions.

9c515e64 07/26/2017 06:21 PM Eric Allman

Initial implementation of unsubscribe.

2f4bea8c 07/14/2017 11:30 AM Eric Allman

Better fix for problem with attempted fix in commit:a2d73fc0.

7d2fc3ef 07/07/2017 10:34 AM Eric Allman

Temporary fix for bug introduced by previous commit.

a2d73fc0 07/07/2017 08:44 AM Eric Allman

Fix a resource exhaustion problem (GCL reference counting with
subscriptions).

2a8cda22 06/24/2017 02:50 PM Eric Allman

Cleanup (and hopefully simplification) of assertion subsystem. Lots of
changes, but nothing particularly obtuse.

4c5bdf54 05/09/2017 03:31 PM Eric Allman

Clear stale data out of old req before adding to free list.

6dfc87d5 04/11/2017 10:31 AM Eric Allman

Significant changes to locking to avoid problems with threaded clients.

aa62c039 03/29/2017 01:46 PM Eric Allman

Avoid diagnosing a non-error having to do with reqs on a gcl list.

a08413a6 03/29/2017 01:18 PM Eric Allman

Add request allocation statistics (for debugging).

639ca33f 03/28/2017 03:17 PM Eric Allman

Remove some commented out code lines that will not be revived.

7882a3b0 03/23/2017 10:13 PM Eric Allman

Add order specification for some leaf locks.

a72e8d44 03/23/2017 09:27 PM Eric Allman

Declare expected lock ordering for chan, datum, gcl, and req objects.

eadbbf93 03/15/2017 02:54 PM Eric Allman

Add additional debugging on "locking free request" error.

25a3c2d2 03/14/2017 03:35 PM Eric Allman

Fix some locking issues.

7816b117 03/13/2017 05:17 PM Eric Allman

Implement extreme list checking in an attempt to find circular loops in
linked lists.

951b6fb6 03/10/2017 05:05 PM Eric Allman

Change EP_ASSERT_MUTEX_IS(UN)?LOCKED to EP_THR_MUTEX_ASSERT_IS(UN)LOCKED
for consistency with other routine names.

e6b0a042 03/09/2017 02:54 PM Eric Allman

Subscriptions seem to get expired now, but only when another record is
written (should happen spontaneously in order to mop up file
descriptors).

65f23ae5 03/08/2017 04:20 PM Eric Allman

Subscriptions work again.

d15adf12 03/07/2017 12:55 PM Eric Allman

More work on locking issues.

1c226959 03/06/2017 01:17 PM Eric Allman

Fairly extensive locking changes to avoid races/deadlocks.

47d57c57 03/02/2017 03:26 PM Eric Allman

Move GDP_ASSERT_MUTEX_IS[UN]LOCKED into EP library; this implementation
is much less portable and relies on peeking into the secret mutex data,
but has the advantage that it seems to work. Also improves some of the
lock debugging output to add more information.

85b68a8f 02/10/2017 03:23 PM Eric Allman

Fix a lock ordering problem; add Helgrind hints.

897accec 02/06/2017 10:41 AM Eric Allman

Update licenses to 2017.

6c41bcf1 02/06/2017 10:36 AM Eric Allman

Minor debug cleanups.

a104a10f 02/06/2017 10:36 AM Eric Allman

Split req->pdu into two: cpdu holds the PDU for the command and rpdu
holds the PDU for the response. This makes it easier to make restart
work and the code to understand. However, it also means more copying of
data and a general lack of elegance. It also means that the datum in...

7bc0e2cf 12/11/2016 08:51 AM Eric Allman

Snapshot: get basic reading working again with new lock changes.

e7b2a80c 11/22/2016 02:22 PM Eric Allman

Extensive work on reliability, notably sharing a datum between threads.
Increases reliability, but still not fully tested. Basic things seem to
work though. Note: this may disrupt the replication code.

590c85ec 11/12/2016 12:06 PM Eric Allman

Fix bogus error message when flushing GCLs from cache.

4f976ad9 11/11/2016 01:52 PM Eric Allman

Soften another assertion; this one is a true "can't happen", but
recovery is easy.

6a78b581 10/29/2016 12:06 PM Eric Allman

Attempt to recover a bit more gracefully from one "assertion failure"
like error. There is a memory leak in the recovery, but that can't be
helped.

2bd3be0e 10/28/2016 11:24 AM Eric Allman

First cut at using kinder, gentler assertions (mostly in gdp library).

8a3b26f9 10/21/2016 02:30 PM Eric Allman

Fix a couple of locking problems: one a doublely locked mutex and one a
thread that is spawned while holding a mutex.

c3992831 10/06/2016 07:08 AM Eric Allman

Fix a potential infinite loop in _gdp_req_freeall.

3348ff2a 10/05/2016 10:36 AM Eric Allman

Fix bogus assertion failures or other errors when gdp_gcl_open can't
find a route to the log.

8204d4b2 09/28/2016 10:45 AM Eric Allman

Implement gdp_gcl_read_async. This also fixes some obscure problems
with other asynchronous operations, and tweaks debugging flags to make
it easier to pick out the operations you want to see.

4d6ff457 09/27/2016 04:52 PM Eric Allman

Fix problem that might occur if an initial response PDU to a command
such as MULTIREAD is delayed; follow-on data was treated as a response to
the initial command rather than as asynchronous events.

6811b717 09/26/2016 01:41 AM Eric Allman

Log a message if request id wraps around (it's only a uint32). This
does not prevent reuse! There should really be a _gdp_rid_free to make
this possible.

1aa4fc2c 09/04/2016 03:14 PM Eric Allman

Fix problem raised by issue #16 (threaded applications failed in
unpredicatable ways). The problem was that calls to gdp_gcl_open need to
be single threaded so that there will be a unique name => handle mapping
for GCLs. This puts a mutex around the entire open call; not ideal...

0d8c13e8 08/27/2016 08:36 AM Eric Allman

Make many parameters "const" where possible.

4dc7a0b2 08/24/2016 12:08 PM Eric Allman

Fix Assertion Failure after timeout (related to GCLF_INUSE).

5a64048e 07/15/2016 10:17 AM Eric Allman

Fix likely race condition in GCL req list handling; improved debugging.

8cc28925 05/26/2016 05:10 PM Eric Allman

First cut on implementing FWD_APPEND command. Still not working for all
cases.

7d33511d 05/17/2016 11:21 AM Eric Allman

More cleanup of reference counting for GCLs that have error conditions.

283110f2 05/15/2016 11:06 AM Eric Allman

Additional debugging for request locking. Change _gdp_req_free API so
it can NULL-out the pointer to the request.

4a7d784f 05/12/2016 07:32 PM Eric Allman

Change _gdp_gcl_decref API to automatically null out the pointer ---
this makes it less prone to error.

18624361 04/28/2016 10:37 AM Eric Allman

Fix a race condition where _gdp_req_lock could return a pointer to a
free req. This could only happen if thread T1 got a pointer to the req,
then T2 locked the req, then T1 blocked on the lock, T2 freed the req
and unlocked the req, allowing T1 to continue with the now freed req....

33a462b8 04/27/2016 11:33 AM Eric Allman

Fix gdplogd cache flushing so that expired subscriptions don't prevent
reclaimation of a GCL. This caused long-running gdplogd processes to
run out of file descriptors.

19efef53 04/15/2016 01:26 PM Eric Allman

Fix a problem that was causing occassional unlocks of unlocked mutexes
(only when a subscription timed out).

6bd5476b 12/01/2015 01:22 PM Eric Allman

Update license. No code changes.

055d3009 11/21/2015 01:20 PM Eric Allman

Include copyright in all files. There are scripts to do bulk
replacement of the copyright if desired.

08be4c9c 10/31/2015 11:34 AM Eric Allman

Reliability fixes: reinitialize subscriptions periodically and retry
commands on router or server errors (allows gdplogd restarts with open
connections). Subscriptions will catch up and commands such as APPEND
will complete when the daemon returns. Adds a new thread in clients to...

29f683eb 10/28/2015 02:53 PM Eric Allman

First cut at recovering from gdplogd failures (which will result in
NAK_R_NOROUTE errors). Still doesn't work, but no worse than before.

02de7463 10/27/2015 04:53 PM Eric Allman

Implement subscription renewals. This causes clients who have open
subscriptions to re-issue the subscription every
swarm.gdp.subscr.pokeintvl seconds. It's not complete yet: gdplogd
still needs to expire subscriptions after a timeout.

e87e0a1a 10/27/2015 10:46 AM Eric Allman

Work-in-Progress: automatic subscription renewals. Incomplete.

a14a218e 10/09/2015 11:01 AM Eric Allman

Improve GCL safety checks (asserts on function entry).

f7307b63 07/27/2015 04:30 PM Eric Allman

Signature checking working. No action taken on bad signature yet.
Still considerable cleanup left to be done.

9a47c6c7 07/24/2015 09:38 AM Eric Allman

Another signing snapshot: signatures are created and sent but not
checked. Metadata is returned when a GCL is opened. Still have a
problem with reference counting of GCL handles.

f80ae9c0 07/22/2015 09:54 AM Eric Allman

Snapshot adding signing. Signatures are computed but not sent or
checked.

e2682ba9 05/28/2015 10:36 AM Eric Allman

Implement asynchronous writes. Adds new interfaces:
* gdp_gcl_append_async -- initiate an asynchronous write.
* gdp_event_getstat -- get the status code from an event.
Asynchronous reads can be done using multiread.
Note that asynchronous writes do not attempt retransmissions, unlike...

4102ad3a 03/05/2015 05:59 PM Eric Allman

Cleanup for code walkthrough:
* comments
* some function rearrangement
* make data structure detail printing more consistent

d05d61be 02/21/2015 05:14 PM Eric Allman

Eliminate troublesome GDP_REQ_LOCKED flag; it was really only for
debugging anyway.

4d4f1be3 02/21/2015 05:09 PM Eric Allman

Fix a lock ordering problem (req/gcl). Unfortunately this fix may
introduce a race condition.

7c79c6bc 02/20/2015 02:08 PM Eric Allman

Significant changes to locking and subscriptions:
* Locks on gdp_req_t are now more extensive to avoid data races.
* Care is given to prevent subscription data being processed before
the subscription itself is in place.
Minor changes:
* Move channel pointer into gdp_pdu_t purely as a convenience when...