| Branch: | Tag: | Revision:

gdp / gdp / gdp_priv.h @ master

History | View | Annotate | Download (33 KB)

# Date Author Comment
21a05dde 09/24/2020 12:55 PM Eric Allman

Get better elapsed time information for performance tuning.

3e95a6ed 09/19/2020 12:27 PM Eric Allman

Make elapsed time reporting more generic & measure more operations.

c9a90ca9 09/17/2020 12:40 PM Eric Allman

Don't assume Berkeley creation service address. It's caused too much

49d76189 01/14/2020 01:06 PM Eric Allman

Version 2.2.3: Make failure to read a secret key for a writable log an
error instead of waiting for the first append to fail (this can be
turned off using gdp_open_info_set_no_skey_nonfatal, which causes
gdp_gin_open to return a warning instead of an error). Add a -S flag to...

94e663bc 05/07/2019 01:40 PM Eric Allman

Fix subscription renewal to avoid having gaps. It isn't clear that
req->act_ts is still useful, but we'll keep it around for a bit longer.

b6fd1b13 04/09/2019 04:16 PM Eric Allman

Try to make sure re-advertising doesn't occur too quickly after a

5c666732 04/08/2019 05:00 PM Eric Allman

HONGD compilation glitches.

0e2c49f6 04/06/2019 04:40 PM Eric Allman

Force MariaDB/MySQL to use a TCP connection instead of a named pipe;
refactor to centralize database initialization to make this easier;
using new interfaces, make gdp-name-add easier to use (notably password

e1c055ad 04/02/2019 05:51 PM Eric Allman

First cut at getting signature checking working again. This is just a
partial fix --- more to come.

59fe1e09 04/02/2019 05:51 PM Eric Allman

Change datum->inuse to a flags word to allow additional flags.

1042a70f 04/02/2019 05:51 PM Eric Allman

Change ep_crypto_{md|sign|vrfy}_new to return an error status so we can
do a better job of diagnosing problems.

a1e5eca3 03/22/2019 11:59 AM Eric Allman

Fix advertising and withdrawal to actually delay between each
advertisement. Required changing handling of event loop startup and

c70b1607 02/09/2019 07:49 AM Eric Allman

Updates for clean compile on Ubuntu 18.04, including finishing upgrade
of OpenSSL to 1.1.0. Changes were surprisingly large. Bump the patch
number for ease of identification (this version also includes updates
for log creation service).

8f99877c 01/24/2019 08:41 AM Eric Allman

Don't default to using Berkeley's HONGDS.

866c2ca2 01/23/2019 01:17 PM Eric Allman

Pass flags to GDP library initialization routines. Add flags to disable
Zeroconf and suppress initialization of HONGDS, which is unused in some

c87dd166 01/23/2019 07:56 AM Eric Allman

Update license date on all files. No functional changes.

32dc12ee 01/22/2019 05:06 PM Eric Allman

Fix a problem with locating secret keys for logs when using a search
path. Do some prep for making this more flexible. It turns out the
semantics of -K in some applications has changed, but the current
behavior still matches the documentation.

There may be security implications with allowing the ability to name...

d953d8ef 01/02/2019 11:59 AM Eric Allman

Put a default password on the gdp_user MariaDB user to avoid getting
caught by security scans.

47efc98f 12/20/2018 04:37 PM Eric Allman

Change name of log creation service to
edu.berkeley.eecs.gdp.service.creation for consistency with the naming
scheme; tell the log creation service to respond to this name.

0c195bbd 12/04/2018 05:26 PM Eric Allman

Make all references to the Human-Oriented Name to GDPname Directory
Service Database use "hongdb" in the name rather than "namedb" for

e435842c 11/28/2018 07:42 PM Eric Allman

Make be the default database location for the
Human-Oriented Name to GDPname directory.

Standardize on user gdp being uid 133; consistent uids are important for
volumes in containers.

92adefd5 11/28/2018 09:00 AM Eric Allman

Split variable swarm.gdplogd.log.dir into (the root
of persistent data) and swarm.gdplogd.log.dir (relative to the other).
This is primarily to make container management easier.

Also, when reading parameter files, allow auto-qualification with the...

80da996e 10/24/2018 12:53 PM Eric Allman

Mega-change: new object creation API with updated metadata; internal
name is hash of metadata, not hash of human name; query and maintain
human-oriented name to GDPname directory (HONGD).

34e502ee 09/12/2018 01:41 PM Eric Allman

Second cut at sorting asynchronous data records. Currently all done with
simulation (-Dtest.gdp.chan.seqno=102 randomizes the incoming sequence
numbers), and so missing or duplicated incoming records have not been

e6319882 09/04/2018 09:09 AM Eric Allman

First pass at ordering results of asynchronous reads. Works if the
reads come in pre-sorted, but so far untested for out-of-order delivery.

ee0cddb9 08/24/2018 02:16 PM Eric Allman

Add optional signature (proof) checking when reading records. Lightly
tested. Seems to do the right thing if the signature does verify, but
fails incorrectly if it does not. Does not do any hash chain checking

5edf4ad2 08/23/2018 12:43 PM Eric Allman

Pass Layer 4 network sequence number to higher layers.

825689f9 08/10/2018 05:07 PM Eric Allman

Tweaks to hopefully do a better job of handling out-of-order results for
asynchronous reads.

80b79281 07/31/2018 01:07 PM Eric Allman

Remove vestiges of old signature implementation.

caef25cd 07/31/2018 11:08 AM Eric Allman

Restructure signature verification code to move it from gdplogd into
libgdp in preparation for verifying signatures on reads. Version number
bumped to 2.0.4.

c9f4745b 07/27/2018 10:52 AM Eric Allman

Begin implementation of signature checking by readers.

59c6c7e0 07/26/2018 10:50 AM Eric Allman

First cut at external => internal name mapping using MySQL. It is
currently not an error if the mapping doesn't exist --- it just falls
back to the old SHA256 behavior --- and there is no automated addition
of the mapping to the database when you create a log. To make this work...

caa69652 07/20/2018 01:02 PM Eric Allman

Make event queue a TAILQ instead of an STAILQ to avoid n2 algorithm
when results arrive in order.

e95fef13 07/20/2018 11:59 AM Eric Allman

First cut at fixing bug that sometimes caused an "End of Results" PDU to
be delivered before some of the results themselves. This involves
passing a "number of results sent" field in the "End of Results" PDU and
saving that PDU until the indicated number of results are processed....

b7d4c21d 07/14/2018 10:47 AM Eric Allman

Recompute data hash on every append to avoid common error of people
resetting the datum buffer but not the datum itself; the cached data
hash was left over from the previous data and hence was wrong.

2bd9199f 06/27/2018 11:28 AM Eric Allman

Don't throw an assertion failure if an unimplemented command comes in.

4aa32996 06/22/2018 02:46 PM Eric Allman

Pass GIN through to requests so that events can be properly correlated.

ada5f328 06/20/2018 07:15 AM Eric Allman

Improvements to event and metadata printing.

1c4fea03 06/20/2018 07:14 AM Eric Allman

Fix problems with hash chains on logs that did not have have crypto

9f6be413 06/15/2018 03:26 PM Eric Allman

Cleanup signature and hash type handling. There's no point in storing
the MD algorithm with each datum since it is fixed per GOB, nor in
passing it over the wire since it is just a blob unless you have either
the secret or public key, either of which contain that information...

bd278f2e 06/08/2018 11:45 AM Eric Allman

Snapshot: change physical interface to make more efficient use of SQLite
when retrieving sets. Use this for asynchronous reads.

8cfe4117 05/25/2018 01:22 PM Eric Allman

It turns out that including a GOB pointer in the datum causes too many
locking problems. Just pass the gob in as needed.

eae1d3ec 05/25/2018 01:22 PM Eric Allman

The grand renaming: GCL is (mostly) gone in favor of GOB and GIN.

2e65953f 05/25/2018 01:22 PM Eric Allman

Simplify the datum interface back to what it was before. Basic
subscriptions and subscription timeout seems to work now.

fa7cfdb9 05/25/2018 01:22 PM Eric Allman

Snapshot: compiles cleanly with updated protocol and crypto
support, but completely untested.

92243bf4 05/25/2018 01:22 PM Eric Allman

Snapshot in on-disk format conversion from ad hoc files to SQLite
databases. Compiles but is completely untested.

10125206 02/27/2018 06:02 PM Eric Allman

First step at adding QoS information to subscriptions.

c9f8e411 02/07/2018 01:31 PM Eric Allman

Better debugging for GOB reference counting.

44a5b0ef 01/24/2018 09:57 AM Eric Allman

Other fixes necessary to compile with a C++ compiler:
* Subtlety different semantics for extern keyword.
* Inability to do a forward declaration of a static variable.
* Structs must be initialized in the same order as declared.
* Cannot initialize a fixed size character array with a quoted string....

f67d0170 01/24/2018 09:45 AM Eric Allman

Be more careful with enums, which are not just aliases for integers in
C++. Most importantly, use gdp_cmd_t instead of various int types for
commands and acks.

328ab6a8 01/23/2018 05:34 PM Eric Allman

Changes for signed/unsigned conflicts and other type nits. C++ is much
more picky about these than C, but this is probably justified. Sadly,
some of the changes are pretty ugly.

7f74c3b3 01/18/2018 04:09 PM Eric Allman

Fix signature generation and checking.

63dc8142 01/17/2018 12:02 PM Eric Allman

Change req->md to req->digest to mirror the name in the GOB and to avoid
confusion between "message digest" and "metadata".

4a89fdda 01/17/2018 11:34 AM Eric Allman

First cut at flattening out protobuf definition. Some changes to data
structure printing (e.g., _gdp_pdu_dump) to get indenting more rational
so as to make reading the output easier.

d65c01e2 01/09/2018 11:18 AM Eric Allman

First cut at running Layer 5 network protocol version 4 over old
(version 3) Layer 4 stack. Substantial parts are unimplemented,
including signature generation and checking, read or subscribe by
timestamp or hash, and others. No asynchronous operations including...

194715b4 11/27/2017 10:12 AM Eric Allman

Don't leak reqs when doing async read/append.

6c983368 11/13/2017 10:09 AM Eric Allman

Snapshot. Trying to get advertising working on startup. This may be a
larger problem if advertisements need to use challenge-response given
that the I/O thread has been eliminated in gdplogd.

d92b6c0d 11/08/2017 12:56 PM Eric Allman

Merge branch 'eric/net4' of into eric/net4

0c9fce43 11/08/2017 12:38 PM Eric Allman

Snapshot. Not tested.

06d39be9 11/08/2017 12:38 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

f2bf61a4 11/08/2017 12:38 PM Eric Allman

Snapshot. Not tested.

5a73c6ad 11/08/2017 12:36 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

4a6db795 10/27/2017 07:35 PM Eric Allman

First cut at "delete log" command (lightly tested).

fec93aac 10/24/2017 02:38 PM Eric Allman

First cut at separating GCL instances (GINs) from GDP Objects (GOBs).
This is intended to fix some semantic problems with subscriptions. End
users always use GINs (aliased to gdp_gcl_t for now for compatibility),
and there is one GIN for each call to gdp_gcl_open (i.e., no returning...

54ef7d04 09/03/2017 10:51 AM Eric Allman

Fixed a case where gdp_gcl_incref touched the GCL cache via
touch even though the cache was not locked.

f2bf1027 08/31/2017 03:55 PM Eric Allman

Snapshot. Basic things still seem OK, create still seems iffy.

9cacb7ea 08/29/2017 04:43 PM Eric Allman
  • Fix resource leak on some subscriptions.
  • Fix bogus assertion failure on gdplogd shutdown.
11029b9c 08/29/2017 01:59 PM Eric Allman

Snapshot on reliability work.

b8f93c05 08/09/2017 01:33 PM Eric Allman

Basic things seems to work, including passing helgrind race condition
tests. Still more testing needed.

42443eab 08/09/2017 07:46 AM Eric Allman

SNAPSHOT: still with some locking problems.

1b7c480e 08/05/2017 02:17 PM Eric Allman

SNAPSHOT: fewer race conditions; still need more testing.

9c515e64 07/26/2017 06:21 PM Eric Allman

Initial implementation of unsubscribe.

f095a16d 07/25/2017 01:41 PM Eric Allman

When doing final close on a GCL, free all events associated with it.
Add some additional parameter checking macros.

3a1f110b 07/14/2017 02:37 PM Eric Allman

Snapshot. Not tested.

7ac90e00 07/14/2017 02:34 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

b28f9bd6 07/14/2017 01:33 PM Eric Allman

Snapshot. Not tested.

3bc20136 07/14/2017 01:33 PM Eric Allman

First cut at new net protocol. Compiles, but not yet tested (not even
once --- do not try to run this!).

f4611e29 07/07/2017 01:24 PM Eric Allman

Add allocated GCL statistics gathering and printing.

cf05aa39 06/22/2017 02:46 PM Eric Allman

Some code cleanup for previous commit.

ddb857b7 06/20/2017 12:52 PM Eric Allman

Cache freed GCL handles so we can avoid double frees.

284a8bac 05/11/2017 01:50 PM Eric Allman

Add gdp_gcl_open_info_set_signkey_cb to set a callback to get the secret
key. This is to avoid asking for a decryption password if it isn't
going to be used (e.g., if you are opening a GCL that is already in the
cache). This is particularly important for gdp-rest.

ee6b5440 05/10/2017 01:17 PM Eric Allman

Attempt to allow GCLs to be cached in applications. This is
specifically for gdp-rest, but there may be other cases where it is

6dfc87d5 04/11/2017 10:31 AM Eric Allman

Significant changes to locking to avoid problems with threaded clients.

a08413a6 03/29/2017 01:18 PM Eric Allman

Add request allocation statistics (for debugging).

59f6ff3f 03/28/2017 03:20 PM Eric Allman

Adjust some lock ordering.

7882a3b0 03/23/2017 10:13 PM Eric Allman

Add order specification for some leaf locks.

a72e8d44 03/23/2017 09:27 PM Eric Allman

Declare expected lock ordering for chan, datum, gcl, and req objects.

b068acef 03/18/2017 07:46 PM Eric Allman

Rationalize swarm.gdp.subscr.deadintvl and swarm.gdplogd.subscr.timeout.
Rename deadintvl to be timeout and swarm.gdp.subscr.pokeintvl to
swarm.gdp.subscr.refresh for consistency.

006cd152 03/14/2017 03:35 PM Eric Allman

Default extended cache debugging on for now. Should be turned off
before release.

f52acd4d 03/13/2017 06:33 PM Nitesh Mor

Fixed typo in IF_LIST_CHECK_OK declaration

7816b117 03/13/2017 05:17 PM Eric Allman

Implement extreme list checking in an attempt to find circular loops in
linked lists.

a6e527fb 03/10/2017 11:17 AM Eric Allman

Arrange to lock initialization so that it isn't possible for an activity
thread to continue before I/O event thread has initialized.

e6b0a042 03/09/2017 02:54 PM Eric Allman

Subscriptions seem to get expired now, but only when another record is
written (should happen spontaneously in order to mop up file

1c226959 03/06/2017 01:17 PM Eric Allman

Fairly extensive locking changes to avoid races/deadlocks.

3e98a709 03/03/2017 04:59 PM Eric Allman

More lock debugging improvements. Some of these changes may be too
expensive when running without debugging, so they should be on a compile

47d57c57 03/02/2017 03:26 PM Eric Allman

Move GDP_ASSERT_MUTEX_IS[UN]LOCKED into EP library; this implementation
is much less portable and relies on peeking into the secret mutex data,
but has the advantage that it seems to work. Also improves some of the
lock debugging output to add more information.

85b68a8f 02/10/2017 03:23 PM Eric Allman

Fix a lock ordering problem; add Helgrind hints.

897accec 02/06/2017 10:41 AM Eric Allman

Update licenses to 2017.

a104a10f 02/06/2017 10:36 AM Eric Allman

Split req->pdu into two: cpdu holds the PDU for the command and rpdu
holds the PDU for the response. This makes it easier to make restart
work and the code to understand. However, it also means more copying of
data and a general lack of elegance. It also means that the datum in...

b9083a08 12/11/2016 03:36 PM Eric Allman

GDP_MUTEX_CHECK_IS[UN]LOCKED only work if the mutex is of type
EP_THR_MUTEX_ERRORCHECK. Adjust accordingly.