GDP Project List¶
Projects on this list are intended to be "student size", that is, not too large and not too small. "Fix this bug" is too small. "Redesign the universe" is too large.
See also GDPv1 Prototype for Nitesh's project about his research prototype of the next iteration of the GDP.
Wiki links are to pages to flesh out the description, give current status, and document any issues.
WARNING: This list has not been kept up to date. Check with Eric to see if any tasks you're interested in have already been completed or are already being worked on. Also, some priorities have changed.
See also https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F19/index_projects.html and https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F19/index_projects.html. Note that these sites include projects not related to the GDP project.
The order is vaguely from highest to lowest priority (in my opinion), then from most specific and smallest tasks to less specific and larger tasks, with the proviso that if B depends on A, A will be listed first. See also https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F19/index_projects.html for some bigger projects (but beware, that list will probably not be maintained after the end of 2019).
- Integrate Replication Code. EricA.
- Proofs in Content Responses. Include metadata in read requests that prove that the data is legitimate. Related to Light Weight Signatures.
- Light Weight Signatures. Reduce append overhead in client and server, probably by using hashes on intermediate records.
- Compressed Network Protocol. FlowIDs etc.
- Compressed Disk Representation.
- Advisory Log Truncation. Mark old data as expendable.
- Mandatory Log Truncation. Guarantee that old data is expunged.
- Ephemeral Logs, i.e., in-memory.
- Control Plane Interface, i.e., some way of adding Control Plane services into the GDP infrastructure.
- Log Directory Service mapping human names to internal names. Probably requires Control Plane Interface, since this service should probably exist in the control plane.
- Service Directory Service. Ways to find available services.
Log Creation Service. Requires Control Plane Interface.(Completed).
- System Visualization.
- Sensor Visualization. Already have some.
- Stress Testing. GrifffinP working on test program.
- Unit Testing. We have neither infrastructure nor tests.
- Functional Testing. The current tests are limited.
- Regression Testing. Jenkins verifies that checkins compile, but that's as far as it goes.
- Agent Identity. Agents (including Log Servers and Routers) must have keypairs in order to authenticate responses. Needed by Authenticated Routing, Authenticated Responses, Authenticated Advertisements, Scoped Data, and probably others. Design done, implementation half complete.
- Authenticated Routing. GDP Endpoints, including clients and log servers, need to mutually authenticate to routers on startup. Endpoints may decline to share traffic with untrusted routers. An untested version of the Endpoint-side code exists, but the router side needs to be implemented.
- Authenticated Responses, notably on APPEND commands. Requires Agent Identity. Design done, need implementation.
- Authenticated Advertisements, so a rogue server can't advertise a log that they do not have the permissions to serve. Design done, need implementation.
- Separate Creator vs Writer Authorization. This will require introducing Organization CAs.
Metadata Hash as Log Name. Requires Log Directory Service.
- Scoped Data, i.e., keeping private data within an organization. Requires Agent Identity.
- Client Writer State, that is, maintaining current write state between client restarts.
GDP 1.0 Release. We are committed to doing a release in October for Terraswarm. However, we are not committed to a particular set of deliverables.
Tasks That Require Elucidation¶
These tasks aren't yet specific enough to be actionable, but in some cases people are investigating the issues.
- Scalable Routing. DHT based.
- Distributed Logs, so that not all data need appear on a single server.
Perhaps these should not happen.
- Deprecate "GCL" name — just use "log".
Move things here when they are completed so that discussions are not orphaned from the Wiki document tree.