GDP

I think we may need a way to unsubscribe from a subscription.

gdplogd/logd_proto.c invokes a private _gdp_gcl_unsubscribe(), shouldn't gdp-reader.c?

It looks like gdp-reader is leaking memory when subscribing, part of which could be from not closing the subscription.

Also, how do I add Edward to the list of interested people for this issue

Details below.

Edward wrote:

This subscription demo now works:

$PTII/org/terraswarm/accessor/demo/WatchHandRaisingLight/GDPSubscribe.xml

One thing you could help with would be to figure out how to properly
shut down a GDP subscription.

GDPHelper has this:

public void unsubscribe() {
// FIXME: Properly close the C side. How to do that?
_subscribed = false;
}

The subscription keeps running after the model stops, which is a
problem. Maybe the cause of some deadlocks I've seen...

gdp/doc/gdp-programmatic-api.html says:

Name
gdp_gcl_unsubscribe — Unsubscribe from a GCL [NOT IMPLEMENTED; this interface is probably wrong]
Synopsis

EP_STAT gdp_gcl_unsubscribe(gdp_gcl_t *gcl,
gdp_event_cbfunc_t *cbfunc,
void *udata)

Notes

Unsubscribes to the given {gcl, cbfunc, udata} tuple.
(Should subscribe/multiread return a handle to be passed to this function rather than this interface?)

There is no public gdp_gcl_unsubscribe, but .gdp/gdp_subscr.c has:

/*                                                                                                                                                                               
**  Unsubscribe all requests for a given gcl and destination.                                                                                                                    
*/

EP_STAT
_gdp_gcl_unsubscribe(
                gdp_gcl_t *gcl,
                gdp_name_t dest)
{
        EP_STAT estat;
        gdp_req_t *req;
        gdp_req_t *nextreq;

        do
        {
                estat = EP_STAT_OK;
                for (req = LIST_FIRST(&gcl->reqs); req != NULL; req = nextreq)
                {
                        estat = _gdp_req_lock(req);
                        EP_STAT_CHECK(estat, break);
            nextreq = LIST_NEXT(req, gcllist);
                        if (!GDP_NAME_SAME(req->pdu->dst, dest))
            {
                                _gdp_req_unlock(req);
                                continue;
                        }

                        // remove subscription for this destination                                                                                                              
            EP_ASSERT_INVARIANT(req->gcl == gcl);
                        LIST_REMOVE(req, gcllist);
                        _gdp_req_free(&req);
                }
        } while (!EP_STAT_ISOK(estat));
        return estat;
}

It looks like gdp/gdplogd/logd_proto.c has a cmd_close() function that invokes _gdp_gcl_unsubscribe():

/*                                                                                      
**  CMD_CLOSE --- close an open GCL                                                     
**                                                                                      
**              XXX     Since GCLs are shared between clients you really can't just     
**                      close things willy-nilly.  Thus, this is currently a no-op      
**                      until such time as reference counting works.                    
**                                                                                      
**              XXX     We need to have a way of expiring unused GCLs that are not      
**                      closed.                                                         
*/

EP_STAT
cmd_close(gdp_req_t *req)
{
        EP_STAT estat = EP_STAT_OK;

        req->pdu->cmd = GDP_ACK_SUCCESS;

        // should have no input data; ignore anything there                             
        flush_input_data(req, "cmd_close");

        // a bit wierd to open the GCL only to close it again....                       
        estat = get_open_handle(req, GDP_MODE_ANY);
        if (!EP_STAT_ISOK(estat))
        {
                return gdpd_gcl_error(req->pdu->dst, "cmd_close: GCL not open",
                                                        estat, GDP_STAT_NAK_BADREQ);
        }

        // remove any subscriptions                                                     
        _gdp_gcl_unsubscribe(req->gcl, req->pdu->src);

        //return number of records                                                      
        req->pdu->datum->recno = req->gcl->nrecs;

        if (ep_dbg_test(Dbg, 20))
        {
                char ebuf[100];
                ep_dbg_printf("<<< cmd_close(%s): %s\n", req->gcl->pname,
                                        ep_stat_tostr(estat, ebuf, sizeof ebuf));
        }

        // drop reference                                                               
        _gdp_gcl_decref(&req->gcl);

        return estat;
}

It looks like gdp-reader is leaking memory during subscription, it could be that things are not being closed.

I don't think it is worth fixing memory leaks this close to the annual meeting, but the leaks do indicate that we are not freeing resources.

To replicate this, under RHEL, I create a log ptolemy.actor.lib.jjs.modules.gdp.test.auto.GDPLogSubscribeJS.0.33254535198571245

and then ran

valgrind --leak-check=full gdp-reader -n 1 -s ptolemy.actor.lib.jjs.modules.gdp.test.auto.GDPLogSubscribeJS.0.33254535198571245

Then, on a different machine, ran

echo "bar" | $PTII/vendors/gdp/gdp/apps/gdp-writer -G gdp-03.eecs.berkeley.edu ptolemy.actor.lib.jjs.modules.gdp.test.auto.GDPLogSubscribeJS.0.33254535198571245

The output of valgrind shows:

==6724== LEAK SUMMARY:
==6724==    definitely lost: 67 bytes in 3 blocks
==6724==    indirectly lost: 0 bytes in 0 blocks
==6724==      possibly lost: 33,923 bytes in 11 blocks
==6724==    still reachable: 30,858 bytes in 640 blocks
==6724==         suppressed: 0 bytes in 0 blocks
==6724== Reachable blocks (those to which a pointer was found) are not shown.
==6724== To see them, rerun with: --leak-check=full --show-reachable=yes

It looks like we are leaking around subscription creation?

==6724== 560 bytes in 1 blocks are possibly lost in loss record 634 of 646
==6724==    at 0x4A057BB: calloc (vg_replace_malloc.c:593)
==6724==    by 0x344A611A82: _dl_allocate_tls (in /lib64/ld-2.12.so)
==6724==    by 0x344B2072CC: pthread_create@@GLIBC_2.2.5 (in /lib64/libpthread-2.12.so)
==6724==    by 0x40FFE5: _gdp_gcl_subscribe (gdp_subscr.c:252)
==6724==    by 0x405BE0: gdp_gcl_subscribe (gdp_api.c:472)
==6724==    by 0x405465: do_multiread (gdp-reader.c:238)
==6724==    by 0x405939: main (gdp-reader.c:499)

For your reference, I've attached the output of valgrind.