Re: [Slony1-general] Slony1_funcs broken with 8.1

-- Crossposting to pgsql-hackers --

Jan Wieck wrote:

>
> For the record,
>
> Slony-I uses a pg_listener entry as a locking mechanism to prevent
> multiple concurrent slon processes serving the same node. The function
> Async_Unlisten() is used in a backend function that is called during
> slon startup in an attempt to remove stale pg_listener entries left over
> from a PostgreSQL crash. This function scans pg_listener, tries to
> kill(pid, 0) the backends listed in there in order to check if they are
> alive and if not, calls Async_Unlisten() _with that foreign pid_.
>
> This means, that the very functionality we use is gone. What we have to
> find out is if we still need that functionality, or if something else in
> 8.1 does the cleanup for us during postmaster restart already, so that
> we can skip the whole thing.

Just tried it: LISTEN something, sudo killall -9 postmaster, and after
restart the pg_listener entry was still there.
So postmaster doesn't clean up pg_listener, is slon supposed to do that
with a DELETE FROM pg_listener?

Regards,
Andreas

Andreas Pflug <pgadmin(at)pse-consulting(dot)de> writes:
> So postmaster doesn't clean up pg_listener,

It never has. If you're complaining about this patch
http://archives.postgresql.org/pgsql-committers/2005-10/msg00073.php
you ought to say so, rather than expecting us to guess it from an
out-of-context quote from another mailing list.

As near as I can tell, the technique Jan describes is an abuse of
pg_listener, and I won't feel any great sympathy when it does break
completely, which it will do before long when pg_listener goes away
in the planned rewrite of LISTEN/NOTIFY.

regards, tom lane

Tom Lane wrote:
> Andreas Pflug <pgadmin(at)pse-consulting(dot)de> writes:
>
>> So postmaster doesn't clean up pg_listener,
>
>
> It never has. If you're complaining about this patch
> http://archives.postgresql.org/pgsql-committers/2005-10/msg00073.php
> you ought to say so, rather than expecting us to guess it from an
> out-of-context quote from another mailing list.

Not complaining, just RFC.
But I wonder why postmaster doesn't truncate pg_listener at restart,
since PIDs can't be valid any more (truncating would reduce bloating
too). A redesign based on shmem or so wouldn't keep the data either.

>
> As near as I can tell, the technique Jan describes is an abuse of
> pg_listener, and I won't feel any great sympathy when it does break
> completely, which it will do before long when pg_listener goes away
> in the planned rewrite of LISTEN/NOTIFY.

Well slony uses LISTEN for its main purpose too. I'd guess there's
always a demand to find out which backend is listening, so a pg_listener
(probably a view wrapping a function) will be necessary.

AFAICS a backend that notices loss of client connection will usually
clean up its listener entries, so apparently slony doesn't need to take
care of this, at least for 8.1 (with the postmaster crash exception).

Regards,
Andreas

pgadmin(at)pse-consulting(dot)de (Andreas Pflug) writes:
> Tom Lane wrote:
>> Andreas Pflug <pgadmin(at)pse-consulting(dot)de> writes:
>>
>>> So postmaster doesn't clean up pg_listener,
>> It never has. If you're complaining about this patch
>> http://archives.postgresql.org/pgsql-committers/2005-10/msg00073.php
>> you ought to say so, rather than expecting us to guess it from an
>> out-of-context quote from another mailing list.
>
> Not complaining, just RFC.

It looks as though we can accomplish an equivalent pretty readily.

create table sl_nodelock (
nl_node integer not null unique;
nl_pid integer not null;
);

The "interlock" could be accomplished via the following bit of sorta
pseudocode...

try {
execute "insert into sl_nodelock (nl_node, nl_pid) values (getlocalnodeid, pg_backend_pid());";
} on error {
execute "SELECT pg_stat_get_backend_pid(s.backendid) AS procpid FROM (SELECT pg_stat_get_backend_idset() AS backendid) AS s where procpid = $OTHERPID;";
if (not_found) {
printf ("Found dead slon connection info - cleaning it out!\n");
execute "delete from sl_nodelock;";
} else {
printf ("Could not start node N - other slon still running!");
}
exit -1;
}

> But I wonder why postmaster doesn't truncate pg_listener at restart,
> since PIDs can't be valid any more (truncating would reduce bloating
> too). A redesign based on shmem or so wouldn't keep the data either.

Truncating things at restart just feels scary, even though it is, in
this case, pretty appropriate. I'd rather see the shmem redisgn...

>> As near as I can tell, the technique Jan describes is an abuse of
>> pg_listener, and I won't feel any great sympathy when it does break
>> completely, which it will do before long when pg_listener goes away
>> in the planned rewrite of LISTEN/NOTIFY.
>
> Well slony uses LISTEN for its main purpose too. I'd guess there's
> always a demand to find out which backend is listening, so a
> pg_listener (probably a view wrapping a function) will be necessary.

There are two usages of LISTEN in Slony-I; the other one needs to get
improved, as well, as the event propagation system generates way too
many dead tuple entries at present. I have a patch that easily cuts
it by about half; the other half seems pretty doable too...

> AFAICS a backend that notices loss of client connection will usually
> clean up its listener entries, so apparently slony doesn't need to
> take care of this, at least for 8.1 (with the postmaster crash
> exception).

Interesting.
--
"cbbrowne","@","cbbrowne.com"
http://cbbrowne.com/info/rdbms.html
Rules of the Evil Overlord #53. "If the beautiful princess that I
capture says "I'll never marry you! Never, do you hear me, NEVER!!!",
I will say "Oh well" and kill her." <http://www.eviloverlord.com/>