Re: Schedule for migration to pglister

Lists: pgsql-hackerspgsql-www
From: Stephen Frost <sfrost(at)snowman(dot)net>
To: pgsql-hackers(at)postgresql(dot)org, pgsql-www(at)postgresql(dot)org
Subject: Schedule for migration to pglister
Date: 2017-11-06 15:36:38
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

Greetings,

The PostgreSQL Infrastructure team is working to migrate the project's
mailing lists from the existing system (an ancient and unmaintained
piece of software called "majordomo2") to a newly developed mailing list
system (known as "PGLister"), which better addresses the needs of the
PostgreSQL community and is updated to work with recent improvements in
email technology and spam filtering. These changes will impact certain
aspects of the system but we are hopeful that these changes will have a
minimal impact on users, although everyone will notice the differences.

The changes which we expect to be most significant to users can be found
on the wiki here: https://wiki.postgresql.org/wiki/PGLister_Announce

Our planned migration schedule is as follows:

Nov 6 -
pgsql-www

Nov 13 -
pgsql-hackers
pgsql-bugs
pgsql-committers

Nov 20 -
pgsql-admin
pgsql-general
pgsql-sql
pgsql-jobs
pgsql-novice

Nov 27 -
pgsql-announce

After -
the rest

We will be starting the migration of pgsql-www shortly.

Each list will receive an email with a link to the wiki about the
migration after the list has been migrated.

Thanks!

Stephen


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org, pgsql-www(at)postgresql(dot)org
Subject: Re: Schedule for migration to pglister
Date: 2017-11-06 15:40:16
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

Stephen Frost <sfrost(at)snowman(dot)net> writes:
> Each list will receive an email with a link to the wiki about the
> migration after the list has been migrated.

I suggest doing that the other way 'round. Otherwise, the email
about the change will inevitably go into a lot of peoples' bit
buckets if they haven't adjusted their mail filters yet.

regards, tom lane


From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: [HACKERS] Schedule for migration to pglister
Date: 2017-11-06 15:42:49
Message-ID: CABUevEwXX5Fjv-PyZk_bw4q41HwJ7t3RSu4bcmx84i-EAR=KhQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > Each list will receive an email with a link to the wiki about the
> > migration after the list has been migrated.
>
> I suggest doing that the other way 'round. Otherwise, the email
> about the change will inevitably go into a lot of peoples' bit
> buckets if they haven't adjusted their mail filters yet.
>

The argument for doing it after the migration is that the complaints that
we have received so far have all been from people where email ends up in
the *inbox* after the migration, not the bitbucket. That's the default
action in most peoples MUAs when their rules no longer match...

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>


From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-hackers(at)postgresql(dot)org, pgsql-www(at)postgresql(dot)org
Subject: Re: Schedule for migration to pglister
Date: 2017-11-06 15:43:15
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

Tom,

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > Each list will receive an email with a link to the wiki about the
> > migration after the list has been migrated.
>
> I suggest doing that the other way 'round. Otherwise, the email
> about the change will inevitably go into a lot of peoples' bit
> buckets if they haven't adjusted their mail filters yet.

My thought had been to do before-and-after, but I got complaints from
others that we'd then be spamming a lot of people with email.

We definitely need one after the migration because the new mail *won't*
end up caught in people's filters, and for those who intentionally
filter the list traffic into the garbage because they couldn't figure
out how to unsubscribe, this is going to be most annoying (this is what
we saw with the pgadmin lists and it was quite painful).

Thanks!

Stephen


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: [HACKERS] Schedule for migration to pglister
Date: 2017-11-06 15:46:24
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I suggest doing that the other way 'round. Otherwise, the email
>> about the change will inevitably go into a lot of peoples' bit
>> buckets if they haven't adjusted their mail filters yet.

> The argument for doing it after the migration is that the complaints that
> we have received so far have all been from people where email ends up in
> the *inbox* after the migration, not the bitbucket. That's the default
> action in most peoples MUAs when their rules no longer match...

Hm, around here it's no match -> spam bucket. But in any case, why
would you not want to send it before so that it would end up where
they're accustomed to seeing the list's traffic?

regards, tom lane


From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: [HACKERS] Schedule for migration to pglister
Date: 2017-11-06 15:51:45
Message-ID: CABUevEwZc9XJd3qoRf78Wrs8UnB=Qka539mk6kXT4QbSXuzdQA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Magnus Hagander <magnus(at)hagander(dot)net> writes:
> > On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> I suggest doing that the other way 'round. Otherwise, the email
> >> about the change will inevitably go into a lot of peoples' bit
> >> buckets if they haven't adjusted their mail filters yet.
>
> > The argument for doing it after the migration is that the complaints that
> > we have received so far have all been from people where email ends up in
> > the *inbox* after the migration, not the bitbucket. That's the default
> > action in most peoples MUAs when their rules no longer match...
>
> Hm, around here it's no match -> spam bucket. But in any case, why
>

I think you're quite uncommon in that setup. For obvious reasons, but I've
never heard of anybody other than you doing that :)

> would you not want to send it before so that it would end up where
> they're accustomed to seeing the list's traffic?
>

The experience from the pgadmin lists is that a lot of people have the
lists filtered into folders that they don't check often (or at all). So
they don't notice the migraiton message. But they start noticing once all
the list mail shows up in their inbox instead.

It might well be that we end up getting the other half of people when we do
it this order, but we definitely at a *lot* of people in that first bucket.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: [HACKERS] [pgsql-www] Schedule for migration to pglister
Date: 2017-11-06 16:00:20
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Hm, around here it's no match -> spam bucket. But in any case, why

> I think you're quite uncommon in that setup.

Interesting, because "it's not addressed to me (or any list I'm on)"
is the best single spam filtering rule I know, and has been for a
decade or two.

But we veer far off topic here. Do as you will.

regards, tom lane


From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: [pgsql-www] Schedule for migration to pglister
Date: 2017-11-06 16:07:19
Message-ID: CABUevEzPGP9US_Vj-cGXBPxLZkHo1qaor-6RG6HVXCfBnUtCfQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On Mon, Nov 6, 2017 at 5:00 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Magnus Hagander <magnus(at)hagander(dot)net> writes:
> > On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> Hm, around here it's no match -> spam bucket. But in any case, why
>
> > I think you're quite uncommon in that setup.
>
> Interesting, because "it's not addressed to me (or any list I'm on)"
> is the best single spam filtering rule I know, and has been for a
> decade or two.
>

Oh, I think I misunderstood you. I thought you meant "any list tagged email
that's for a list I don't know what it is". As in "list-id exists but is
unknown".

The way you explain there makes a lot more sense. I think not many people
do that either, mainly since gmail/yahoo/whatnot doesn't make it very easy
to do that. But it does make a lot more sense that way.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>


From: Noah Misch <noah(at)leadboat(dot)com>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org, pgsql-www(at)postgresql(dot)org
Subject: Re: Schedule for migration to pglister
Date: 2017-11-15 09:01:31
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote:
> Our planned migration schedule is as follows:
>
> Nov 6 -
> pgsql-www
>
> Nov 13 -
> pgsql-hackers

When each list migrated, its mbox archives stopped receiving new messages:

https://www.postgresql.org/list/pgsql-www/mbox/pgsql-www.201711
https://www.postgresql.org/list/pgsql-hackers/mbox/pgsql-hackers.201711


From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Noah Misch <noah(at)leadboat(dot)com>
Cc: Magnus Hagander <magnus(at)hagander(dot)net>, pgsql-hackers(at)postgresql(dot)org, pgsql-www(at)postgresql(dot)org
Subject: Re: Schedule for migration to pglister
Date: 2017-11-15 14:18:31
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

Noah,

* Noah Misch (noah(at)leadboat(dot)com) wrote:
> On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote:
> > Our planned migration schedule is as follows:
> >
> > Nov 6 -
> > pgsql-www
> >
> > Nov 13 -
> > pgsql-hackers
>
> When each list migrated, its mbox archives stopped receiving new messages:

This was actually anticipated, though we had been thinking that the
migration would be faster and so it wouldn't end up being such a long
time that the mbox's wouldn't get updated.

The plan is to replace those mbox's (which are created by mj2 and then
rsync'd over currently) with ones generated from the archives database,
but that's something which will basically happen to the entire site and
all the lists at once.

Further, there will actually be some differences between the
archive-generated mbox's vs. those that came from mj2; in particular,
the 'date' used by mj2 is the 'received' date (from what I can tell)
while the 'date' in the archives database is the 'Date:' header from
when the message is sent by the user.

That said, this will make the mbox's match the actual webpages, which
they don't currently because the webpages are built from the archive
database while the mbox's are from mj2. In addition, while trying to
cross-compare the two, I think I'm seeing some cases where mj2 is just
outright dropping messages also (for example, I don't see

https://www.postgresql.org/message-id/20170930224424.ud5ilchmclbl5y5n%40alap3.anarazel.de

anywhere in the mj2 mboxes..).

I'm chatting w/ Magnus about this now and I'm not sure exactly when
we'll end up making the change, but my feeling is that we should either
do it now, or on Monday when we migrate the 'user' lists. Doesn't seem
to me like we should delay it any longer than that.

Thanks!

Stephen


From: Noah Misch <noah(at)leadboat(dot)com>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: Magnus Hagander <magnus(at)hagander(dot)net>, pgsql-hackers(at)postgresql(dot)org, pgsql-www(at)postgresql(dot)org
Subject: Re: Schedule for migration to pglister
Date: 2017-11-15 16:05:50
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> * Noah Misch (noah(at)leadboat(dot)com) wrote:
> > When each list migrated, its mbox archives stopped receiving new messages:
>
> This was actually anticipated, though we had been thinking that the
> migration would be faster and so it wouldn't end up being such a long
> time that the mbox's wouldn't get updated.
>
> The plan is to replace those mbox's (which are created by mj2 and then
> rsync'd over currently) with ones generated from the archives database,
> but that's something which will basically happen to the entire site and
> all the lists at once.

> I'm chatting w/ Magnus about this now and I'm not sure exactly when
> we'll end up making the change, but my feeling is that we should either
> do it now, or on Monday when we migrate the 'user' lists. Doesn't seem
> to me like we should delay it any longer than that.

Gotcha. I, for one, can wait awhile.


From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Noah Misch <noah(at)leadboat(dot)com>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: Schedule for migration to pglister
Date: 2017-11-16 15:20:48
Message-ID: CABUevEyF27LPXkSXidYqeqVqkcSURtzHda=BrPWAFrBaUbcj_w@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On Wed, Nov 15, 2017 at 5:05 PM, Noah Misch <noah(at)leadboat(dot)com> wrote:

> On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> > * Noah Misch (noah(at)leadboat(dot)com) wrote:
> > > When each list migrated, its mbox archives stopped receiving new
> messages:
> >
> > This was actually anticipated, though we had been thinking that the
> > migration would be faster and so it wouldn't end up being such a long
> > time that the mbox's wouldn't get updated.
> >
> > The plan is to replace those mbox's (which are created by mj2 and then
> > rsync'd over currently) with ones generated from the archives database,
> > but that's something which will basically happen to the entire site and
> > all the lists at once.
>
> > I'm chatting w/ Magnus about this now and I'm not sure exactly when
> > we'll end up making the change, but my feeling is that we should either
> > do it now, or on Monday when we migrate the 'user' lists. Doesn't seem
> > to me like we should delay it any longer than that.
>
> Gotcha. I, for one, can wait awhile.
>

mbox access should now be restored, and as Stephen says, now handled by the
Date in the message. Let me know if there are any issues with the new ones.
(They're now generated by the same code as the per-thread ones).

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>


From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Noah Misch <noah(at)leadboat(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: Schedule for migration to pglister
Date: 2017-11-16 15:34:06
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

Magnus, all,

* Magnus Hagander (magnus(at)hagander(dot)net) wrote:
> mbox access should now be restored, and as Stephen says, now handled by the
> Date in the message. Let me know if there are any issues with the new ones.
> (They're now generated by the same code as the per-thread ones).

And just fyi, that also means that complete threads end up in each mbox,
so you'll see cases where a message was posted to, eg, -committers
initially but the reply cc'd hackers, bringing the thread over there,
and now the initial email to -committers and all of the subsequent
emails on the thread will be included in the mbox for both -hackers and
-committers.

Hopefully everyone finds that to actually provide a better experience,
but it's a difference to be aware of.

Thanks!

Stephen


From: Justin Clift <justin(at)postgresql(dot)org>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: Magnus Hagander <magnus(at)hagander(dot)net>, Noah Misch <noah(at)leadboat(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: Schedule for migration to pglister
Date: 2017-11-16 15:51:41
Message-ID: [email protected]
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On 2017-11-16 15:34, Stephen Frost wrote:
> Magnus, all,
>
> * Magnus Hagander (magnus(at)hagander(dot)net) wrote:
>> mbox access should now be restored, and as Stephen says, now handled
>> by the
>> Date in the message. Let me know if there are any issues with the new
>> ones.
>> (They're now generated by the same code as the per-thread ones).
>
> And just fyi, that also means that complete threads end up in each
> mbox,
> so you'll see cases where a message was posted to, eg, -committers
> initially but the reply cc'd hackers, bringing the thread over there,
> and now the initial email to -committers and all of the subsequent
> emails on the thread will be included in the mbox for both -hackers and
> -committers.
>
> Hopefully everyone finds that to actually provide a better experience,
> but it's a difference to be aware of.

In theory :), that sounds better for people reading the web archives.

Only one real way to find out "for sure" though, so fingers crossed. :)

+ Justin


From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Justin Clift <justin(at)postgresql(dot)org>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, Noah Misch <noah(at)leadboat(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, PostgreSQL WWW <pgsql-www(at)postgresql(dot)org>
Subject: Re: Schedule for migration to pglister
Date: 2017-11-16 21:37:21
Message-ID: CABUevExKwR79xO=CA2h9HzMuorLA+rtnJh+Tk5jcEyM_9LB7Lw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Lists: pgsql-hackers pgsql-www

On Thu, Nov 16, 2017 at 4:51 PM, Justin Clift <justin(at)postgresql(dot)org> wrote:

> On 2017-11-16 15:34, Stephen Frost wrote:
>
>> Magnus, all,
>>
>> * Magnus Hagander (magnus(at)hagander(dot)net) wrote:
>>
>>> mbox access should now be restored, and as Stephen says, now handled by
>>> the
>>> Date in the message. Let me know if there are any issues with the new
>>> ones.
>>> (They're now generated by the same code as the per-thread ones).
>>>
>>
>> And just fyi, that also means that complete threads end up in each mbox,
>> so you'll see cases where a message was posted to, eg, -committers
>> initially but the reply cc'd hackers, bringing the thread over there,
>> and now the initial email to -committers and all of the subsequent
>> emails on the thread will be included in the mbox for both -hackers and
>> -committers.
>>
>> Hopefully everyone finds that to actually provide a better experience,
>> but it's a difference to be aware of.
>>
>
> In theory :), that sounds better for people reading the web archives.
>

Um. Nothing has changed in the web interface, only the mbox downloads.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>