[cairo] Server RAID failure and various mailing list problems
cworth at cworth.org
Mon Aug 15 13:40:31 PDT 2005
The machine that hosts cairographics.org recently suffered some disk
So, far I've verified that the cairo CVS repository remains intact.
But the mailing lists had several problems. Here are the problems I've
identified so far and the progress we've been able to make on
1) Delivery of new messages was broken for about a day.
If you're receiving this message, then delivery is now working
again. If you had sent a message and received a rejection,
please try again.
2) New subscriptions were broken for about a day.
This should be fixed now, (and I'll double check it). If you
had this problem then I don't know why I'm even mentioning it,
since you won't be receiving this answer.
3) The existing subscription list was corrupted and the system
automatically reverted to an old copy (from some hard-to-identify
time).The old copy was missing on the order of 200 subscriptions,
and would certainly also include addresses that people had since
For the missing subscriptions, I grepped through the mail
logs from Saturday and added back all the addresses I could
identify as having received some list messages that went
out. If I missed some, unfortunately I don't have any way to
notify the affected individuals.
For the previously-removed subscriptions, I also don't have
any information on which addresses those are. So, I apologize
if you have received this message in error. Please feel free
to unsubscribe again  and we'll try to avoid this problem
occurring again in the future.
4) The mailing list archives were definitely corrupted. As of this
morning, several of the index pages from the last few months were
filled with garbage.
I ran the command to rebuild the archives from the giant mbox
file that mailman maintains. After the rebuild, the archive
indexes look much better. They're not quite perfect, as can be
seen by the few "No subject" messages with partial contents at
the end of this page:
I personally don't plan to put much more effort into repairing
the archives further. But if anyone wants to look at the .mbox
files and find the corrupted messages, we could replace them
with known-good messages from private copies. I can provide
known-good messages from my private copy, and I can also
supply the giant .mbox file that mailman uses if the per-month
mbox files it provides on the web page are insufficient for
tracking down the problems.
 I had a pre-failure, rsync-based copy of the cairo CVS repository
which I compared against a post-failure version. The differences that
I saw all seem to correlate with recent commits.
 For the benefit of anyone who needs to fix similarly affected
lists, here's what I did. I took the Message-Id from the header of a
message that previously went out with the correct subscription
list. Then I found the log file that corresponds to that date and
un-gzip-ed it. Finally I ran the following commands:
export MSGID=1124029513.30753.7.camel at localhost.localdomain
for id in $(grep "message-id=<$MSGID>" $LOG | sed -e 's/^.*: \([0-9A-F]*\): message-id.*/\1/'); do grep "$id: to=" $LOG | sed -e "s/^.*$id: to=<\([^>]*\)>.*/\1/"; done | sort | uniq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/cairo/attachments/20050815/a2872d71/attachment.pgp
More information about the cairo