Discussion Forums/Proposal
This is bug 598060.
What's the current set-up?
All our discussion forums have three access points - Google Groups for web, news.mozilla.org (hosted by Giganews) for NNTP, and lists.mozilla.org (hosted by us) for email. Each is used by different sections of the community, and web-based is particularly useful for emailing links to people and for searching.
Messages put into GG go from there to news, and from there to mail. Messages put in as news go in both directions. Messages put in as mail go from there to news and from there to GG.
mail <----> news <----> Google Groups
What's the problem?
Spam. Coming in from Google Groups to the newsgroups and mailing lists.
The spam is currently coming in from Google Groups, then goes to news, and thereby to the mailing lists. So we can run it through SpamAssassin then, when it comes under our control, but it's already in news and GG.
What can be done about the spam in Google Groups archives?
The way we have it set up now, nothing. :-( Once spam gets in to the GG archive, Google Groups don't provide a way to cancel it.
Doesn't Google Groups have moderation controls?
Only for groups which are natively hosted there, not ones which are mirrored from elsewhere.
So could we move to being fully hosted at Google Groups, to get moderation controls?
John Resig would suggest that was a mistake and a move in the wrong direction.
So what can we do?
1. Make Google fix things (unlikely).
2. Move to another provider for the web-based interface to our groups. Previously, we've rejected this as their UIs are not as good and their search is not as powerful. But we could consider:
3. The plan below.
What is the cunning plan?
If we set all the Google Groups and newsgroups to "moderated", both Giganews and Google Groups will email us new posts for approval before posting them. We can then run them through SpamAssassin (as we currently do with new mailing list posts) and any other approval mechanisms we want before they appear anywhere.
After that, Mailman can auto-moderate the lists to approve everything which passes the filters.
So all messages effectively enter the system via the mailing list interface, and are propagated to news and GG.
But hang on. Wouldn't mailman prevent posts from people who aren't list subscribers?
Good catch. We'd need a global whitelist for posters, made up of all those subscribed to any of the lists, plus the individual "can post even though he's not a member" whitelists for each group. That would allow most people to post. Some people who have previously just used news or GG would need to be added, in a one time operation, which would be done by the moderator of the group they first posted to after the change was made, when that moderator got their "X tried to post" message.
We already have such a global "can post" list, made from the subscriber lists (i.e. if you are a subscriber to any list right now, you can post to all of them); we'd just need to add the individual whitelists to the pool as well.
What positive effects would this have?
Assuming SpamAssassin is working well, this would stop the spam. And if it doesn't, SpamAssassin is under our control and we can tune it. Our destiny is in our own hands.
What negative effects will this have?
During the 24 hours of changeover: some messages may appear in one medium and not in others. We would blog to make people aware of the issue.
Messages may take longer to propagate, because there are extra steps (email from Google Groups or Giganews to us, and a reply). We're not sure what the delay will be, but it could be up to 7-8 minutes from submitting a message to seeing it appear in any of the three access points.
The above-mentioned one-time approvals would need to be done.
Groups which, by their nature, have a regular influx of new posters (e.g. support groups) might have trouble. What we do is an open question - we might use good moderator coverage, or we might exclude them from this (trading off spam for easy access).
What resources are needed to make this happen?
A couple of hours of IT's time, plus whatever's necessary to warn people that it's about to happen.
