[Bast-commits] r9607 - ironman/branches/mk-ii/IronMan-Web
idn at dev.catalyst.perl.org
idn at dev.catalyst.perl.org
Tue Jun 29 13:02:00 GMT 2010
Author: idn
Date: 2010-06-29 14:02:00 +0100 (Tue, 29 Jun 2010)
New Revision: 9607
Removed:
ironman/branches/mk-ii/IronMan-Web/Changes
ironman/branches/mk-ii/IronMan-Web/Makefile.PL
ironman/branches/mk-ii/IronMan-Web/README
ironman/branches/mk-ii/IronMan-Web/ironman_web.conf
ironman/branches/mk-ii/IronMan-Web/ironman_web_ironboy.conf
ironman/branches/mk-ii/IronMan-Web/ironman_web_ironman.conf
ironman/branches/mk-ii/IronMan-Web/ironman_web_localhost.conf
ironman/branches/mk-ii/IronMan-Web/todo.pod
Log:
Moving
Deleted: ironman/branches/mk-ii/IronMan-Web/Changes
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/Changes 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/Changes 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1,4 +0,0 @@
-This file documents the revision history for Perl extension IronMan::Web.
-
-0.01 2009-07-01 21:07:26
- - initial revision, generated by Catalyst
Deleted: ironman/branches/mk-ii/IronMan-Web/Makefile.PL
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/Makefile.PL 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/Makefile.PL 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1,43 +0,0 @@
-use inc::Module::Install;
-
-name 'IronMan-Web';
-all_from 'lib/IronMan/Web.pm';
-
-requires 'Catalyst::Runtime' => '5.7015';
-requires 'Catalyst::Plugin::ConfigLoader';
-requires 'Catalyst::Plugin::Static::Simple';
-requires 'Catalyst::Plugin::StackTrace';
-requires 'Catalyst::Action::RenderView';
-requires 'YAML'; # This should reflect the config file format you've chosen
- # See Catalyst::Plugin::ConfigLoader for supported formats
-
-requires 'Catalyst::Controller::reCAPTCHA';
-requires 'Catalyst::View::TT';
-requires 'Catalyst::Model::DBIC::Schema';
-requires 'Data::UUID';
-requires 'Email::Valid';
-requires 'LWP::Simple';
-requires 'XML::Feed';
-requires 'DateTime';
-requires 'XML::OPML';
-requires 'DateTime::Format::HTTP';
-requires 'YAML::XS';
-requires 'IronMan::Schema';
-
-# We need DateTime::Format::SQLite for script/import_csv.pl if using SQLite
-recommends 'DateTime::Format::SQLite';
-
-# We need TryCatch::Error for script/pull_urls.pl
-recommends 'TryCatch::Error';
-
-# We need FCGI::ProcManager to run this as a Fast CGI process.
-recommends 'FCGI::ProcManager';
-
-# Testing deps
-test_requires 'Catalyst::Test';
-
-catalyst;
-
-install_script glob('script/*.pl');
-auto_install;
-WriteAll;
Deleted: ironman/branches/mk-ii/IronMan-Web/README
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/README 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/README 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1 +0,0 @@
-Run script/ironman_web_server.pl to test the application.
Deleted: ironman/branches/mk-ii/IronMan-Web/ironman_web.conf
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/ironman_web.conf 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/ironman_web.conf 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1,9 +0,0 @@
-name IronMan::Web
-frontpage_entries 20
-<branding>
- page_title "Planet Perl Iron Man"
- banner "<h1 class='title'>enlightened perl organisation</h1><p class='dict'><strong>enlightened</strong> |en'litnd|: <em>adjective</em>:<br />having or showing a rational, modern, and well-informed outlook</p>"
- page_header "Planet Perl Iron Man"
- page_footer "Planet Perl Iron Man"
-</branding>
-default_view TT
Deleted: ironman/branches/mk-ii/IronMan-Web/ironman_web_ironboy.conf
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/ironman_web_ironboy.conf 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/ironman_web_ironboy.conf 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1,10 +0,0 @@
-<recaptcha>
-## share ironman "global" key
- priv_key 6LdtfQwAAAAAAHPUBd4M0YquNYsAqjARPPO1jEXn
- pub_key 6LdtfQwAAAAAANfQgaj4u9dACa8-mr6J18HCXWhf
-</recaptcha>
-<Model::FeedDB>
- <connect_info>
- dsn dbi:SQLite:/var/www/ironboy.enlightenedperl.org/ironman/subscriptions.db
- </connect_info>
-</Model::FeedDB>
Deleted: ironman/branches/mk-ii/IronMan-Web/ironman_web_ironman.conf
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/ironman_web_ironman.conf 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/ironman_web_ironman.conf 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1,10 +0,0 @@
-<recaptcha>
-## ironman keys
- priv_key 6Le7UQwAAAAAAFYPa1kYrjElL4IBEoACORjghDk5
- pub_key 6Le7UQwAAAAAAMi84pCrncgnp0lYHEVZmB7h7yI6
-</recaptcha>
-<Model::FeedDB>
- <connect_info>
- dsn dbi:SQLite:/var/www/ironman.enlightenedperl.org/ironman/subscriptions.db
- </connect_info>
-</Model::FeedDB>
Deleted: ironman/branches/mk-ii/IronMan-Web/ironman_web_localhost.conf
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/ironman_web_localhost.conf 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/ironman_web_localhost.conf 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1,10 +0,0 @@
-<recaptcha>
-## localhost keys
- priv_key 6LcsbAAAAAAAANQQGqwsnkrTd7QTGRBKQQZwBH-L
- pub_key 6LcsbAAAAAAAAPDSlBaVGXjMo1kJHwUiHzO2TDze
-</recaptcha>
-<Model::FeedDB>
- <connect_info>
- dsn dbi:SQLite:/home/castaway/plagger/subscriptions.db
- </connect_info>
-</Model::FeedDB>
Deleted: ironman/branches/mk-ii/IronMan-Web/todo.pod
===================================================================
--- ironman/branches/mk-ii/IronMan-Web/todo.pod 2010-06-29 13:01:46 UTC (rev 9606)
+++ ironman/branches/mk-ii/IronMan-Web/todo.pod 2010-06-29 13:02:00 UTC (rev 9607)
@@ -1,216 +0,0 @@
-=head1 IronMan-Web release issues.
-
-Remaining issues for live deployment on 14th of April 2010.
-
-=head2 Page banner. - DONE - IDN
-
-The banner as follows:
-
- Are you flesh? Or are you Iron?
- Take the challenge
- http://ironman.enlightenedperl.org/signup/new_feed
-
-Missing from the top of the page
-
-=head2 Paging.
-
-The detail between the banner detailed above and the very first feed post:
-
- Join the program | Learn about the program | Report a problem
-
- Only showing posts tagged "perl", "cpan" or "ironman" (or containing those words).
- Last updated: 21:25:27 13-Apr-2010 First Previous 1 2 3 4 5 Next Last
-
-The above is missing at the top of the page, though the Older and Newer posts
-links are shown at the bottom of the page.
-
-=head2 Captcha on signup form.
-
-Castaway has added a recaptcha to the signup form on the existing
-site. This needs back porting to the dev site.
-
-=head2 Branding.
-
-These are minor issues that are inconsistent with the existing UI.
-Whilst we need to fix these up, we also need provision to have the
-all.things.per.ly branding restored in the future. Please bear
-this in mind when making the changes such that this can be achieved
-simply.
-
-Page title should be "Planet Perl Iron Man"
-
-The epo banner at the top should read:
-
- enlightened perl organisation
- enlightened |en'litnd|: adjective:
- having or showing a rational, modern, and well-informed outlook
-
-Header in the yellow bar at the top of the page should similarly be:
-
- Planet Perl Iron Man
-
-Footer should be:
-
- Perl Iron Man Planet
-
-or:
-
- Planet Perl Iron Man
-
-The second would be more consistent, though the former is the existing text.
-
-=head2 Images.
-
-For some reason images are being removed from posts. No idea why this might be
-but it's a degredation to existing functionality if we can't figure it out.
-
-=head2 Article headers.
-
-Tags and such are missing from the article headers.
-
-I'm sure this has been fixed in the data move once, but I don't recall how :(
-
-=head1 IronMan-Web feature requests.
-
-=head2 OPML file link.
-
-The existing site shows a link to download the OPML file.
-
-We need something here to download a similar SQLite file with the email addresses removed.
-
-=head2 Spam handling.
-
-The existing site has many spam handling issues. There's lots of crap in the
-database.
-
-We need:
-
-=over
-
-=item * Tools to remove the existing crap from the database.
-
-=item * Support included into Perlanet-IronMan to try and limit the amount that then re-appears.
-
-A number of people have suggested that using Spam Assassin might be a starting
-point for scanning existing content and possibly new content.
-
-=item * We need a quick and simple way to remove spam feeds once they're identified.
-
-This should probably be by feed or post URL.
-
-From IRC discussion:
-
- 12:55 < mst> for the live one it's easy
- 12:55 < mst> just cp the sqlite db first
- 12:55 < castaway> on ironman I always do: login, cd plagger, cp subscriptions.db scubscriptions_pewdespam.db; sqlite3 subscriptions.db
- 12:56 < castaway> any mess, re copy and start again ;)
- 12:56 < castaway> backups++
-
-=item * A name should not appear on the index page until at least one post has been made.
-
-=back
-
-Note from robinsmidsrod in #epo-ironman:
-
- 12:20 < robinsmidsrod> idn: I just wanted to suggest to you to use the http_bl support from https://www.projecthoneypot.org to reduce spam entering the ironman database - I've
- successfully used it on my blog - now I barely have spam entering my blog, and I don't have a captcha installed
- 12:21 < robinsmidsrod> I used the mod_httpbl apache implementation from https://www.projecthoneypot.org/httpbl_implementations.php
- 12:21 < idn> robinsmidsrod: Thanks for the suggestion, I'll stick it in the todo list for investigation.
- 12:23 < robinsmidsrod> sorry, my bad, I didn't use the mod_httpbl module - I actually used a b2evolution plugin - but projecthoneypot has a simple DNS-based API, so it shouldn't be to
- hard to make a perl module to handle it
- 12:23 < robinsmidsrod> it works just as any other DNS-based blacklist
- 12:24 < robinsmidsrod> but the cool thing is that you can actually choose the threshold level for when you will block users
-
-Discussion followed:
-
- 12:24 < idn> That's an interesting idea that I hadn't thought of.
- 12:25 < idn> How would you look to implement it, at collection time or at signup time?
- 12:25 < robinsmidsrod> actually, if you run your own DNS server (and http server) I would suggest to support the project - it is an awesome project (I've been a member for a bit over a
- year)
- 12:25 < robinsmidsrod> the new site is dynamic, right?
- 12:25 < idn> Yes
- 12:26 < idn> Hmm, I'd love to support the project, but my employer isn't very community minded or responsible in that respect.
- 12:26 < robinsmidsrod> so just look up REMOTE_IP, do a lookup against projecthoneypot BL (via DNS) and check the response - if it shows something that looks bad, just block or redirect
- the user to a page that explains the problem, or enable captcha
- 12:26 < robinsmidsrod> idn: well, I support it with my private stuff
- 12:27 < robinsmidsrod> anyone can donate a spare MX pointer ;)
- 12:27 < robinsmidsrod> for some hostname you would probably never use
- 12:27 < idn> Ah, I'm with you.
- 12:28 < robinsmidsrod> mine is XXXmailserver.smidsrod.no which points to a honeypot
- 12:29 < idn> So all I need is some domains rather than any actual kit ;) All of mine are on 123reg which neatly solves that problem.
- 12:30 < robinsmidsrod> me alone have helped catching approx. 20 harvesters and spammers in the last year
- 12:30 < robinsmidsrod> which is nice to know :)
- 12:31 < robinsmidsrod> you can support them by donating an MX entry in your DNS, setting up a "hidden" link on your own websites linking to a honeypot, or you can setup an actual
- honeypot - I've only done the two first
- 12:32 < idn> That seems like a good idea, I'll put it forward to the boss too and see if we can't do something here at work.
- 12:32 < robinsmidsrod> what I do in my blog is that if the remote_ip looks suspicies I redirect the user to my honeypot page, which means that those IPs that are already somewhat fishy
- will be redirected to something that will make them more fishy if they harvest it :)
- 12:33 < idn> I had been contemplating using spam assassin to scan content too
- 12:34 < robinsmidsrod> idn: this is my honeypot page: http://minmailserver.smidsrod.no/
- 12:34 < robinsmidsrod> if you look at the HTML content you'll see that there are some hidden links that harvesters will catch
- 12:35 < idn> There are a couple of problems to address, one is the existing bad feeds (most of which don't appear because they don't use the right keywords) and secondly preventing new
- bad feeds.
- 12:36 < idn> The former is more of a problem due to the way in which the list of signed up users appears on the front page.
- 12:36 < robinsmidsrod> honeypot will only help with the new bad feeds
- 12:36 < idn> Possibly, the site hosting the spam might well be listed
- 12:37 * mst still thinks "only appears in the right bar if they've got at least one post" would be a start
- 12:37 < robinsmidsrod> if you have any IP-adresses linked with existing content you could of course manually run it throuh their BL and see what you find
- 12:37 < robinsmidsrod> mst: I agree with that one
- 12:37 < mst> actual spam posts will get nailed pretty quickly, I think
- 12:37 < idn> mst: Yes, that's what's in the todo I think
- 12:38 < robinsmidsrod> a "report spam" feature is available?
- 12:38 < castaway> robinsmidsrod: no but that'd be handy, care to write one?
- 12:38 < idn> I've thought about that and I've mixed opinions. It seems like it could be open to abuse and or create work for someone to deal with.
- 12:38 < castaway> idn: we run a website, its gonna create work ;
- 12:38 < castaway> ;)
- 12:39 < idn> Yes. But I like to try my best to minimise that ;)
- 12:39 < robinsmidsrod> castaway: I don't have any time available, 200% workload with work + full time studies, but I can explain how it could be created to mitigate moderator
- intervention.
- 12:40 < idn> I was contemplating some kind of scoring system that would blacklist a feed once so many reports have been received from different requesting hosts, but it's still a
- little open to abuse. Coupled with administrative notification and oversight to re-enable if needed.
- 12:40 < robinsmidsrod> castaway: create a form (POST) with a "Report spam" button so that behaving robots won't access it. When enough people have clicked that button the article will
- be blacklisted until a moderator actually clears it from blacklist
- 12:40 < castaway> idn: like, say, bayes? ;)
- 12:41 < robinsmidsrod> idn: exactly the same as I thought
- 12:41 < castaway> robinsmidsrod: makes sense.. (user moderation, yay)
- 12:41 < robinsmidsrod> that way either the author needs to complain to a moderator that his post doesn't show up
- 12:41 < idn> castaway: Erm, not quite, though my understanding of things statistical could be written on the back of a very small pin head....
- 12:41 < robinsmidsrod> because it got blacklisted
- 12:42 < idn> That wouldn't work in that each and every time the feed generated a spam, it would need to be black listed. Though I like where you're going.
- 12:42 < robinsmidsrod> mst: do you have a suggestion on how to calculate how many reports should cause blacklisting to be triggered?
- 12:42 < idn> Blacklist the individual post with some kind of hysteresis, then blacklist the feed once enough posts have been blacklisted.
- 12:42 < mst> idn: er, what?
- 12:43 < mst> why would you have to regen?
- 12:43 < robinsmidsrod> idn: or enable reporting spam on both posts and feeds
- 12:43 < mst> oh, each time. yes.
- 12:43 < mst> idn: that's simple.
- 12:43 < mst> two blacklists and the feed goes.
- 12:43 < idn> There we go then :)
- 12:44 < robinsmidsrod> I'd suggest to put the feed in a quarantine so that it is easy for a moderator to un-blacklist feeds - and once a feed has been un-blacklisted you would increase
- the blacklist threshold
- 12:45 < robinsmidsrod> sometimes member blogs get hijacked and start generating spam, but I guess that problem is much smaller than spammer blogs in general
- 12:46 < idn> I wouldn't remove the feed, they could just sign it up again. I'd opt for blacklisting it and never collecting it again
- 12:47 < robinsmidsrod> if the feed has been in the blacklist for , let's say a month or two, it will be automatically purged from the database
- 12:47 < robinsmidsrod> what you said makes more sense yes :)
- 12:47 < robinsmidsrod> and if they try to signup again you redirect them to a projecthoneypot page :)
- 12:49 < robinsmidsrod> just make sure the report spam button is a form/post button, not a link, or else you'll gather report spam-reports en masse when misbehaving robots come in
- 12:51 < idn> Hmm, that's a good place to use the honeypot stuff too.
- 12:52 < robinsmidsrod> catch the spammers/harvesters by their own bad behaviour :)
-
-
-=head2 standards compliance
-
-=head3 Atom feed validation
-
-L<http://feedvalidator.org/check.cgi?url=http://ironman.enlightenedperl.org/atom.xml>
-
-I'm told this involves fixing L<XML::Feed>.
-
-=head2 Gravatar support?
-
-12:12 < poisonbit> gravatar support could be funny
-12:13 < idn> poisonbit: Cool. I like. I'll add it to the wishlist.
-
-http://en.gravatar.com/
-
-=cut
More information about the Bast-commits
mailing list