Date post: | 01-Apr-2015 |
Category: |
Documents |
Upload: | sydni-ricketts |
View: | 216 times |
Download: | 1 times |
Writing Pluggable Software
Tatsuhiko Miyagawa [email protected]
Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo
Tatsuhiko MiyagawaTatsuhiko Miyagawa
For non-JP attendees …
If you find \ in the code,Replace that with backslash.
(This is MS' fault)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
PlaggableSoftware
Tatsuhiko MiyagawaTatsuhiko Miyagawa
PlaggableSoftware
Tatsuhiko MiyagawaTatsuhiko Miyagawa
PluggableSoftware
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Agenda
Tatsuhiko MiyagawaTatsuhiko Miyagawa
#1How to make
your app pluggable
Tatsuhiko MiyagawaTatsuhiko Miyagawa
#2TMTOWTDP
There's More Than One Way To Deploy Plugins
Pros/Cons by examples
Tatsuhiko MiyagawaTatsuhiko Miyagawa
First-of-all:Why pluggable?
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Benefits
Tatsuhiko MiyagawaTatsuhiko Miyagawa
#1Keep the app design
and code simple
Tatsuhiko MiyagawaTatsuhiko Miyagawa
#2Let the app users
customize the behavior
(without hacking the internals)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
#3It's fun to write
pluginsfor most hackers
(see: Plagger and Kwiki)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
"Can your app do XXX?"
"Yes, by plugins."
Tatsuhiko MiyagawaTatsuhiko Miyagawa
"Your app has a bug in YYY""No, it's the bug in plugin
YYY,Not my fault."
(Chain Of Responsibilities)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Good EnoughReasons, huh?
Tatsuhiko MiyagawaTatsuhiko Miyagawa
#1Make your app
pluggable
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Example
Tatsuhiko MiyagawaTatsuhiko Miyagawa
ack (App::Ack)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
grep –r for programmers
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Ack is a "full-stack"software now.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
By "full-stack" I mean:
Easy installNo configurationNo way to extend
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Specifically:These are hardcoded
Ignored directoriesFilenames and types
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Ignored Directories
@ignore_dirs = qw( blib CVS RCS SCCS .svn
_darcs .git );
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Filenames and languages mapping
%mappings = ( asm => [qw( s S )], binary => …, cc => [qw( c h xs )], cpp => [qw( cpp m h C H )], csharp => [qw( cs )],… perl => [qw( pl pm pod tt ttml t )],
…);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
What if makingthese pluggable?
Tatsuhiko MiyagawaTatsuhiko Miyagawa
DISCLAIMER
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Don't get me wrong Andy,I love ack the way it is…
Just thought it can bea very good example for the
tutorial.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Quickstart:Class::Trigger
Module::Pluggable
© Six Apart Ltd. Employees
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Class::Trigger SYNOPSIS
package Foo;use Class::Trigger;
sub foo { my $self = shift; $self->call_trigger('before_foo'); # some code ... $self->call_trigger('after_foo');}
package main;Foo->add_trigger(before_foo => \&sub1);Foo->add_trigger(after_foo => \&sub2);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Class::TriggerHelps you to
implementObserver Pattern.
(Rails calls this Observer)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Module::Pluggable SYNOPSIS
package MyClass;use Module::Pluggable;
use MyClass;my $mc = MyClass->new();# returns the names of all plugins installed
under MyClass::Plugin::*my @plugins = $mc->plugins();
package MyClass::Plugin::Foo;sub new { … }1;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Setup plugins in App::Ack
package App::Ack;
use Class::Trigger;use Module::Pluggable require => 1;
__PACKAGE__->plugins;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Setup plugins in App::Ack
package App::Ack;
use Class::Trigger;use Module::Pluggable require => 1;
__PACKAGE__->plugins; # "requires" modules
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Ignored Directories (Before)
@ignore_dirs = qw( blib CVS RCS SCCS .svn
_darcs .git );
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Ignored Directories (After)
# lib/App/Ack.pm__PACKAGE__->call_trigger('ignore_dirs.add', \@ignore_dirs);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Ignored Directories (plugins)
# lib/App/Ack/Plugin/IgnorePerlBuildDir.pmpackage App::Ack::Plugin::IgnorePerlBuildDir;
App::Ack->add_trigger( "ignore_dirs.add" => sub { my($class, $ignore_dirs) = @_; push @$ignore_dirs, qw( blib ); },);
1;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Ignored Directories (plugins)
# lib/App/Ack/Plugin/IgnoreSourceControlDir.pmpackage
App::Ack::Plugin::IgnoreSourcdeControlDir;
App::Ack->add_trigger( "ignore_dirs.add" => sub { my($class, $ignore_dirs) = @_; push @$ignore_dirs, qw( CVS RCS .svn _darcs .git ); },);
1;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Filenames and languages (before)
%mappings = ( asm => [qw( s S )], binary => …, cc => [qw( c h xs )], cpp => [qw( cpp m h C H )], csharp => [qw( cs )],… perl => [qw( pl pm pod tt ttml t )],
…);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Filenames and languages (after)
# lib/App/Ack.pm__PACKAGE__->call_trigger('mappings.add', \%mappings);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Filenames and languages (plugins)
package App::Ack::Plugin::MappingCFamily;use strict;
App::Ack->add_trigger( "mappings.add" => sub { my($class, $mappings) = @_; $mappings->{asm} = [qw( s S )]; $mappings->{cc} = [qw( c h xs )]; $mappings->{cpp} = [qw( cpp m h C H )]; $mappings->{csharp} = [qw( cs )]; $mappings->{css} = [qw( css )]; },);
1;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Works great with few lines of
code!
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Now it's time to add Some useful stuff.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Example Plugin:Content Filter
Tatsuhiko MiyagawaTatsuhiko Miyagawa
sub _search { my $fh = shift; my $is_binary = shift; my $filename = shift; my $regex = shift; my %opt = @_;
if ($is_binary) { my $new_fh; App::Ack->call_trigger('filter.binary', $filename, \
$new_fh); if ($new_fh) { return _search($new_fh, 0, $filename, $regex, @_); } }
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Example:Search PDF content
with ack
Tatsuhiko MiyagawaTatsuhiko Miyagawa
PDF filter plugin
package App::Ack::Plugin::ExtractContentPDF;use strict;use CAM::PDF;use File::Temp;
App::Ack->add_trigger( 'mappings.add' => sub { my($class, $mappings) = @_; $mappings->{pdf} = [qw(pdf)]; },);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
PDF filter plugin (cont.)
App::Ack->add_trigger( 'filter.binary' => sub { my($class, $filename, $fh_ref) = @_; if ($filename =~ /\.pdf$/) { my $fh = File::Temp::tempfile; my $doc = CAM::PDF->new($file); my $text; for my $page (1..$doc->numPages){ $text .= $doc->getPageText($page); } print $fh $text; seek $$fh, 0, 0; $$fh_ref = $fh; } },);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
PDF search
> ack --type=pdf Audreyyapcasia2007-pugs.pdf:3:Audrey Tang
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Homework
Use File::ExtractTo handle arbitrary media files
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Homework 2:
Search non UTF-8 files(hint: use Encode::Guess)You'll need another hook.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Summary
Class::Trigger+ Module::Pluggable= Pluggable app easy
Tatsuhiko MiyagawaTatsuhiko Miyagawa
#2TMTOWTDP
There's More Than One Way To Deploy Plugins
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Module::Pluggable+ Class::Trigger
= Simple and Nicebut has limitations
Tatsuhiko MiyagawaTatsuhiko Miyagawa
In Reality, we need more control
over how plugins behave
Tatsuhiko MiyagawaTatsuhiko Miyagawa
1)The order of
plugin executions
Tatsuhiko MiyagawaTatsuhiko Miyagawa
2)Per user
configurationsfor plugins
Tatsuhiko MiyagawaTatsuhiko Miyagawa
3)Temporarily
Disable pluginsShould be easy
Tatsuhiko MiyagawaTatsuhiko Miyagawa
4)How to install
& upgrade plugins
Tatsuhiko MiyagawaTatsuhiko Miyagawa
5)Let plugins
have storage area
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Etc, etc.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Examples:Kwiki
Plaggerqpsmtpd
Movable Type
Tatsuhiko MiyagawaTatsuhiko Miyagawa
I won't talk aboutCatalyst plugins
(and other framework thingy)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Because they're NOT
"plug-ins"
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Install plugins And now you write
MORE CODE
Tatsuhiko MiyagawaTatsuhiko Miyagawa
95% of Catalyst plugins
Are NOT "plugins"But "components"
95% of these statistics is made up by the speakers.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Kwiki 1.0
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Kwiki Plugin code
package Kwiki::URLBL;use Kwiki::Plugin -Base;use Kwiki::Installer -base;
const class_id => 'urlbl';const class_title => 'URL Blacklist DNS';const config_file => 'urlbl.yaml';
sub register { require URI::Find; my $registry = shift; $registry->add(hook => 'edit:save', pre =>
'urlbl_hook'); $registry->add(action => 'blacklisted_url');}
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Kwiki Plugin (cont.)
sub urlbl_hook { my $hook = pop; my $old_page = $self->hub->pages->new_page($self-
>pages->current->id); my $this = $self->hub->urlbl; my @old_urls = $this->get_urls($old_page->content); my @urls = $this->get_urls($self->cgi-
>page_content); my @new_urls = $this->get_new_urls(\@old_urls, \
@urls); if (@new_urls && $this->is_blocked(\@new_urls)) { $hook->cancel(); return $self->redirect("action=blacklisted_url"); }}
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Magic implementedin Spoon(::Hooks)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
"Install" Kwiki Plugins
# order doesn't matter here (according to Ingy)Kwiki::DisplayKwiki::EditKwiki::Theme::BasicKwiki::ToolbarKwiki::StatusKwiki::Widgets# Comment out (or entirely remove) to disable# Kwiki::UnnecessaryStuff
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Kwiki plugin config
# in Kwiki::URLBL plugin__config/urlbl.yaml__urlbl_dns: sc.surbl.org, bsb.spamlookup.net,
rbl.bulkfeeds.jp
# config.yamlurlbl_dns: myowndns.example.org
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Kwiki plugins are CPAN modules
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Install and Upgrade plugins
cpan> install Kwiki::SomeStuff
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Using CPAN as a repository
Pros #1:reuse most of current CPAN infrastructure.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Using CPAN as a repository
Pros #2:Increasing # of
modules= good motivation
for Perl hackers
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Cons #1:Installing CPAN deps
could be a mess(especially for Win32)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Cons #2:Whenever Ingy
releasesnew Kwiki, lots of plugins just break.
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Kwiki plugin storage
return if grep {$page->id} @{$self->config->cached_display_ignore};
my $html = io->catfile( $self->plugin_directory,$page->id)->utf8;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Kwiki 2.0
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Same as Kwiki 1.0
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Except:plugins are now
in SVN repository
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Plagger plugin
package Plagger::Plugin::Publish::iCal;use strict;use base qw( Plagger::Plugin );
use Data::ICal;use Data::ICal::Entry::Event;use DateTime::Duration;use DateTime::Format::ICal;
sub register { my($self, $context) = @_; $context->register_hook( $self, 'publish.feed' => \&publish_feed, 'plugin.init ' => \&plugin_init, );}
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Plagger plugin (cont)
sub plugin_init { my($self, $context) = @_;
my $dir = $self->conf->{dir}; unless (-e $dir && -d _) { mkdir $dir, 0755 or $context->error("Failed to mkdir $dir: $!"); }}
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Plagger plugin storage
$self->conf->{invindex} ||= $self->cache->path_to('invindex');
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Plagger plugin config
# The order matters in config.yaml# if they're in the same hooksplugins: - module: Subscription::Config config: feed: - http://www.example.com/ - module: Filter::DegradeYouTube config: dev_id: XYZXYZ - module: Publish::Gmail disable: 1
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Plugins Install & Upgrade
> notest cpan Plagger# or …> svn co http://…/plagger/trunk plagger> svn update
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Plagger impl.ripped off by
many apps now
Tatsuhiko MiyagawaTatsuhiko Miyagawa
qpsmtpd
Tatsuhiko MiyagawaTatsuhiko Miyagawa
mod_perl for SMTP
Runs with tcpserver, forkserver or Danga::Socket standalone
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Plugins: Flat files
rock:/home/miyagawa/svn/qpsmtpd> ls -F pluginsasync/ greylistingauth/ hosts_allowcheck_badmailfrom http_configcheck_badmailfromto ident/check_badrcptto logging/check_badrcptto_patterns miltercheck_basicheaders parse_addr_withhelocheck_earlytalker queue/check_loop quit_fortunecheck_norelay rcpt_okcheck_relay relay_onlycheck_spamhelo require_resolvable_fromhostcontent_log rhsblcount_unrecognized_commands sender_permitted_fromdns_whitelist_soft spamassassindnsbl tlsdomainkeys tls_cert*dont_require_anglebrackets virus/
Tatsuhiko MiyagawaTatsuhiko Miyagawa
qpsmtpd plugin
sub hook_mail { my ($self, $transaction, $sender, %param) = @_;
my @badmailfrom = $self->qp->config("badmailfrom") or return (DECLINED);
for my $bad (@badmailfrom) { my $reason = $bad; $bad =~ s/^\s*(\S+).*/$1/; next unless $bad; $transaction->notes('badmailfrom', $reason) … } return (DECLINED);}
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Actually qpsmtpdPlugins are "compiled"to modules
Tatsuhiko MiyagawaTatsuhiko Miyagawa
my $eval = join("\n", "package $package;", 'use Qpsmtpd::Constants;', "require Qpsmtpd::Plugin;", 'use vars qw(@ISA);', 'use strict;', '@ISA = qw(Qpsmtpd::Plugin);', ($test_mode ? 'use Test::More;' : ''), "sub plugin_name { qq[$plugin] }", $line, $sub, "\n", # last line comment without newline? );
$eval =~ m/(.*)/s; $eval = $1;
eval $eval; die "eval $@" if $@;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
qpsmtpd plugin config
rock:/home/miyagawa/svn/qpsmtpd> ls config.sample/config.sample:IP logging
require_resolvable_fromhostbadhelo loglevel rhsbl_zonesbadrcptto_patterns plugins
size_thresholddnsbl_zones rcpthosts
tls_before_authinvalid_resolvable_fromhost relayclients tls_ciphers
Tatsuhiko MiyagawaTatsuhiko Miyagawa
config/plugins
# content filtersvirus/klez_filter
# rejects mails with a SA score higher than 2spamassassin reject_threshold 20
Tatsuhiko MiyagawaTatsuhiko Miyagawa
config/badhelo
# these domains never uses their domain when greeting us, so reject transactions
aol.comyahoo.com
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Install & Upgrade plugins
Just use subversion
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Tatsuhiko MiyagawaTatsuhiko Miyagawa
MT plugins are flat-files
(or scripts that call modules)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
MT plugin code
package MT::Plugin::BanASCII; our $Method = "deny";use MT; use MT::Plugin;
my $plugin = MT::Plugin->new({ name => "BanASCII v$VERSION", description => "Deny or moderate ASCII or Latin-1
comment",}); MT->add_plugin($plugin);MT->add_callback('CommentFilter', 2, $plugin, \
&handler);
Tatsuhiko MiyagawaTatsuhiko Miyagawa
MT plugin code (cont)
sub init_app { my $plugin = shift; $plugin->SUPER::init_app(@_); my($app) = @_; return unless $app->isa('MT::App::CMS'); $app->add_itemset_action({ type => 'comment', key => 'spam_submission_comment', label => 'Report SPAM Comment(s)', code => sub { $plugin->submit_spams_action('MT::Comment',
@_) }, } );
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Tatsuhiko MiyagawaTatsuhiko Miyagawa
MT plugin storage
require MT::PluginData;my $data = MT::PluginData->load({ plugin => 'sidebar-manager', key => $blog_id },);unless ($data) { $data = MT::PluginData->new; $data->plugin('sidebar-manager'); $data->key($blog_id);}$data->data( \$modulesets );$data->save or die $data->errstr;
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Order control
MT->add_callback('CMSPostEntrySave', 9, $rightfields, \&CMSPostEntrySave);
MT->add_callback('CMSPreSave_entry', 9, $rightfields, \&CMSPreSave_entry);
MT::Entry->add_callback('pre_remove', 9, $rightfields, \&entry_pre_remove);
Defined in plugins. No Control on users end
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Conclusion
Flat-filesvs.
Modules
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Flat-files:☺ Easy to install (Just grab it)
☻ Hard to upgradeOK for simple plugins
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Modules:☺ Full-access to Perl OO
goodness☺ Avoid duplicate efforts of CPAN ☻ Might be hard to resolve deps.
Subversion to the rescue(could be a barrier for newbies)
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Nice-to-haves:Order control
Temporarily disable pluginsPer plugin config
Per plugin storage
Tatsuhiko MiyagawaTatsuhiko Miyagawa
Resources
Class::Triggerhttp://search.cpan.org/dist/Class-Trigger/Module::Pluggablehttp://search.cpan.org/dist/Module-Pluggable/Ask Bjorn Hansen: Build Easily Extensible Perl
Programshttp://conferences.oreillynet.com/cs/os2005/view/e_sess/6806qpsmtpdhttp://smtpd.develooper.com/MT pluginshttp://www.sixapart.com/pronet/plugins/Kwikihttp://www.kwiki.org/Plaggerhttp://plagger.org/