CPAN6Bottom - UP

German Perl Workshop, 21 feb 2007 Mark Overmeer

CPAN is..

“CPAN is the best thingthat ever happened

to Perl”

GDB#1

CPAN is..

“CPAN is the best thingthat ever happened

to Perl”

CPAN is..

“CPAN is the best thingthat ever happened

to Perl5”

CPAN is...

Just over 10 years youngyounger than Perl5 WWW UNIX

CPAN is...

Just over 10 years youngyounger than Perl5 WWW UNIX

... developed before break-through of XML/SOAP/WSDL CVS/SVN/SVK/GIT spam, viruses unicode

...concerns...

11k distributions, a lot is bad quality orpoorly maintained

5700 authors, who is who?

who is making the rules of CPAN?

...concerns...

11k modules, a lot is bad quality orpoorly maintained

5700 authors, who is who?

who is making the rules of CPAN? GBD#2

What will happen when infected code is detected on CPAN? Will Perl loose itsbest thing?

...future...

Where will we store Perl6 modules pir,

pbc,pod only,let the big name-space battles commence!

ParTcl? Brainf*ck?

etc etc

Perl6PodPirPieTonParTclPbc

C AN = CP6ANCPAN6

CPAN6

The Beginning

The process

Author

CPAN Userspause download

CPAN modules

The process

Author

Store Readers

publish buy

Books

Release

Why do you release material? to share with many people

Release

Why do you release material? to share with many people

When do you release material? when it is ready

Release

Why do you release material? to share with many people

When do you release material? when it is ready

What can be “released”? coherent collections of bytes in files

Release

Why do you release material? to share with many people

When do you release material? when it is ready

What can be “released”? coherent collections of bytes in files

How do we release materials? ftp-servers (CPAN), CDs/DVDs,

Releases ≠ Revisions

Revisions a logical sequence of development steps of a file each step may solve some bugs each step extends the code and documentation steps are created by a single person

Releases are created to be distributed a defined `frozen' state of the software

(some combination of file versions) published by someone who is responsible

CPAN ≠ YA VCS

CPAN is a collection of releases distributing information name-spaces and trust searching in the collection

Version Control Systems manage (related) (development) revisions of files collecting pieces of information traceable changes tracing and solving bugs selected revisions of selected file are released

What is interesting?

publish the information authors, publisher

What is interesting?

publish the information authors, publisher

storing the information stored where for how long who sets the rules

What is interesting?

publish the information authors, publisher

storing the information stored where for how long who sets the rules

download the information search, distinguish who can download: user

What is interesting?

publish the information authors, publisher signatures

storing the information stored where archive for how long rules who sets the rules trust

download the information search, distinguish name-spaces who can download access

What is interesting?

publish the information[CPAN] authors, publisher signatures pause-id

storing the information stored where archive

ftp.cpan.org for how long rules

ever who sets the rules trust Andreas &

co

download the information search, distinguish name-spacesone who can download access

everyone

Out of scope

for the end-users: how the release was created, as long as it is

“stable” and “trustable”

for the authors: what the end-user is doing with it, as long as

he/she has access permissions.

Out of scope

for the end-users: how the release was created, as long as it is

“stable” and “trustable” packaging tools: MakeMaker/Module::Build development versions: CVS, ...

for the authors: what the end-user is doing with it, as long as

he/she has access permissions. installation tools: CPAN.pm/CPANPLUS

Out of scope

for the end-users: how the release was created, as long as it is

“stable” and “trustable” packaging tools: MakeMaker/Module::Build development versions: CVS, ...

for the authors: what the end-user is doing with it, as long as

he/she has access permissions. installation tools: CPAN.pm/CPANPLUS

Data distribution can be content independent

Really unimportant

how the data it is packaged tar, zip, separate files

Really unimportant

how the data it is packaged tar, zip, separate files

how it is compressed gzip, bzip2, diff

Really unimportant

how the data it is packaged tar, zip, separate files

how it is compressed gzip, bzip2, diff

how it is transported ftp, http, mail, tape, printed

Really unimportant

how the data it is packaged tar, zip, separate files

how it is compressed gzip, bzip2, diff

how it is transported ftp, http, mail, tape, printed

where it is actually located ftp mirror, cd/dvd distribution, git depot

Really unimportant

how the data it is packaged tar, zip, separate files

how it is compressed gzip, bzip2, diff

how it is transported ftp, http, mail, tape, printed

where it is actually located ftp mirror, cd/dvd distribution, git depot

These could bestall be hidden topublishers and

end-users

Needs for Perl6

place for Perl6, Pir, Pasm, ParTcl, ...

requires Perl5 to install Perl6?

extended versioning scheme

explicit version no deletions: huge pkg list

parallel authoring DBI-TIMB, DBI-MARKOV

Problems with Perl5

Interface breaking changesspamassassin 2 vs. spamassassin 3

Version of distribution/which distribution contains installed package.

Long @INC search paths for pm files

Impossible to oversee effect of an install request to the system

NO QUICK FIX!but a good alternative

CPAN6 because

Compared to CPAN a wish for information independent: reuse of coding effort more name-spaces: DBI for perl5, DBI for perl6 more security; PGP/SSL authentication clear maintenance rules; no GBD archive “tricks” like subsets (cpan-mini) remove ad-hoc configuration of mirrors complex search, server-side know before download: license,

nested dependencies, ... release deprecation, release under embargo and more

CPAN6 definitions

Release: a set of files distributed for a specific purpose to end-users. in Perl5: “a distribution”

Project: a set of releases, sharing the name but with different version identifiers. a name-space entry

Archive: a set of projects, collected for some purpose. storage, rules, entry-point an URI

Collections

Perl5 distributions Perl6 (& friends) distributions

conference materials use case wikipedia articles linux distributions photo or music collections use case company's press releases newspaper articles cooking recipes pgp public keys

Split CPAN

CPAN6: idea of having a collection of releases,

combined into archives

uploading releases downloading releases

distributing the archives like mirrors combining archives like ftp sites filtering archives like cpan-mini

CPAN6 is the DISTRIBUTING

Split CPAN

CPAN6 is the distributing (network) Pause6:

handles one archiveis one name-spaceis set of releaseswith some functional relation

a set of releases relate to a project(name) projects are allocated to authors authors have a well-defined identity an archive has well-defined rules, by a board

Pause6 is ADMINISTRATION

Split CPAN

CPAN6 is the distributing (network) Pause6 is the administration (archives) CPAN6.pm

installs Perl5, Perl6 etc distributions blends into the platform collects meta-data to publish releases programming language specific component

CPAN6.pm for INSTALLATION

Split CPAN

CPAN6 is the distributing Pause6 is the administration CPAN6.pm installs releases locally

Of courseperl -MCPAN6 'install Test::More'

will work!

Pause6

Pause6 is just one possible implementation of the CPAN6 ideas

Releases have project-name utf8 version utf8 state type loads of (optional) meta-data follow-up information

Release strands

0 = allocate project name and get name-space rules

solid arrow:non-breaking follow-up

dashed arrow:code fork

Release states

Release state changes require signatures

Release states are uploading, initiated by publisher published, distributable to deployers embargo, uploaded but blocked released, by the authors deprecated, by the authors expired, by the authors rejected, by the board/authors installed, by the end-user

Project types

project types are: publication by user, set of

files constitution archive

configuration identity public keys, ... archive reference license description

therefore everything follows same release protocol one transport implementation covers all aspects

Use Cases

Board writes constitution Changes in constitution Opening and restricting name-spaces

Authors start project Publisher uploads a release

CPAN

CPAN6/Pause6 terms

CommissionerCPAN Pause point to upload to, for publishers executes constitution for the archive board

Deployers search for end-users

StoresCPAN mirrors

download locations

Scribes transport processes publishers to commissioner, comm. to deployers,

deployers to users, archive hierarchies

CPAN6 concept

Pause6 Implementation

Hierarchy

Scribes can be configured to create archive hierarchies, for example: global cpan-perl5 business level sub-set department sub-set system local personal installed personal development

CPAN6 network

Use Case

Ftp-server with mirrors:initializing a deployer

User contacting new archive

Status

CPAN6/Pause6 global design document

CPAN6 design and implementation doc

Pause6 design and implementation doc

Start of an implementation:the creation of large building bricks

Promises

All data handling UTF8 All data definitions in XML schema's

No new data exchange protocolsuse HTTP, FTP, RSYNC, ...

No new communication mechanismsuse SOAP, RPC, ...

No new security systemsuse PGP, SSL, MD5, SHA-1, ...

Coherent Documentation

Major parts of documentation are generated.Saved Mail::Box 700k characters

A must for OO programsexist for many years, used for many years

OODoc

Data Exchange

I hate XML, but...

Data Exchange

I hate XML, but...

I hate SOAP::Lite, so...

Data Exchange

I hate XML, but...

I hate SOAP::Lite, so...

in usable state, for this CPAN6 purpose

XML::Compile

XML::Compile

Under control of a schema pure Perl real XML real XML pure Perl perl template

(example) XML template

(example)

Validating, name-spaces, any/anyAttribute, substitutionGroups, WSDL, SOAP, fast

No XML in your Perl program!

Platform independence

OS independence File::Spec File::Basename File::Copy File::Remove File::Temp File::Path File::Find

often used, OS related Cwd Slurp tainting

delete $ENV{PATH} $^O IO::File File::stat Sys::Hostname $0 $PROGRAM

Platform independence

No File::Spec, File::Basename, File::Find, etc Path::Class:my $dir = dir 'foo','bar'; # Path::Class::Dirmy $file = file 'bob','file.txt'; # Path::Class::File

# 'bob/file.txt' on Unix, 'bob\file.txt' on Windowsprint "file: $file\n";

my $subdir = $dir->subdir('baz'); # foo/bar/bazmy $parent = $subdir->parent; # foo/barmy $parent2 = $parent->parent; # foo

my $dir2 = $file->dir; # dirname = bobmy $text = $file->slurp;

Platform Independence

No File::Spec, File::Basename, ... Path::Class super-set:

No Cwd, $^O, delete $ENV{PATH}, etc etc

first steps made

Sys::Indep

Sys::Indep

use Sys::Indep devel => 'linux';my $dir = dir '/etc/';my $f1 = file '/etc/passwd'; # absmy $f2 = file 'here.txt'; # rel

my $cwd = program->cwd;my $f3 = file os->home, '.ssh', 'known_keys';

print os->family; # 'unix'my @m = os->volumes; # mount-points

# filename mutilated based on file-system rulesmy $fh = $f3->open('r');

program->find(...);

OS Program File/Dir Filesys

User Interaction

Would like to have command-line for scripting (many options) interactive commands, like IOS Curses::UI Tk / wxWidgets Gtk / Qt XHTML / XUL

Transparently based on GNU gettext() schema generated components set of standard code building bricks

?

where we are...

... discussion and attention contacts with other potential user groups

signing, licensing, trust algorithmsstill a lot of detailed design to do

funding to create base implementationmoney, man-power, test setups

30k€