Portable perl

How to write code that will work everywhere

Some common fallacies

• Perl is portable, my code is written in perl

– Therefore my code is portable.

Some common fallacies

• Perl is portable, my code is written in perl

– Therefore my code is portable. • WRONG!

Some common fallacies

• Perl is portable, my code is written in perl

– Therefore my code is portable.• WRONG!

• My code uses no XS, calls no XS,

– Therefore portable.

Some common fallacies

• Perl is portable, my code is written in perl

– Therefore my code is portable.• WRONG!

• My code uses no XS, calls no XS,

– Therefore portable.• WRONG!

Who cares about portability

• Portability is only an issue if you genuinely want your code to work everywhere.

• Portability is not a requirement for ad-hoc scripts written for a single environment

Who cares about portability

• Portability is only an issue if you genuinely want your code to work everywhere.

• Portability is not a requirement for ad-hoc scripts written for a single environment

• But… Think of CPAN.– You publish your module– You want it to work on as many platforms as

possible

Nearly all of Perl is portable

• Perl tries to bridge the gaps between different platforms

• It is usually worth the mileage to make modules portable.

• Module authors should be interested in making/keeping their modules portable

Operating system platforms

• Unix

Operating system platforms

• Unix

• MS Win32

Operating system platforms

• Unix

• MS Win32

• Apple Mac

Operating system platforms

• Unix

• MS Win32

• Apple Mac

• VMS

• Other (see perldoc perlport for the full list)

What are the main issues

• File names and paths

• Shell commands

• Environment variables

• User interaction

• Communications

• Multitasking

• Internationalisation

File names

• Unix– /home/me/my_dir/my_file

• Windows– C:\My documents\my_dir\My file.txt

• VMS– MY_DEVICE:[ME.MY_DIR]MYFILE.TXT;1

POSIX filenames

• A platform independent standard/foo/bar/module.pm

lib/Foo/Bar/module.pm

POSIX filenames

• A platform independent standard/foo/bar/module.pm

lib/Foo/Bar/module.pm

• Works on Windows:C:/perl/lib/test.pm

• Even works on VMS/my_device/me/my_dir/myfile.txt

Problems with POSIX filenames

• No provision for a 'volume' or 'device' – (in Unix terminology, a mount point)

• Variations in character set– E.g. Windows allows spaces– Is underscore allowed? Dash? Dollar? More than 1 dot?

Punctuation characters?

• Case sensitivity– On Unix, FOO, Foo and foo are different files– Accented letters

• Problems if you mix native and POSIX syntaxC:/mydir\myfile.txt OK sometimesMYDEV:[MYDIR]SUBDIR/FILE.TXT NO!!

The alternative

• Use native syntax throughout

• Test $^O for the operating system• Do what is necessary to construct a filename

The alternative

• Use native syntax throughout

• Test $^O for the operating system• Do what is necessary to construct a filename

• Fortunately, this has been done for you

use File::Spec;

my $file = File::Spec->catfile( qw( mydev mydir file.txt));

File::Spec and File::Spec::Functions

• File::Spec provides a class methods API

• File::Spec::Functions provides equivalent functions for export.

• The following are exported by default:– canonpath catdir catfile curdir rootdir updir

no_upwards file_name_is_absolute path

• There are also other functions available:– devnull tmpdir splitpath splitdir catpath abs2rel

rel2abs case_tolerant

A word of warning

• File::Spec merely manipulates strings– curdir returns ‘.’ on Unix– canonpath merely removes /. and /foo/.. etc.

• use Cwd to handle real paths– cwd gives you what curdir really is– abs_path does what you would expect

canonpath to do

Other things to watch out for with files

• Symbolic links– Are Unix specific– Many O/S don't even support hard links

• Security and permissions– (rwx rwx rwx) is a Unix convention– Some O/S don’t have the same concept of a

user and a group– VMS ends directories with a .DIR extension

Calling the shell

• Don't do it!– (if you want your code to be portable)

• Thus, avoid pipe files, backticks and system– Unless providing a mechanism for your

interactive user to type in commands.

• If you absolutely must– Stick to commands with a common syntax– Test $^O and provide variants

Shell globbing

• Not all command interpreters expand wildcards– (i.e. not all shells glob). Unix shells do glob.

perl myprog.pl *.txt

• Perl on MS-DOS does not glob• Perl on VMS does glob!

– Though the VMS command line interpreter does not

Environment Variables

• Unix scripters are used to having standard ones:• HOME TERM SHELL USER etc.• These are all Unix specific.• $ENV{PATH} is common to most O/S's

– But not all, VMS for example.

– VMS does not need PATH, as you are required to enter the command "RUN" in order to run a program, or @ for a script.

Interacting with the user

Problem:

• You really need to talk to the user.

• stdin, stdout and stderr could be redirected

• You want to prompt him for a password.

• You don't want to proceed until you have received the password.

Non-portable solutionopen TTYIN,'/dev/tty' or die "Failed to open terminal";open TTYOUT,'>/dev/tty' or die "Failed to open terminal";

# Autoflush on for TTYOUTselect TTYOUT;$| = 1;

# Echo offsystem "stty -echo";print "Enter password:";my $pass = <TTYIN>;chomp $pass;

# Echo on againsystem "stty echo";

validate_pass($pass);

What was non-portable?

• Use of "system" to control echoing

• /dev/tty - the terminal will not be called this

• A minor point: echoing might already be off

A better way

• Refer to perlfaq8:– How do I ask the user for a password?

• Unfortunately, the POD for Term::ReadKey says:

– ReadLine MODE [, Filehandle]• This call is currently not available under Windows.

use Term::ReadKey;

ReadMode('noecho'); my $password = ReadLine(0);

Combining Term::ReadKey and Term::ReadLine

use Term::ReadKey; use Term::ReadLine;

my $term = Term::ReadLine->new;ReadMode('noecho'); my $password = $term->readline ("Enter Password:");ReadMode('restore');

Other nice features of Term::ReadLine

• Allows command recall from the history if the O/S supports it

• Can operate inside Tk

Communications

• File sharing with a different architecture

• Exchanging packets over a network

Communications

• File sharing with a different architecture

• Exchanging packets over a network

Look to standards• Each side changes the data to comply to the

standard format

Line termination

• How exactly is "\n" stored or transmitted?– On Unix, a single ASCII \012 (line feed)– On Windows, \015 \012 (carriage return, line feed)

– On Mac OS, just plain \015 (carriage return)

Line termination

• How exactly is "\n" stored or transmitted?– On Unix, a single ASCII \012 (line feed)– On Windows, \015 \012 (carriage return, line feed)

– On Mac OS, just plain \015 (carriage return)• This means that "\n" can stand for two characters

– This is why chomp is better than chop

• This is important when reading foreign files• This is also important when sending and receiving

packets

Line termination

• Remember: "\n" ne "\012"

• perldoc perlport goes into more detail.

| Unix | DOS | Mac | --------------------------- \n | LF | LF | CR | \r | CR | CR | LF | \n * | LF | CRLF | CR | \r * | CR | CR | LF | --------------------------- * text-mode STDIO

Use binmode on non-standard files(i.e. other than local text files)

• binmode makes no difference on Unix ASCII files.

• Other O/S need to differentiate between text files and binary files.

• binmode is needed for UTF-8 locale differences

• "seek" and "tell" will produce spurious results if binmode is not correctly set.

Number representation

• You will often be unable to read another system's binary numbers– Differing endianness– Different floating point representations

• Use a "network" standard– pack and unpack format 'n' and 'N'– IEEE floats are standard and portable

Multitasking

• Beware forks or threads– If you want to be portable, you are best off avoiding

fork altogether.

– But you can probably use a multitasking module if it has been written with portability in mind.

– Threads support is far from brilliant on many platforms.

• Not all O/S are that good at non-blocking I/O

Portability and XS modules

• For the module writer, this is a question of writing portable C code, not portable perl.

• XS code can be a pain for the installer as it needs a C compiler.

• Not only does it need a C compiler; it needs the same one (and the same options) that the perl you are running was built with.

Choices for the perl admin

• Download and install a pre-built binary

• Roll your own

Choices for the perl admin

• Download and install a pre-built binary• Roll your own

• Rolling your own is usually better, but you always need a working ANSI C compiler

• Even with pre-built binaries, you need a C compiler to extend perl (i.e. add XS modules) - the same one perl was built with.

Choices for Win32

• Download ActiveState perl– Module binaries available through ppm

– MVC can be used to build CPAN modules

• Roll your own with MVC and DOS– Good luck!

– But it's do-able. Use nmake to make

• Roll your own with Cygwin and MingW– You might need to spend time getting the C compiler to

work first

Choices for most Unix O/S

• O/S may come with perl pre-installed– But it's probably a really old version

• Roll your own, to get the perl build you want– You may need to build and install gcc first– Or pay for a licensed ANSI C compiler

C compilers on Unix

• HP-UX: /bin/cc not suitable for much apart from building kernels. gcc works fine.

• Solaris: native cc is fine. Pre-installed perls were built with gcc though.

C compiler for VMS

• Buy a licence for DECC - tough.

• Get a test drive account and do all your building there, for free :)

• gcc not recommended on this platform according to the README.VMS

Reporting portability problems in other people's code

• Core modules (are meant to work everywhere)– Port problems in the core are bugs– Report to p5p via perlbug– Discuss on platform mailing lists e.g. vmsperl

• Other CPAN modules– Consider whether the module was intended to be

portable– Check reports from the CPAN testers– Use RT (rt.cpan.org)– Or mail the module author