Chapter 11. Configuring Perl Programs

A bit more sophisticated although still not good, I can put that same configuration in a separate file and pull it into the main program. In config.pl, I have the code I previously had at the top of my program. I can’t use lexical variables because those are scoped to their file. Nothing outside config.pl can see them, which isn’t what I want for a configuration file. These have to be package (global) variables:

# config.pl
use vars qw( $Debug $Verbose $Email $DB );

$Debug   = 0;
$Verbose = 1;
$Email   = 'alice@example.com';
$DB      = 'DBI:mysql';

1;

I pull in the configuration information with require, but I have to do it inside a BEGIN block so Perl sees the use vars declaration before it compiles the rest of my program. We covered this in more detail in Intermediate Perl, Chapter 3, when we started to talk about modules:

#!/usr/bin/perl
use strict;
use warnings;

BEGIN { require "config.pl"; }

Or, I can use our, but I think that confuses the non-techie crowd even more:

# config-our.pl

our $Debug   = 0;
our $Verbose = 1;
our $Email   = 'alice@example.com';
our $DB      = 'DBI:mysql';

1;

Of course, I don’t have to go through these shenanigans if I don’t mind getting rid of use strict, but I don’t want to do that. That doesn’t stop other people from doing that though, and I find millions of references to config.pl. in http://code.ohloh.net.

Environment variables set values that every process within a session can access and use. Subprocesses can see these same values, but they can’t change them for other processes above them. Most shells set some environment variables automatically, such as HOME for my home directory and PWD for the directory I’m working in.

In Perl, the environment variables show up as keys in the %ENV hash. On most machines I write a testenv program to see how things are set up:

#!/usr/bin/perl

print "Content-type: text/plain\n\n" if $ENV{REQUEST_METHOD};

foreach my $key ( sort keys %ENV ) {
    printf "%-20s %s\n", $key, $ENV{$key};
    }

Notice the line that uses $ENV{REQUEST_METHOD}. If I use my program as a CGI program, the web server sets several environment variables including one called REQUEST_METHOD. If my program sees that it’s a CGI program, it prints a CGI response header. Otherwise, it figures I must be at a terminal and skips that part.

I particularly like using environment variables in CGI programs because I can set the environment in an .htaccess file. This example is Apache-specific and requires http://httpd.apache.org/docs/2.2/mod/mod_env.html, but other servers may have similar facilities:

# Apache .htaccess
SetEnv DB_NAME mousedb
SetEnv DB_USER buster
SetEnv DB_PASS pitrpat

Any variables that I set in .htaccess show up in my program and are available to all programs affected by that file. If I change the password, I only have to change it in one place. Beware, though, since the web server user can read this file, other users may be able to get this information. Almost any way you slice it, though, eventually the web server has to know these values, so I can’t keep them hidden forever.

Perl uses several environment variables to do its work. The PERL5OPT environment variable simulates me using those switches on the command line, and the PERL5LIB environment variable adds directories to the module search path. That way, I can change how Perl acts without changing the program.

To add more options just as if I had specified them on the command line of the shebang line, I add them to PERL5OPT. This can be especially handy if I always want to run with warnings, for instance:

% export PERL5OPT=w

The PERL5LIB value stands in place of the use lib directives in the code. I often have to use this when I want to run the same programs on different computers. As much as I’d like all of the world to have the same filesystem layout and to store modules, home directories, and other files in the same place, I haven’t had much luck convincing anyone to do it. Instead of editing the program to change the path to the local modules, I set it externally. Once set in a login program or Makefile, it’s there and I don’t have to think about it. I don’t have to edit all of my programs to have them find my new Perl library directory:

% export PERL5LIB=/Users/brian/lib/perl5

While developing, I usually add a lot of extra print statements so I can inspect the state of the program as I’m tracking down some bug. As I get closer to a working program, I leave these statements in there, but I don’t need them to execute every time I run the program; I just want them to run when I have a problem.

Similarly, in some instances I want my programs to show me normal output as it goes about its work when I’m at the terminal but be quiet when run from cron, a shell program, and so on.

In either case, I could define an environment variable to switch on, or switch off, the behavior. With an environment variable, I don’t have to edit the use of the program in other programs. My changes can last for as little as a single use by setting the environment variable when I run the program:

% DEBUG=1 ./program.pl

or for the rest of the session when I set the environment variable for the entire session:

% export DEBUG=1
% ./program.pl

Now I can use these variables to configure my program. Instead of coding the value directly in the program, I get it from the environment variables:

#!/usr/bin/perl
use strict;
use warnings;

my $Debug   = $ENV{DEBUG};
my $Verbose = $ENV{VERBOSE};

...

print "Starting processing\n" if $Verbose;

...

warn "Stopping program unexpectedly" if $Debug;

I can set environment variables directly on the command line and that variable applies only to that process. I can use my testenv program to verify the value. Sometimes I make odd shell mistakes with quoting and special character interpolation so testenv comes in handy when I need to figure out why the value isn’t what I think it is:

% DEBUG=1 testenv

I can also set environment variables for all processes in a session. Each shell has slightly different syntax for this:

% export DEBUG=2   # bash
% setenv DEBUG=2   # csh
<C: set DEBUG=2    # Windows>>

If I don’t set some of the environment variables I use in the program Perl complains about an uninitialized value since I have warnings on. When I try to check the values in the if statement modifiers in the last program, I get those warnings because I’m using undefined values. To get around that, I set some defaults. The || short circuit operator is handy here:

my $Debug   = $ENV{DEBUG}   || 0;
my $Verbose = $ENV{VERBOSE} || 1;

Sometimes 0 is a valid value even though it’s false so I don’t want to continue with the short circuit if the value is defined. In these cases, the conditional operator along with defined comes in handy:

my $Debug   = defined $ENV{DEBUG} ? $ENV{DEBUG} : 0;
my $Verbose = defined $ENV{VERBOSE} ? $ENV{VERBOSE} : 1;

Perl 5.10 and later has the defined-or (//) operator. It evaluates that argument on its left and returns it if it is defined, even if it is false. Otherwise, it continues onto the next value:

my $Verbose = $ENV{VERBOSE} // 1;  # new in Perl 5.10

Some values may even affect others. I might want a true value for $DEBUG to imply a true value for $VERBOSE, which would otherwise be false:

my $Debug   = $ENV{DEBUG}   // 0;
my $Verbose = $ENV{VERBOSE} // $ENV{DEBUG} //s 0;

Before I consider heavy reliance on environment variables, I should consider my target audience and which platform it uses. If those platforms don’t support environment variables, I should come up with an alternative way to configure my program.

I don’t need a module to process switches. Perl’s -s switch can do it as long as I don’t get too fancy. With this Perl switch, Perl turns the program switches into package variables. It can handle either single hyphen or double hyphens (which is just a single hyphen with a name starting with a hyphen). The switches can have values, or not. I can specify -s either on the command line or on the shebang line:

#!/usr/bin/perl -sw
# perl-s-abc.pl
use strict;

use vars qw( $a $abc );

print "The value of the -a switch is [$a]\n";
print "The value of the -abc switch is [$abc]\n";

Without values, Perl sets to 1 the variable for that switch. With a value that I attach to the switch name with an equal sign (and that’s the only way in this case), Perl sets the variable to that value:

% perl -s ./perl-s-abc.pl -abc=fred -a
The value of the -a switch is [1]
The value of the -abc switch is [fred]

I can use double hyphens for switches that -s will process:

% perl -s ./perl-s-debug.pl --debug=11

This causes Perl to create an illegal variable named ${'-debug'} even though that’s not strict safe. This uses a symbolic reference to get around Perl’s variable naming rules so I have to put the variable name as a string in curly braces. This also gets around the normal strict rules for declaring variables so I have to turn off the 'refs' check from strict to use the variables:

#!/usr/bin/perl -s
# perl-s-debug.pl
use strict;

{
no strict 'refs';
print "The value of the --debug switch is [${'-debug'}]\n";
print "The value of the --help switch is [${'-help'}]\n";
}

The previous command line produces this output:

The value of the --debug switch is [11]
The value of the --help switch is []

I don’t really need the double dashes. The -s switch doesn’t cluster switches so I don’t need the double dash to denote the long switch name. Creating variable names that start with an illegal character is a convenient way to segregate all of the configuration data, however, I still don’t endorse that practice.

I can’t go over all of the modules I might use or that I mentioned earlier, so I’ll stick to the two that come with Perl, Getopt::Std and Getopt::Long (both available since the beginning of Perl 5). You might want to consider if you really need more than these modules can handle. You’re pretty sure to have these available with the standard Perl distribution, and they don’t handle odd formats that could confuse your users.

#!/usr/bin/perl
# getopt-std.pl
use strict;

use Getopt::Std;

getopt('dog', \ my %opts );

print <<"HERE";
The value of
    d   $opts{d}
    o   $opts{o}
    g   $opts{g}
HERE

% perl getopt-std.pl -d 1
The value of
        d       1
        o
        g

% perl getopt-std.pl -d
The value of
        d
        o
        g

#!/usr/bin/perl
# getopts-std.pl

use Getopt::Std;

getopts('dog:', \ my %opts );

print <<"HERE";
The value of
    d   $opts{d}
    o   $opts{o}
    g   $opts{g}
HERE

% perl getopts-std.pl -g foo -d
The value of
    d       1
    o
    g       foo

% ./getopts.pl -g -d -o
The value of
    d
    o
    g       -d

% perl getopts-std.pl  -d foo -g bar -o
The value of
    d       1
    o
    g

#!/usr/bin/perl
# getoptions-v.pl

use Getopt::Long;

my $result = GetOptions(
    'debug|d'   => \ my $debug,
    'verbose|v' => \ my $verbose,
    );

print <<"HERE";
The value of
    debug       $debug
    verbose     $verbose
HERE

% perl getoptions-v.pl -verbose
The value of
    debug
    verbose         1

% perl getoptions-v.pl -v
The value of
    debug
    verbose         1

% perl getoptions-v.pl -v -d
The value of
    debug           1
    verbose         1

% perl getoptions-v.pl -v -debug
The value of
    debug           1
    verbose         1

% perl getoptions-v.pl -v --debug
The value of
    debug           1
    verbose         1

#!/usr/bin/perl
# getopt-long-args.pl

use Getopt::Long;

my $result = GetOptions(
    "file=s" => \ my $file,
    "line=i" => \ my $line,
    );

print <<"HERE";
The value of
    file        $file
    line        $line
HERE

% perl getopt-long-args.pl -line=-9
The value of
    file
    line            -9
% perl getopt-long-args.pl -line=9.9
Value "9.9" invalid for option line (number expected)
The value of
    file
    line

#!/usr/bin/perl
# getopt-long-mult.pl

use Getopt::Long;

my $result = GetOptions(
    "file=s@" => \ my @files,
    );


{
local $" = ", ";

print <<"HERE";
The value of
    file        @files
HERE
}

% perl getopt-long-mult.pl --file foo --file bar
The value of
    file            foo, bar

I’m a bit partial to ConfigReader::Simple because I maintain it (although I did not originally write it). It can handle multiple files (for instance, including a user configuration file that can override a global one) and has a simple line-oriented syntax:

# configreader-simple.txt
file=foo.dat
line=453
field value
field2 = value2
long_continued_field This is a long \
    line spanning two lines

The module handles all of those formats:

#!/usr/bin/perl
# configreader-simple.pl

use ConfigReader::Simple;

my $config = ConfigReader::Simple->new(
    "configreader-simple.txt" );
die "Could not read config! $ConfigReader::Simple::ERROR\n"
    unless ref $config;

print "The line number is ", $config->get( "line" ), "\n";

I reach for this module when the configuration keys are unrelated to one another.

Windows folks are used to INI files and there are modules to handle those too. The basic format breaks the configuration into groups with a heading inside square brackets. Parameters under the headings apply to that heading only, and the key and value have an equals sign between them (or in some formats, a colon). Comment lines start with a semi-colon. The INI format even has a line continuation feature. The Config::IniFiles module, as well as some others, can handle these. Here’s a little INI file I might use to work on this book:

[Debugging]
;ComplainNeedlessly=1
ShowPodErrors=1

[Network]
email=brian.d.foy@gmail.com

[Book]
title=Mastering Perl
publisher=O'Reilly Media
author=brian d foy

I can parse this file and get the values from the different sections:

#!/usr/bin/perl
# config-ini.pl

use Config::IniFiles;

my $file = "mastering_perl.ini";

my $ini = Config::IniFiles->new(
    -file    => $file
    ) or die "Could not open $file!";

my $email = $ini->val( 'Network', 'email' );
my $author = $ini->val( 'Book', 'author' );

print "Kindly send complaints to $author ($email)\n";

Besides just reading the file, I can use Config::IniFiles to change values, add or delete values, and rewrite the INI file.

Config::Scoped is similar to INI in that it can limit parameters to a certain section but it’s more sophisticated. It allows nested section, Perl code evaluation (remember what I said about that earlier, though), and multivalued keys:

book {
    author = {
        name="brian d foy";
        email="brian.d.foy@gmail.com";
        };
    title="Mastering Perl";
    publisher="O'Reilly Media";
}

The module parses the configuration and gives it back to me as a Perl data structure:

#!/usr/bin/perl
# config-scoped.pl

use Config::Scoped;

my $config = Config::Scoped->new( file => 'config-scoped.txt' )->parse;
die "Could not read config!\n" unless ref $config;

print "The author is ", $config->{book}{author}{name}, "\n";

There are many other configuration formats and each of them probably already has a Perl module to go with it. Win32::Registry gives me access to the Windows Registry, Mac::PropertyList deals with Mac OS X’s plist format, and Config::ApacheFile parses the Apache configuration format. Go through the list of Config:: modules on CPAN to find the one that you need. You can use Config::JSON for JSON or YAML (or one of several variants) for YAML. Those two formats have good support in most of the mainstream languages. See Chapter 14 for more about JSON and YAML.

I may need my program to do different things based on which platform I invoke it. On a unix platform, I may load one module, whereas on Windows I load another. Perl knows where it’s running and puts a distinctive string in $^O (mnemonic: O for Operating system), and I can use that string to decide what I need to do. Perl determines that value when it’s built and installed. The value of $^O is the same as $Config{'osname'}. If I need something more specific, I can use the $Config{archname}.

I have to be careful, though, to specify exactly which operating system I want. Table 1 shows the value of $^O for popular systems, and the perlport documentation lists several more. Notice that I can’t just look for the pattern m/win/i to check for Windows since Mac OS X identifies itself as darwin.

I can conditionally load modules based on the operating system. For instance, the File::Spec module comes with Perl and is really a façade for several operating system specific modules behind the scenes. Here’s the entire code for the module. It defines the %module hash to map the values of $^O to the module it should load. It then requires the right module. Since each submodule has the same interface, the programmer is none the wiser:

package File::Spec;

use strict;
use vars qw(@ISA $VERSION);

$VERSION = '0.87';

my %module = (MacOS   => 'Mac',
              MSWin32 => 'Win32',
              os2     => 'OS2',
              VMS     => 'VMS',
              epoc    => 'Epoc',
              NetWare => 'Win32', # Yes, File::Spec::Win32 works on NetWare.
              dos     => 'OS2',   # Yes, File::Spec::OS2 works on DJGPP.
              cygwin  => 'Cygwin');


my $module = $module{$^O} || 'Unix';

require "File/Spec/$module.pm";
@ISA = ("File::Spec::$module");

1;