I’m not sure whether I’ve just reinvented the wheel, but … a new way of building command-line interface in Python!

Well, it probably won’t work for everything. Where it works at it’s best is when you have a number of modules, which meant to do something, and you don’t want to build the CLI interface to each of them. Say, in my case — I have 5 modules for the running various machine-learning tasks, which act as wrappers around various Java applications (written by me and third-party). Each module has a number of functions, which have something in common — they take input file(s), process it and write output to another file.

Now, let’s get to the code.

tagger.py - performs feature extraction and invokation of Mallet tagger

Actual code is not included — just methods’ names.

import os
import sysfrom common import *
def extractFeaturesForTagger(filename, output, jrunner = None):
...def invokeTagger(featuresFile, modelFile, train = True, output = None, jrunner = None):
...
def despatch(args):
if len(args) < 3:
return False    cmd = args[2]

if cmd == 'fe':
try:
(ifile, ofile) = args[3:]
except:
return False        extractFeaturesForTagger(ifile, ofile)
elif cmd == 'train':
try:
(ifile, ofile) = args[3:]
except:
return False
        invokeTagger(ifile, ofile)
elif cmd == 'tag':
try:
(fts, model, ofile) = args[3:]
except:
return False        invokeTagger(fts, model, False, ofile)
else:
print("Couldn't recognise the options: ")
print(args)
return False
    return True
def describe():
return ("tagger", [
("fe", "[I]uafFile, [O]featureVectorsFile", "Extract features for tagger"),
("train", "[I]featureVectorsFile, [O]modelFile", "Train the model"),
("tag", "[I]featureVectorsFile, [I]modelFile, [O]taggerOutputFile", "Tag locations in file")
])

What matters is two functions - despatch and describe. First one actually takes standard sys.argv, assumes that it applies to current module and tries to apply the arguments to it’s functions. When it works, it invokes an appropriate method. When it’s not, it returns False.

Second method returns a tuple of string and list, which, in it’s turn consists of tuples of three elements — subcommand name, parameters and short description. Now when we have this module, let’s get to the despatcher.py:

#!/usr/bin/env python
# encoding: utf-8
"""
despatcher.pyCreated by Roman Kirillov on 2009-09-25.
Copyright (c) 2009 Yahoo! Inc. All rights reserved.
"""
import sys
import osimport tagger
import disambiguator
modules = [tagger,disambiguator]def printHelp():
    helpString = '\nUsage options: %s <module> <command> <command-specific-arguments>\n' % (sys.argv[0])
for m in modules:
(cmd, subcmds) = m.describe()
        helpString = helpString + '\n' + cmd + '\n'        for scd in subcmds:
(s, params, descr) = scd
            helpString = helpString + (s + "\t").rjust(15) #  + params + " -- " + descr + "\n"
            helpString = helpString  + params + " -- " + descr + "\n"
    print(helpString)def canIHaz(val):
if not val:
        printHelp()
        sys.exit(3)
def despatch():
if len(sys.argv) < 2 or sys.argv[1] == 'help':
        printHelp()
        sys.exit(0)
else:
for m in modules:
(cmd, subcmds) = m.describe()
if cmd == sys.argv[1]:
                canIHaz(m.despatch(sys.argv))
if __name__ == '__main__':
    despatch()

While it’s pretty easy to anyone who can read Python to understand, what’s this all about, let’s just show what happens when you run despatcher.py without parameters:

[kirillov@chiark integration]$ python despatcher.pyUsage options: despatcher.py <module> <command> <command-specific-arguments>
tagger
            fe  [I]uafFile, [O]featureVectorsFile -- Extract features for tagger
         train  [I]featureVectorsFile, [O]modelFile -- Train the model
           tag  [I]featureVectorsFile, [I]modelFile, [O]taggerOutputFile -- Tag locations in filedisambiguator
            fe  [I]uafFile, [O]outputFileName, [I]useTaggedLocations (true/false) -- Invoke disambiguation feature extractor
        import  [I]featureVector, [O]malletImport -- Import feature vector to Mallet format
         train  [I]featureVector, [O]modelName -- Train disambiguators model
      disambig  [I]featureVector, [I]modelName, [O]output -- Run trained disambiguator over the feature vector
     normalise  [I]featureVector, [O]normalisedFeatureVector -- Normalises the feature vector

Pretty easy, huh? Of course, still needs a lot of tuning and polishing, but it feels so much better to have a centralised dispatcher, which takes care of printing help and handling options — rather than having a one for each module?

Posted via web from Roman’s blog | Comment »