sumolib package

Submodules

sumolib.Builds module

Database file handling.

class sumolib.Builds.BuildCache(dict_=None, lock_timeout=None)[source]

Bases: sumolib.JSON.Container

Detailed dependency information.

Taken from sumo-scan and from the build database. Datastructure:

{ "modulename": { "versionname": { "depmodule" :
                                   {
                                     "depvers1": state,
                                     "depvers2": state
                                   }
                                 }
                }
}
add_dependency(modulename, versionname, dep_name, dep_version, state)[source]

add a single dependency with a state.

relation(modulename, versionname, dep_name, dep_version)[source]

return the relation between two modules.

None: unrelated <state>: built together in a build with state <state>.

update_from_builddb(builddb, db)[source]

update data from a builddb.

was_built(modulename, versionname)[source]

return True when the module was built sometime.

class sumolib.Builds.DB(dict_=None, lock_timeout=None)[source]

Bases: sumolib.JSON.Container

the buildtree database.

add_build(other, build_tag)[source]

add build data from another Builddb to this one.

Note: this does NOT do a deep copy, it copies just references.

add_module(build_tag, module_build_tag, modulename, versionname)[source]

add a module definition.

change_state(build_tag, new_state)[source]

sets the state to a new value.

classmethod check_state(state, new_build)[source]

checks if a state is allowed.

delete(build_tag)[source]

delete a build.

filter_by_modulespecs(modulespecs)[source]

return a new Builddb that satisfies the given list of specs.

Note that this function treats versions like “R1-3” and “1-3” to be different.

generate_buildtag(buildtag_stem)[source]

generate a new buildtag.

A new buildtag in the form “STEM-nnn” is generated.

has_build_tag(build_tag)[source]

returns if build_tag is contained.

has_module(build_tag, modulename)[source]

returns if the module is contained here.

is_disabled(build_tag)[source]

returns True if the build is marked disabled.

is_empty()[source]

shows of the object is empty.

is_fully_linked(build_tag)[source]

returns True if the build consists only of links.

static is_generated_buildtag(buildtag)[source]

return True of the buildtag was generated.

is_incomplete(build_tag)[source]

returns True if the build is marked incomplete.

is_linked_to(build_tag, other_build_tag)[source]

returns True if there are links to other_build_tag.

is_stable(build_tag)[source]

returns True if the build is marked stable.

is_testing_or_stable(build_tag)[source]

returns True if the build is marked testing or stable.

is_unstable(build_tag)[source]

returns True if the build is marked testing or stable.

iter_builds()[source]

return a build iterator.

iter_modules(build_tag)[source]

return an iterator on the modules.

linked_builds(build_tag)[source]

return a set of tags of all builds that depend on this.

merge(other)[source]

merge with another builddb.

Returns the list of new added builds.

return linked build_tag if the module is linked or None.

module_specs(build_tag)[source]

return the modules of a build in form module spec strings.

This function returns a list of strings that ccan be parsed by sumolib.ModuleSpec.Spec.from_string().

module_version(build_tag, modulename)[source]

returns the version of the module or None.

modules(build_tag)[source]

return all modules of a build.

The returned structure is a dictionary mapping modulenames to versionnames.

new_build(build_tag, state)[source]

create a new build with the given state.

rec_linked_builds(build_tag)[source]

return a set of tags of all builds that recursively depend on this.

selfcheck(msg)[source]

raise exception if obj doesn’t look like a builddb.

This does not do a complete check of the datastructure, it only ensures that the JSON datastructure isn’t something completely different.

state(build_tag)[source]

return the state of the build.

states = set(['disabled', 'testing', 'unstable', 'incomplete', 'stable'])
class sumolib.Builds.DB_overlay(dict_=None, lock_timeout=None)[source]

Bases: sumolib.Builds.DB

Implement a builddb with overlays.

Overlays are other build databases that are added to the local build database but cannot be modified and whose build specifications are not saved when the object is saved.

change_state(build_tag, new_state)[source]

sets the state to a new value.

delete(build_tag)[source]

delete a build.

dirname_from_tag(build_tag)[source]

return the name of the builddb directory from a tag.

filename_from_tag(build_tag)[source]

return the name of the builddb file from a tag.

overlay(filename)[source]

merge with another builddb from a file.

overlaymode(new_mode=None)[source]

get or set the overlay mode.

overlay_mode True:
All JSON representations of the object do not contain builds that were added with overlay(). Builds added with overlay() cannot be modified.
overlay_mode False:
All JSON representations of the object contain all the builds. Builds added with overlay() can be modified.
tag_is_overlayed(build_tag)[source]

return True if build_tag is from overlayed builddb.

to_dict()[source]

return the object as a dict.

By overriding this method, we change all JSON representations of the object.

sumolib.Builds.warn(text)[source]

print a warning to the console.

sumolib.Config module

Configuration file support.

class sumolib.Config.ConfigFile(filename, env_name, dict_)[source]

Bases: object

store options in a JSON file.

classmethod from_optionlist(filename, env_name, optionlist)[source]

Create object from optionlist.

get(optionname)[source]

get an option.

load(filenames, enable_loading)[source]

load from all files in filenames list.

enable_loading - If True, commands like “#preload” are enabled,
otherwise these keys are just treated like ordinary values.
merge_options(option_obj, merge_opts_set)[source]

create from an option object.

Merge Config object with command line options and command line options with Config object.

All options that are part of the set <merge_opts_set> must be lists. For these options the lists are concatenated.

optionlist()[source]

return all known options of the Config object.

static paths_from_env(varname)[source]

read configuration paths from environment variable.

real_paths()[source]

return the list of files that should be loaded or were loaded.

save(filename, keys, verbose, dry_run)[source]

dump in json format

set(optionname, value)[source]

set an option to an arbitrary value.

sumolib.Dependencies module

Database file handling.

class sumolib.Dependencies.DB(dict_=None, lock_timeout=None)[source]

Bases: sumolib.JSON.Container

the dependency database.

add_alias(modulename, versionname, alias_name, real_name)[source]

add an alias for modulename:versionname.

add_dependency(modulename, versionname, dep_modulename)[source]

add dependency for a module:version.

assert_complete_modulelist(moduledict)[source]

test if a set of modules is complete.

This means that all dependencies are part of the given modules.

moduledict is a dictionary mapping modulename–>versionname.

assert_module(modulename, versionname)[source]

do nothing if the module is found, raise KeyError otherwise.

If versionname is None, just check that the modulename is known.

check()[source]

do a consistency check on the db.

clonemodule(old_modulename, modulename, versions)[source]

Take all versions of old_modulename to create modulename.

complete_sets_dict(sets_dict)[source]

makes a sets_dict complete with respect to dependencies.

A sets dict has this form:

convert modulespecs to a sets dict:

{ modulename1 : set(version1,version2),
  modulename2 : set(version1,version2),
}

For each dependency that is missing, this program creates a new entry in the sets dict which contains all possible versions for the missing module.

Returns a set of modulenames of added dependencies.

del_dependency(modulename, versionname, dep_modulename)[source]

delete dependency for a module:version if it exists.

dependencies_found(modulename, versionname)[source]

returns True if dependencies are found for modulename:versionname.

depends_on_module(modulename, versionname, dependencyname)[source]

returns True if given dependency is found.

get_alias(modulename, versionname, dep_modulename)[source]

return the alias or the original name for dep_modulename.

import_module(other, module_name, versionname)[source]

copy the module data from another Dependencies object.

This does a deepcopy of the data.

iter_dependencies(modulename, versionname)[source]

return an iterator on dependency modulenames of a module.

iter_modulenames()[source]

return an iterator on module names.

iter_versions(modulename)[source]

return an iterator on versionnames of a module.

merge(other)[source]

merge another Dependencies object to self.

parameters:
self - the object itself other - the other Dependencies object
module_source_dict(modulename, versionname)[source]

return a dict {type:dict} for the module source.

module_source_pair(modulename, versionname)[source]

return a tuple (type,dict) for the module source.

module_source_string(modulename, versionname)[source]

return the source url or tar-file or path for a module.

partial_copy_by_list(list_)[source]

take items from the Dependencies object and create a new one.

List must be a list of tuples in the form (modulename,versionname).

This function copies modules whose versionname match exactly the given name, so “R1-3” and “1-3” are treated to be different.

Note that the new Dependencies object only contains references of the data. This DOES NOT do a deep copy.

partial_copy_by_modulespecs(modulespecs)[source]

take items from the Dependencies object and create a new one.

modulespecs must be a sumolib.ModuleSpec.Specs object.

Note that this function treats versions like “R1-3” and “1-3” to be different.

If no versions are defined for a module, take all versions.

When no moduleversions are found, rause a ValueError exception.

Note that the new Dependencies object only contains references of the data. This DOES NOT do a deep copy so you should NOT modify the result.

patch_version(modulename, versionname, newversionname, do_replace)[source]

add a new version to the database by copying the old one.

do_replace: if True, replace the old version with the new one

remove_missing_deps()[source]

remove dependencies that are not part of the database.

search_modules(rx_object)[source]

search module names and source URLS for a regexp.

Returns a list of tuples (modulename, versionname).

selfcheck(msg)[source]

raise exception if obj doesn’t look like a dependency database.

set_source_(module_name, versionname, sourcespec)[source]

add a module with source spec.

set_source_spec(module_name, versionname, sourcespec)[source]

set sourcespec of a module, creates if it does not yet exist.

returns True if the spec was changed, False if it wasn’t.

set_source_spec_by_tag(module_name, versionname, tag)[source]

try to change sourcespec by providing a tag.

returns True if the spec was changed, False if it wasn’t.

sets_dict(modulespecs)[source]

create a dict of sets according to modulespecs.

modulespecs must be a sumolib.ModuleSpec.Specs object.

convert modulespecs to a sets dict:

{ modulename1 : set(version1,version2),
  modulename2 : set(version1,version2),
}
sortby_dependency(moduleversions, reverse=False)[source]

sorts modules by dependencies.

Order that dependent modules come after the modules they depend on.

moduleversions: a list of pairs (modulename, versionname).

sortby_weight(moduleversions, reverse=False)[source]

sorts modules by weight.

Order in a way that smaller weights come first.

moduleversions: a list of pairs (modulename, versionname).

sorted_moduleversions(modulename)[source]

return an iterator on sorted versionnames of a module.

weight(modulename, versionname, new_weight=None)[source]

set the weight factor.

class sumolib.Dependencies.OldDB(dict_=None, lock_timeout=None)[source]

Bases: sumolib.JSON.Container

convert the old dependency database to the new format.

returns a BuildCache and a Dependency object.

convert()[source]

convert to a Dependencies and BuildCache object.

sumolib.JSON module

JSON utilities.

class sumolib.JSON.Container(dict_=None, lock_timeout=None)[source]

Bases: object

an object that is a python structure.

This is a dict that contains other dicts or lists or strings or floats or integers.

datadict()[source]

return the internal dict.

dirname()[source]

return the directory part of the internal filename.

filename(new_name=None)[source]

return or set the internal filename.

classmethod from_json(json_data)[source]

create an object from a json string.

classmethod from_json_file(filename, keep_lock, lock_timeout=None)[source]

create an object from a json file.

classmethod from_pickle_file(filename)[source]

load from a cPickle file, don’t use lockfiles or anything.

json_print()[source]

print a JSON representation of the object.

json_save(filename, verbose, dry_run)[source]

save as a JSON file.

If filename is empty, use the default filename.

Always remove a file lock if it existed before.

json_string()[source]

return a JSON representation of the object.

lock_file()[source]

lock a file and store filename and lock in the object.

pickle_save(filename)[source]

save using cPickle, don’t use lockfiles or anything.

selfcheck(msg)[source]

raise an exception if the object is not valid.

to_dict()[source]

return the object as a dict.

unlock_file()[source]

remove a filelock if there is one.

exception sumolib.JSON.InconsistentError[source]

Bases: exceptions.Exception

This is raised when we cannot get consistent JSON data.

exception sumolib.JSON.ParseError[source]

Bases: exceptions.Exception

This is raised when the JSON data is invalid.

sumolib.JSON.dump(var)[source]

Dump a variable in JSON format.

Here is an example:

>>> var= {"key":[1,2,3], "key2":"val", "key3":{"A":1,"B":2}}
>>> dump(var)
{
    "key": [
        1,
        2,
        3
    ],
    "key2": "val",
    "key3": {
        "A": 1,
        "B": 2
    }
}
sumolib.JSON.dump_file(filename, var)[source]

Dump a variable to a file in JSON format.

This function uses a technique to write the file atomically. It assumes that we have a lock on the file so the temporary filename does not yet exist.

sumolib.JSON.json_load(data)[source]

decode a JSON string.

sumolib.JSON.json_str(var)[source]

convert a variable to JSON format.

Here is an example:

>>> var= {"key":[1,2,3], "key2":"val", "key3":{"A":1,"B":2}}
>>> print json_str(var)
{
    "key": [
        1,
        2,
        3
    ],
    "key2": "val",
    "key3": {
        "A": 1,
        "B": 2
    }
}
sumolib.JSON.loadfile(filename)[source]

load a JSON file.

If filename is “-” read from stdin.

sumolib.ModuleSpec module

module specifications.

class sumolib.ModuleSpec.Spec(modulename, versionname, versionflag)[source]

Bases: object

a class representing a single module specification.

assert_exact()[source]

raise ValueError exception if spec is not exact.

An exact module specification is a specification where for a module there is exactly one version given.

static compare_versions(version1, version2, flag)[source]

Test if a version matches another version.

classmethod from_string(spec)[source]

create modulespec from a string.

A module specification has one of these forms:
modulename modulename:version
version may be:
versionname : exactly this version +versionname : this version or newer -versionname : this version or older

Here are some examples:

>>> Spec.from_string("ALARM")
Spec('ALARM',None,None)
>>> Spec.from_string("ALARM:R3-2")
Spec('ALARM','R3-2','eq')
>>> Spec.from_string("ALARM:+R3-2")
Spec('ALARM','R3-2','ge')
>>> Spec.from_string("ALARM:-R3-2")
Spec('ALARM','R3-2','le')
is_exact_spec()[source]

return if the spec is an exact version specification.

no_version_spec()[source]

returns True if there is no version spec.

test(version)[source]

Test if a version matches the spec.

Here are some examples: >>> m= Spec.from_string(“ALARM:R3-2”) >>> m.test(“R3-1”) False >>> m.test(“R3-2”) True >>> m.test(“R3-3”) False

>>> m= Spec.from_string("ALARM:-R3-2")
>>> m.test("R3-1")
True
>>> m.test("R3-2")
True
>>> m.test("R3-3")
False
>>> m= Spec.from_string("ALARM:+R3-2")
>>> m.test("R3-1")
False
>>> m.test("R3-2")
True
>>> m.test("R3-3")
True
to_string()[source]

return a spec string.

Here are some examples:

>>> Spec("ALARM","R3-2","eq").to_string()
'ALARM:R3-2'
>>> Spec("ALARM","R3-2","ge").to_string()
'ALARM:+R3-2'
>>> Spec("ALARM","R3-2","le").to_string()
'ALARM:-R3-2'
>>> Spec("ALARM",None,None).to_string()
'ALARM'
class sumolib.ModuleSpec.Specs(speclist)[source]

Bases: object

A class representing a list of Spec objects.

assert_exact()[source]

raise ValueError exception if not all spec are exact.

An exact module specification is a specification where for each module there is one version given.

assert_unique()[source]

raise ValueError exception if a module is found more than once.

This ensures that the module specifications have only one specification for each modulename.

classmethod from_strings(specs, builddb_fn)[source]

scan a list of module specification strings.

specs:
list of module specification strings
builddb_fn:
a function that for builddb_fn(buildtag) returns builddb.module_specs(buildtag), only needed for :build:buildtag.

returns a new Specs object.

Note that if a modulename is used twice, the later definition overwrites the first one. However, the module retains it’s position in the internal list of modules.

Here are some examples:

>>> def p(s):
...     for m in s:
...         print m
>>> p(Specs.from_strings(["A:R2","B:-R3","C:+R1"], None))
Spec('A','R2','eq')
Spec('B','R3','le')
Spec('C','R1','ge')
>>> p(Specs.from_strings(["A:R2","B:-R3","A:R3"], None))
Spec('A','R3','eq')
Spec('B','R3','le')
>>> p(Specs.from_strings(["A:R2","B:-R3",":rm:A"], None))
Spec('B','R3','le')
>>> p(Specs.from_strings(["A:R2","B:-R3",":rm:A","A:R3"], None))
Spec('A','R3','eq')
Spec('B','R3','le')
static scan_special(st)[source]

scan special in-line commands.

to_dist_dict()[source]

convert to a dict mapping modulename–>versionname.

May raise ValueError exception if the specs are not all exact and unique.

to_stringlist()[source]

convert back to a list of strings.

sumolib.cli module

Command line interface.

class sumolib.cli.Arguments(initial_dict=None)[source]

Bases: sumolib.cli.Container

class for arguments and commands.

exception sumolib.cli.CliError[source]

Bases: exceptions.Exception

Error from command line parsing.

class sumolib.cli.CmdSpec(name, completion=None, array=False, optional=False)[source]

Bases: object

a single command specification.

class sumolib.cli.CmdSpecs[source]

Bases: object

hold command specfications.

add(name, completion=None, array=False, optional=False)[source]

add a single specification.

get()[source]

get items.

more()[source]

return if there are more items.

start_get()[source]

start getting items.

class sumolib.cli.Container(initial_dict=None)[source]

Bases: object

hold all option values.

declare(name, is_array=False)[source]

declare a value as empty.

defined(name)[source]

return if the attribute is set and not empty.

defined_items()[source]

return all items that are not empty.

get(name, default=None)[source]

get a value, return default if it doesn’t exist.

static normalize_name(st)[source]

remove dashes from a name.

put(name, value, is_array=False)[source]

put a value.

class sumolib.cli.OptionSpec(name_str, completion=None, arg_name=None, array=False, arg_is_option=False, value_list=None)[source]

Bases: object

Spec for a command line option.

check(value)[source]

simple check of the options value.

class sumolib.cli.OptionSpecs(specs=None)[source]

Bases: object

hold command specfications.

add(name_str, completion=None, arg_name=None, array=False, arg_is_option=False, value_list=None)[source]

add a single specification.

add_spec(spec)[source]

add a single specification.

completion_options(attribute, word_option, new_option)[source]

define the special completion options.

word_option:
complete the last argument further
new_option :
complete a new argument
attribute :
the name of the attribute in “Options” that is set when completion is requested. The flag will have the value “word” or “new” or <None>.
get(name)[source]

get OptionSpec object for a name.

match(name)[source]

return a list of matching optionnames.

class sumolib.cli.Options(initial_dict=None)[source]

Bases: sumolib.cli.Container

class for options.

sumolib.cli.assert_options(do_exit, options, *opt_list)[source]

check for the presence of options.

sumolib.cli.complete_dir(pattern, dummy)[source]

return all files for a pattern.

sumolib.cli.complete_file(pattern, dummy)[source]

return all files for a pattern for command completion

sumolib.cli.complete_list(list_, element, dummy)[source]

complete for a given list.

sumolib.cli.is_opt(st)[source]

Test if a string “looks” like an option.

Here are some examples:

>>> is_opt("")
False
>>> is_opt("x")
False
>>> is_opt("-")
False
>>> is_opt("-$")
False
>>> is_opt("-x")
True
>>> is_opt("-xx")
False
>>> is_opt("--x")
True
>>> is_opt("--$")
False
>>> is_opt("--")
False
>>> is_opt("--x-s")
True
sumolib.cli.process_args(cli_args, argspec, completion_mode, testmode=False)[source]

new command argument processing.

argspec must be None or an CmdSpecs object.

Note: This function contains sys.exit statements !!

Here is some testcode:

>>> tspec= CmdSpecs()
>>> tspec.add("MOD", completion= lambda x,r: [x+"a"+"-mod",x+"b"+"-mod"])
>>> tspec.add("VER", completion= lambda x,r: [x+"1"+"-ver",x+"2"+"-ver"],
...           optional= True)
>>> tspec.add("ARG", completion= lambda x,r: [x+"a"+"-arg",x+"b"+"-arg"],
...           optional= True, array= True)
>>> import pprint
>>> def t(cli_args, completion_mode):
...     r=process_args(cli_args, tspec, completion_mode, True)
...     if r:
...         pprint.pprint(r)
>>> t([], None)
error, mandatory argument MOD missing
>>> t([], "word")
a-mod
b-mod
>>> t(["a"], "word")
aa-mod
ab-mod
>>> t(["mod"], None)
Arguments({'MOD': 'mod', 'VER': None, 'ARG': []})
>>> t(["mod"], "word")
moda-mod
modb-mod
>>> t(["mod"], "new")
1-ver
2-ver
>>> t(["mod","12"], None)
Arguments({'MOD': 'mod', 'VER': '12', 'ARG': []})
>>> t(["mod","12"], "word")
121-ver
122-ver
>>> t(["mod","12"], "new")
a-arg
b-arg
>>> t(["mod","12", "a"], None)
Arguments({'MOD': 'mod', 'VER': '12', 'ARG': ['a']})
>>> t(["mod","12", "a", "b"], None)
Arguments({'MOD': 'mod', 'VER': '12', 'ARG': ['a', 'b']})
>>> t(["mod","12", "a", "b"], "word")
ba-arg
bb-arg
>>> t(["mod","12", "a", "b"], "new")
a-arg
b-arg
sumolib.cli.process_cmd(cli_args, cmd_list, completion_mode, item_name='command', testmode=False)[source]

process a single command.

cmd_list: list of possible commands (cli_args[0])

returns a tuple (cmd, args)

sumolib.cli.process_opts(args, optionspecs, testmode=False)[source]

parse incomplete options.

expects args[0] to be the program’s name.

>>> spcs= OptionSpecs()
>>> spcs.completion_options("list","--list","--listnew")
>>> spcs.add("--help -h")
>>> spcs.add("--flag -F")
>>> spcs.add("-x", arg_name="VALUE")
>>> spcs.add("-a", arg_name="ELM", array= True)
>>> spcs.add("-y", lambda x,r: [x+"a",x+"b"], "YOPT")
>>> def t(args):
...     a= ["dummy"]
...     a.extend(args)
...     (options,rest)= process_opts(a, spcs, True)
...     if options is not None:
...         print options
...         print "rest:", repr(rest)
>>> t([])
Options
    a: None
    flag: None
    help: None
    list: None
    x: None
    y: None

rest: []
>>> t(["-h"])
Options
    a: None
    flag: None
    help: True
    list: None
    x: None
    y: None

rest: []
>>> t(["--help"])
Options
    a: None
    flag: None
    help: True
    list: None
    x: None
    y: None

rest: []
>>> t(["-F"])
Options
    a: None
    flag: True
    help: None
    list: None
    x: None
    y: None

rest: []
>>> t(["--flag"])
Options
    a: None
    flag: True
    help: None
    list: None
    x: None
    y: None

rest: []
>>> t(["-x"])
error, argument 'VALUE' missing for option '-x'
>>> t(["-x", "ab"])
Options
    a: None
    flag: None
    help: None
    list: None
    x: 'ab'
    y: None

rest: []
>>> t(["-a", "a"])
Options
    a: ['a']
    flag: None
    help: None
    list: None
    x: None
    y: None

rest: []
>>> t(["-a", "a", "-h", "-a", "b"])
Options
    a: ['a', 'b']
    flag: None
    help: True
    list: None
    x: None
    y: None

rest: []
>>> t(["-a", "a", "-F", "-a", "b"])
Options
    a: ['a', 'b']
    flag: True
    help: None
    list: None
    x: None
    y: None

rest: []
>>> t(["-F", "--", "-x"])
Options
    a: None
    flag: True
    help: None
    list: None
    x: None
    y: None

rest: ['-x']
>>> t(["-a", "--", "-x"])
Options
    a: ['-x']
    flag: None
    help: None
    list: None
    x: None
    y: None

rest: []
>>> t(["-a", "--", "-x", "a"])
Options
    a: ['-x']
    flag: None
    help: None
    list: None
    x: None
    y: None

rest: ['a']
>>> t(["-a", "--", "b"])
Options
    a: ['b']
    flag: None
    help: None
    list: None
    x: None
    y: None

rest: []
>>> t(["-a", "ab", "--", "-h"])
Options
    a: ['ab']
    flag: None
    help: True
    list: None
    x: None
    y: None

rest: []
>>> t(["-y","a"])
Options
    a: None
    flag: None
    help: None
    list: None
    x: None
    y: 'a'

rest: []
>>> t(["-y","a","--list"])
aa
ab
>>> t(["-y","a","--listnew"])
Options
    a: None
    flag: None
    help: None
    list: 'new'
    x: None
    y: 'a'

rest: []
>>> t(["-y","a","b","--listnew"])
Options
    a: None
    flag: None
    help: None
    list: 'new'
    x: None
    y: 'a'

rest: ['b']
>>> t(["-y","a","b","c","--listnew"])
Options
    a: None
    flag: None
    help: None
    list: 'new'
    x: None
    y: 'a'

rest: ['b', 'c']
>>> t(["-y","a","b","--list"])
Options
    a: None
    flag: None
    help: None
    list: 'word'
    x: None
    y: 'a'

rest: ['b']
>>> t(["--","--list"])
--flag
--help
>>> t(["-","--list"])
-F
-a
--flag
-h
--help
-y
-x
>>> t(["-a","--list"])
-a

sumolib.complete module

Special sumo command completion functions.

sumolib.complete.build_cache()[source]

create a module cache in $HOME/.sumo.modulecache if needed.

returns the cache data.

sumolib.complete.build_cache_callback()

callback dummy func.

sumolib.complete.builds(build, dummy)[source]

complete a build name.

sumolib.complete.clear_build_cache()[source]

remove the build cache.

sumolib.complete.clear_caches()[source]

clear all caches.

sumolib.complete.clear_db_cache()[source]

remove the db cache.

sumolib.complete.db_cache()[source]

create a module cache in $HOME/.sumo.modulecache if needed.

returns the cache data.

sumolib.complete.db_cache_callback()

callback dummy func.

sumolib.complete.dependency(module_par_name, dep, result)[source]

try to complete a dependency for a given module.

sumolib.complete.dummy()[source]

callback dummy func.

sumolib.complete.module(module_, dummy)[source]

complete a module name.

sumolib.complete.moduleversion(module_, dummy)[source]

try to give a sensible completion list to a module.

sumolib.complete.touch(fname, times=None)[source]

do “touch” on a file.

sumolib.complete.version(module_par_name, ver, result)[source]

complete a module version name.

sumolib.darcs module

darcs support

class sumolib.darcs.Repo(directory, hints, verbose, dry_run)[source]

Bases: object

represent a darcs repository.

static checkout(spec, destdir, verbose, dry_run)[source]

spec must be a dictionary with “url” and “tag” (optional).

commit(logmessage)[source]

commit changes.

get_tag_on_top()[source]

return the “tag on top” property.

name()[source]

return the repo type name.

pull_merge()[source]

pull changes and try to merge.

push()[source]

push all changes changes.

rx_darcs_repo = <_sre.SRE_Pattern object at 0x7fef98efb468>
classmethod scan_dir(directory, hints, verbose, dry_run)[source]

return a Repo object if a darcs repo was found.

This function returns <None> if no working repo was found.

If bool(hints[“write check”]) is True, return <None> if the repository directory is not writable.

For parameter “hints” see comment at __init__.

source_spec()[source]

return a complete source specification (for SourceSpec class).

sumolib.darcs.assert_darcs()[source]

ensure that darcs exists.

sumolib.fileurl module

urlsupport

sumolib.fileurl.assert_scp()[source]

test if scp exists.

sumolib.fileurl.get(url, dest, verbose, dry_run)[source]

Get by url.

sumolib.git module

git support

class sumolib.git.Repo(directory, hints, verbose, dry_run)[source]

Bases: object

represent a git repository.

static checkout(spec, destdir, verbose, dry_run)[source]

spec must be a dictionary with “url” and “tag” (optional).

commit(logmessage)[source]

commit changes.

get_revision()[source]

return the current revision.

get_tag_on_top()[source]

return the “tag on top” property.

name()[source]

return the repo type name.

pull_merge()[source]

pull changes and try to merge.

push()[source]

push all changes changes.

rx_repo = <_sre.SRE_Pattern object at 0x7fef993e29d0>
rx_tag = <_sre.SRE_Pattern object at 0x7fef993e2870>
classmethod scan_dir(directory, hints, verbose, dry_run)[source]

return a Repo object if a git repo was found.

This function returns <None> if no working repo was found.

If bool(hints[“write check”]) is True, return <None> if the repository directory is not writable.

For parameter “hints” see comment at __init__.

source_spec()[source]

return a complete source specification (for SourceSpec class).

sumolib.git.assert_git()[source]

ensure that git exists.

sumolib.lock module

Lockfile support.

exception sumolib.lock.AccessError[source]

Bases: exceptions.Exception

No rights to create a lock.

This is raised when we can’t create a lock due to access rights on the directory

exception sumolib.lock.LockedError[source]

Bases: exceptions.Exception

This is raised when we can’t get a lock.

class sumolib.lock.MyLock(filename, timeout=None)[source]

Bases: object

Implement a simple file locking mechanism.

lock()[source]

do the file locking.

raises:
LockedError : can’t get lock AccessError : no rights to create lock OSError : other operating system errors

On linux, create a symbolic link, otherwise a directory. The symbolic link has some information on the user, host and process ID.

On other systems the created directory contains a file whose name has some information on the user, host and process ID.

unlock(force=False)[source]

unlock.

sumolib.lock.current_user()[source]

return the current user in a hopefully portable way.

sumolib.lock.edit_with_lock(filename, verbose, dry_run)[source]

lock a file, edit it, then unlock the file.

sumolib.makefile_scan module

makefile_scan

Scan definitions in a makefile or a list of makefiles. This module actually calls “make” in order to examine the files and returns the values of all variables or makefile macros as a dictionary.

The main function in this module is “scan”. “scan” is called like this:

data= scan(filenames, verbose, dry_run)

Parameters are:

filenames
A single filename or a list of filenames
verbose
A boolean, if True print all system command calls to the console.
dry_run
A boolean, if True just print the system command calls but do not execute them.
sumolib.makefile_scan.scan(filenames, external_definitions=None, pre=None, warnings=True, verbose=False, dry_run=False)[source]

scan makefile-like definitions.

This takes a makefile name or a list of makefile names and returns a dictionary with all definitions made in these files. All definitions are resolved meaning that all variables that are used in the values of definitions are replaces with their values.

filenames
a single filename (string) or a list of filenames (list of strings)
external_definitions
A dict with variable settings that are pre-defined.
pre
None or a dict. For consecutive calls of this function providing an initially empty dictionary here speeds up calls by a factor of 2.
warnings
print a warning when a line cannot be parsed
verbose
if True, print command calls to the console
dry_run
if True, only print command calls to the console, do not return anything.

sumolib.mercurial module

mercurial support

class sumolib.mercurial.Repo(directory, hints, verbose, dry_run)[source]

Bases: object

represent a mercurial repository.

static checkout(spec, destdir, verbose, dry_run)[source]

spec must be a dictionary with “url” and “tag” (optional).

commit(logmessage)[source]

commit changes.

get_revision()[source]

return the current revision.

get_tag_on_top()[source]

return the “tag on top” property.

name()[source]

return the repo type name.

pull_merge()[source]

pull changes and try to merge.

push()[source]

push all changes changes.

rx_tag = <_sre.SRE_Pattern object at 0x7fef993e2870>
classmethod scan_dir(directory, hints, verbose, dry_run)[source]

return a Repo object if a mercurial repo was found.

This function returns <None> if no working repo was found.

If bool(hints[“write check”]) is True, return <None> if the repository directory is not writable.

For parameter “hints” see comment at __init__.

source_spec()[source]

return a complete source specification (for SourceSpec class).

sumolib.mercurial.assert_hg()[source]

ensure that mercurial exists.

sumolib.patch module

patch file support.

sumolib.patch.apply_patches(destdir, patchlist, verbose, dry_run)[source]

apply a list of patches to a directory.

sumolib.patch.assert_patch()[source]

ensure that patch exists.

sumolib.patch.call_patch(patch_file, target_dir, verbose, dry_run)[source]

call the patch utility.

This function does:
cd target_dir && patch -p1 < patch_file

sumolib.path module

path support

class sumolib.path.Repo(directory, hints, verbose, dry_run)[source]

Bases: object

represent a path.

static checkout(spec, destdir, verbose, dry_run)[source]

spec must be a string.

name()[source]

return the repo type name.

classmethod scan_dir(directory, hints, verbose, dry_run)[source]

return a Repo object.

source_spec()[source]

return a complete source specification (for SourceSpec class).

sumolib.repos module

Repository support

class sumolib.repos.ManagedRepo(sourcespec, mode, directory, lock_timeout, verbose, dry_run)[source]

Bases: object

Object for managing data in a repository.

Do pull before read, commit and push after write.

commit(message)[source]

commit changes.

finish_write(message)[source]

do commit and push.

local_changes()[source]

return if there are local changes.

prepare_read()[source]

do checkout or pull.

class sumolib.repos.SourceSpec(dict_=None)[source]

Bases: sumolib.JSON.Container

hold the source specification.

change_source(other)[source]

set source spec by copying information from another object.

This can also handle wildcards.

returns True if the spec was changed, False if it wasn’t.

change_source_by_tag(tag)[source]

change the source spec just by providing a tag.

returns True if the spec was changed, False if it wasn’t.

copy_spec(other)[source]

simply overwrite self with other.

classmethod from_string_sourcespec(elms)[source]

scan a source specification.

A sourcespec is a list of strings. Currently we support here: [“path”,PATH] or [“tar”,TARFILE,{PATCHFILES}] or [<repotype>,URL] or [<repotype>,URL,TAG,{PATCHFILES}]

where <repotype> may be one of the strings in sumolib.repos.known_repos

Here are some examples:

>>> SourceSpec.from_string_sourcespec(["path","ab"])
SourceSpec({'path': 'ab'})
>>> SourceSpec.from_string_sourcespec(["path"])
Traceback (most recent call last):
    ...
ValueError: invalid source spec 'path'
>>> SourceSpec.from_string_sourcespec(["path","a","b"])
Traceback (most recent call last):
    ...
ValueError: invalid source spec 'path a b'
>>> SourceSpec.from_string_sourcespec(["darcs","abc"])
SourceSpec({'darcs': {'url': 'abc'}})
>>> SourceSpec.from_string_sourcespec(["darcs","abc","R1-2"])
SourceSpec({'darcs': {'url': 'abc', 'tag': 'R1-2'}})
>>> SourceSpec.from_string_sourcespec(["darcs"])
Traceback (most recent call last):
    ...
ValueError: invalid source spec 'darcs'
>>> SourceSpec.from_string_sourcespec(["darcs","abc","R1-2","xy"])
SourceSpec({'darcs': {'url': 'abc', 'tag': 'R1-2', 'patches': ['xy']}})
>>> SourceSpec.from_string_sourcespec(["darcs","abc","R1-2","xy","z"])
SourceSpec({'darcs': {'url': 'abc', 'tag': 'R1-2', 'patches': ['xy', 'z']}})
is_repo()[source]

return if SourceSpec refers to a repository.

patches(new_val=None)[source]

return the patches if they exist.

path(new_val=None)[source]

return the path if the type is “path”.

sourcetype()[source]

return the type of the source.

spec_val()[source]

return the value of the source specification.

As SourceSpec is currently used this can be a string (type “path”) or a dict (all other types).

tag(new_val=None)[source]

return the tag if it exists.

url(new_val=None)[source]

return the url if it exists.

sumolib.repos.checkout(sourcespec, destdir, verbose, dry_run)[source]

check out a working copy.

sourcespec must be a SourceSpec object.

sumolib.repos.repo_from_dir(directory, hints, verbose, dry_run)[source]

scan a directory and return a repository object.

Hints must be a dictionary. This gives hints how the directory should be scanned. Currently we know these keys in the dictionary:

“ignore changes”: sumolib.utils.RegexpMatcher
All local changes in files that match the RegexpMatcher object are ignored. By this we can get the remote repository and tag from a directory although there are uncomitted changes. A common application is to ignore changes in file “configure/RELEASE”.
“dir patcher”: sumolib.utils.RegexpPatcher
This patcher is applied to the directory that is stored in the object.
“url patcher”: sumolib.utils.RegexpPatcher
This patcher is applied to the URL that is stored in the object.
“force local”: bool
If this is True, the returned repository object does not contain a remote repository url even if there was one.
“write check”: bool
If this is True, when the repository data directory is not writable the function returns <None>.
sumolib.repos.src_from_dir(directory, hints, verbose, dry_run)[source]

scan a directory and return a repository object.

Hints must be a dictionary. This gives hints how the directory should be scanned. Currently we know these keys in the dictionary:

“ignore changes”: sumolib.utils.RegexpMatcher
All local changes in files that match the RegexpMatcher object are ignored. By this we can get the remote repository and tag from a directory although there are uncomitted changes. A common application is to ignore changes in file “configure/RELEASE”.
“dir patcher”: sumolib.utils.RegexpPatcher
This patcher is applied to the directory that is stored in the object.
“url patcher”: sumolib.utils.RegexpPatcher
This patcher is applied to the URL that is stored in the object.
“force path” : bool
If this is True, a Path object is always returned, even if a repository was found.

sumolib.system module

System utilities.

sumolib.system.system(cmd, catch_stdout, catch_stderr, verbose, dry_run)[source]

execute a command with returncode.

execute a command and return the programs output may raise: IOError(errcode,stderr) OSError(errno,strerr) ValueError

sumolib.system.system_rc(cmd, catch_stdout, catch_stderr, verbose, dry_run)[source]

execute a command.

execute a command and return the programs output may raise: IOError(errcode,stderr) OSError(errno,strerr) ValueError

sumolib.system.test_program(cmd)[source]

test if a program exists.

sumolib.tar module

tar file support

class sumolib.tar.Repo(directory, tar_url, hints, verbose, dry_run)[source]

Bases: object

represent a tar.

static checkout(spec, destdir, verbose, dry_run)[source]

spec must be a dictionary with key “url”.

The tar file is placed in basename(destdir).

name()[source]

return the repo type name.

classmethod scan_dir(directory, hints, verbose, dry_run)[source]

return a Repo object.

source_spec()[source]

return a complete source specification (for SourceSpec class).

sumolib.utils module

Utilities for the SUMO scripts.

class sumolib.utils.Hints(specs=None)[source]

Bases: object

Combine hints for sumo-scan

add(spec)[source]

add a new hint.

flags(path)[source]

return the flags of a path.

class sumolib.utils.RegexpMatcher(regexes=None)[source]

Bases: object

apply one or more regexes on strings.

add(regexp)[source]

add a regexp.

match(str_)[source]

match the regular expression to a string

search(str_)[source]

search the regular expression in a string

class sumolib.utils.RegexpPatcher(tuples=None)[source]

Bases: object

apply one or more regexes on strings.

add(regexp_pair)[source]

add a from-regexp to-regexp pair.

apply(str_)[source]

apply the regular expressions to a string

sumolib.utils.ask_abort(question, force_yes=None)[source]

ask if the user wants to abort the program.

Aborts the program if the user enters “y”.

sumolib.utils.ask_yes_no(question, force_yes=None)[source]

ask a yes or no question.

returns:
True - if the user answered “yes” or “y” False - if the user answered “no” or “n”
sumolib.utils.changedir(newdir)[source]

return the current dir and change to a new dir.

If newdir is empty, return <None>.

sumolib.utils.dict_of_sets_add(dict_, key, val)[source]

add a key, create a set if needed.

Here is an example: >>> import pprint >>> d= {} >>> dict_of_sets_add(d,”a”,1) >>> dict_of_sets_add(d,”a”,2) >>> dict_of_sets_add(d,”b”,1) >>> pprint.pprint(d) {‘a’: set([1, 2]), ‘b’: set([1])}

sumolib.utils.dict_sets_to_lists(dict_)[source]

change values from sets to sorted lists.

Here is an example: >>> import pprint >>> d= {‘a’: set([1, 2]), ‘b’: set([1])} >>> ld= dict_sets_to_lists(d) >>> pprint.pprint(ld) {‘a’: [1, 2], ‘b’: [1]}

sumolib.utils.dict_update(mydict, other, keylist=None)[source]

update mydict with other but do not change existing values.

If keylist is given, update only these keys.

sumolib.utils.dirwalk(start_dir)[source]

walk directories, follow symbolic links.

Implemented to behave like os.walk On Python newer than 2.5 os.walk can follow symbolic links itself.

sumolib.utils.file_w_open(filename, verbose, dry_run)[source]

open a file for write.

sumolib.utils.file_write(fh, st, verbose, dry_run)[source]

write a line to a file.

sumolib.utils.is_standardpath(path, revision_tag)[source]

checks if path is complient to Bessy convention for support paths.

Here are some examples: >>> is_standardpath(“support/mcan/2-3”, “R2-3”) True >>> is_standardpath(“support/mcan/2-3”, “R2-4”) False >>> is_standardpath(“support/mcan/head”, “R2-3”) False >>> is_standardpath(“support/mcan/2-3+001”, “R2-3”) True

sumolib.utils.list_update(list1, list2)[source]

update a list with another.

In the returned list each element is unique and it is sorted.

sumolib.utils.mk_text_file(filename, lines, verbose, dry_run)[source]

create a text file.

sumolib.utils.opt_join(option, do_sort=False)[source]

join command line option values to a single list.

Here is an example: >>> a=[“a b”,”c”,”d e f”] >>> opt_join(a) [‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’]

sumolib.utils.rev2key(rev)[source]

convert a revision number to a comparable string.

This is needed to compare revision tags.

Examples of valid revisions:

2.3.4 2-3-4 R2-3-4 seq-2.3.4 head trunk

Here are some examples: >>> rev2key(“R2-3-4”) ‘002.003.004’ >>> rev2key(“2-3-4”) ‘002.003.004’ >>> rev2key(“head”) ‘-head’ >>> rev2key(“test”) ‘-test’ >>> rev2key(“test”)<rev2key(“R2-3-4”) True >>> rev2key(“R2-3-3”)<rev2key(“R2-3-4”) True >>> rev2key(“R2-3-5”)<rev2key(“R2-3-4”) False >>> rev2key(“R2-4-3”)<rev2key(“R2-3-4”) False >>> rev2key(“R1-3-4”)<rev2key(“R2-3-4”) True >>> rev2key(“R3-3-4”)<rev2key(“R2-3-4”) False

sumolib.utils.show_progress(cnt, cnt_max, message=None)[source]

show progress on stderr.

sumolib.utils.single_key(dict_)[source]

dict must have exactly one key, return it.

Raises ValueError if the dict has more than one key.

sumolib.utils.single_key_item(dict_)[source]

dict must have exactly one key, return it and it’s value.

Raises ValueError if the dict has more than one key.

sumolib.utils.split_path(path)[source]

split a path into (base, version, treetag).

Here are some examples: >>> split_path(“abc/def/1-3+001”) [‘abc/def’, ‘1-3’, ‘001’] >>> split_path(“abc/def/1-3”) [‘abc/def’, ‘1-3’, ‘’] >>> split_path(“abc/def/head+002”) [‘abc/def’, ‘head’, ‘002’]

sumolib.utils.split_treetag(path)[source]

split a path into head,treetag.

A path like /opt/Epics/R3.14.8/support/BIIcsem/1-0+001 is splitted to “/opt/Epics/R3.14.8/support/BIIcsem/1-0”,”001”

Here is an example: >>> split_treetag(“abc/def/1-0”) [‘abc/def/1-0’, ‘’] >>> split_treetag(“abc/def/1-0+001”) [‘abc/def/1-0’, ‘001’]

sumolib.utils.tag2version(ver)[source]

convert a tag to a version.

Here are some examples: >>> tag2version(“1-2”) ‘1-2’ >>> tag2version(“R1-2”) ‘1-2’ >>> tag2version(“R-1-2”) ‘1-2’ >>> tag2version(“seq-1-2”) ‘1-2’ >>> tag2version(“head”) ‘head’ >>> tag2version(“”) ‘’

Module contents