Treant aggregation¶

These are the API components of datreant.core for working with multiple Treants at once, and treating them in aggregate.

Bundle¶

The class datreant.core.Bundle functions as an ordered set of Treants. It allows common operations on Treants to be performed in aggregate, but also includes mechanisms for filtering and grouping based on Treant attributes, such as tags and categories.

Bundles can be created from all Treants found in a directory tree with datreant.core.discover():

datreant.core.discover(dirpath='.', depth=None, treantdepth=None)¶

Find all Treants within given directory, recursively.

Parameters:	dirpath (string, Tree) – Directory within which to search for Treants. May also be an existing Tree. depth (int) – Maximum directory depth to tolerate while traversing in search of Treants. `None` indicates no depth limit. treantdepth (int) – Maximum depth of Treants to tolerate while traversing in search of Treants. `None` indicates no Treant depth limit.
Returns:	found – Bundle of found Treants.
Return type:	Bundle

They can also be created directly from any number of Treants:

class datreant.core.Bundle(*treants, **kwargs)¶

An ordered set of Treants.

Parameters:	treants (Treant, list) – Treants to be added, which may be nested lists of Treants. Treants can be given as either objects or paths to directories that contain Treant statefiles. Glob patterns are also allowed, and all found Treants will be added to the collection.

abspaths¶

Return a list of absolute member directory paths.

Members that can’t be found will have path None.

Returns:	names list giving the absolute directory path of each member, in order; members that are missing will have path `None`

add(*treants)¶

Add any number of members to this collection.

Arguments:	treants treants to be added, which may be nested lists of treants; treants can be given as either objects or paths to directories that contain treant statefiles; glob patterns are also allowed, and all found treants will be added to the collection

attach(*agglimbname)¶: Attach agglimbs by name to this collection. Attaches corresponding limb to member Treants.

categories¶: Interface to categories.

clear()¶: Remove all members.

filepaths¶

Return a list of member filepaths.

Members that can’t be found will have filepath None.

Returns:	names list giving the filepath of each member, in order; members that are missing will have filepath `None`

flatten(exclude=None)¶

Return a flattened version of this Bundle.

The resulting Bundle will have all members of any member Groups, without the Groups.

Parameters:	exclude (list) – uuids of Groups to leave out of flattening; these will not in the resulting Bundle.
Returns:	flattened – the flattened Bundle with no Groups
Return type:	Bundle

globfilter(pattern)¶

Return a Bundle of members that match by name the given globbing pattern.

Parameters:	pattern (string) – globbing pattern to match member names with

limbs¶: A set giving the names of this collection’s attached limbs.

map(function, processes=1, **kwargs)¶

Apply a function to each member, perhaps in parallel.

A pool of processes is created for processes > 1; for example, with 40 members and ‘processes=4’, 4 processes will be created, each working on a single member at any given time. When each process completes work on a member, it grabs another, until no members remain.

kwargs are passed to the given function when applied to each member

Arguments:	function function to apply to each member; must take only a single treant instance as input, but may take any number of keyword arguments
Keywords:	processes how many processes to use; if 1, applies function to each member in member order
Returns:	results list giving the result of the function for each member, in member order; if the function returns `None` for each member, then only `None` is returned instead of a list

names¶

Return a list of member names.

Members that can’t be found will have name None.

Returns:	names list giving the name of each member, in order; members that are missing will have name `None`

relpaths¶

Return a list of relative member directory paths.

Members that can’t be found will have path None.

Returns:	names list giving the relative directory path of each member, in order; members that are missing will have path `None`

remove(*members)¶

Remove any number of members from the collection.

Arguments:	members instances or indices of the members to remove

searchtime¶

Max time to spend searching for missing members, in seconds.

Setting a larger value allows more time for the collection to look for members elsewhere in the filesystem.

If None, there will be no time limit. Use with care.

tags¶: Interface to aggregated tags.

treanttypes¶: Return a list of member treanttypes.

trees¶: Obtain a View giving the Tree for each Treant in this Bundle.

uuids¶

Return a list of member uuids.

Returns:	uuids list giving the uuid of each member, in order

AggTags¶

The class datreant.core.agglimbs.AggTags is the interface used by Bundles to access their members’ tags.

class datreant.core.agglimbs.AggTags(collection)¶

Interface to aggregated tags.

add(*tags)¶

Add any number of tags to each Treant in collection.

Arguments:	tags Tags to add. Must be strings or lists of strings.

all¶: Set of tags present among all Treants in collection.

any¶: Set of tags present among at least one Treant in collection.

clear()¶: Remove all tags from each Treant in collection.

fuzzy(tag, threshold=80, scope='all')¶

Get a tuple of existing tags that fuzzily match a given one.

Parameters:	tags (str or list) – Tag or tags to get fuzzy matches for. threshold (int) – Lowest match score to return. Setting to 0 will return every tag, while setting to 100 will return only exact matches. scope ({'all', 'any'}) – Tags to use. ‘all’ will use only tags found within all Treants in collection, while ‘any’ will use tags found within at least one Treant in collection.
Returns:	matches – Tuple of tags that match.
Return type:	tuple

remove(*tags)¶

Remove tags from each Treant in collection.

Any number of tags can be given as arguments, and these will be deleted.

Arguments:	tags Tags to delete.

AggCategories¶

The class datreant.core.agglimbs.AggCategories is the interface used by Bundles to access their members’ categories.

class datreant.core.agglimbs.AggCategories(collection)¶

Interface to categories.

add(categorydict=None, **categories)¶

Add any number of categories to each Treant in collection.

Categories are key-value pairs that serve to differentiate Treants from one another. Sometimes preferable to tags.

If a given category already exists (same key), the value given will replace the value for that category.

Keys must be strings.

Values may be ints, floats, strings, or bools. None as a value will not the existing value for the key, if present.

Parameters:	categorydict (dict) – Dict of categories to add; keys used as keys, values used as values. categories – Categories to add. Keyword used as key, value used as value.

all¶

Get categories common to all Treants in collection.

Returns:	Categories common to all members.
Return type:	dict

any¶

Get categories present among at least one Treant in collection.

Returns:	All unique Categories among members.
Return type:	dict

clear()¶: Remove all categories from all Treants in collection.

groupby(keys)¶

Return groupings of Treants based on values of Categories.

If a single category is specified by keys (keys is neither a list nor a set of category names), returns a dict of Bundles whose (new) keys are the values of the category specified by keys; the corresponding Bundles are groupings of members in the collection having the same category values (for the category specied by keys).

If keys is a list or set of keys, returns a dict of Bundles whose (new) keys are tuples of category values. The corresponding Bundles contain the members in the collection that have the same set of category values (for the categories specified by keys); members in each Bundle will have all of the category values specified by the tuple for that Bundle’s key.

Parameters:	keys (str, list, set) – Valid key(s) of categories in this collection.
Returns:	Bundles of members by category values.
Return type:	dict

keys(scope='all')¶

Get the keys present among Treants in collection.

Parameters:	scope ({'all', 'any'}) – Keys to return. ‘all’ will return only keys found within all Treants in the collection, while ‘any’ will return keys found within at least one Treant in the collection.
Returns:	keys – Present keys.
Return type:	list

remove(*categories)¶

Remove categories from Treant.

Any number of categories (keys) can be given as arguments, and these keys (with their values) will be deleted.

Parameters:	categories (str) – Categories to delete.

values(scope='all')¶

Get the category values for all Treants in collection.

Parameters:	scope ({'all', 'any'}) – Keys to return. ‘all’ will return only keys found within all Treants in the collection, while ‘any’ will return keys found within at least one Treant in the collection.
Returns:	values – A list of values for each Treant in the collection is returned for each key within the given scope. The value lists are given in the same order as the keys from `AggCategories.keys`.
Return type:	list