experiment package

Submodules

experiment.databaseObj module

class experiment.databaseObj.Database(base=None, force_load=None, discard_zeroes=True, progressbar=False, subpickle=True)

Bases: object

Database object. Holds a list of ExpResult objects.

Variables:

• base – path to the database (string)
• force_load – force loading the text database (“txt”), or binary database (“pcl”), dont force anything if None
• expResultList – list of ExpResult objects

base

This is the path to the base directory.

checkBinaryFile()

checkPathName(path, discard_zeroes)

checks the path name, returns the base directory and the pickle file name. If path starts with http or ftp, fetch the description file and the database. returns the base directory and the pickle file name

createBinaryFile(filename=None)

create a pcl file from the text database, potentially overwriting an old pcl file.

createExpResult(root)

create, from pickle file or text files

databaseVersion

The version of the database, read from the ‘version’ file.

fetchFromScratch(path, store, discard_zeroes)

fetch database from scratch, together with description. :param store: filename to store json file.

fetchFromServer(path, discard_zeroes)

getExpResults(analysisIDs=['all'], datasetIDs=['all'], txnames=['all'], dataTypes=['all'], useSuperseded=False, useNonValidated=False, onlyWithExpected=False)

Returns a list of ExpResult objects.

Each object refers to an analysisID containing one (for UL) or more (for Efficiency maps) dataset (signal region) and each dataset containing one or more TxNames. If analysisIDs is defined, returns only the results matching one of the IDs in the list. If dataTypes is defined, returns only the results matching a dataType in the list. If datasetIDs is defined, returns only the results matching one of the IDs in the list. If txname is defined, returns only the results matching one of the Tx names in the list.

Parameters:

• analysisID – list of analysis ids ([CMS-SUS-13-006,…]). Can be wildcarded with usual shell wildcards: * ? [<letters>] Furthermore, the centre-of-mass energy can be chosen as suffix, e.g. “:13*TeV”. Note that the asterisk in the suffix is not a wildcard.
• datasetIDs – list of dataset ids ([ANA-CUT0,…]). Can be wildcarded with usual shell wildcards: * ? [<letters>]
• txnames – list of txnames ([TChiWZ,…]). Can be wildcarded with usual shell wildcards: * ? [<letters>]
• dataTypes – dataType of the analysis (all, efficiencyMap or upperLimit) Can be wildcarded with usual shell wildcards: * ? [<letters>]
• useSuperseded – If False, the supersededBy results will not be included
• useNonValidated – If False, the results with validated = False will not be included
• onlyWithExpected – Return only those results that have expected values also. Note that this is trivially fulfilled for all efficiency maps.

Returns:

list of ExpResult objects or the ExpResult object if the list contains only one result

inNotebook()

Are we running within a notebook? Has an effect on the progressbar we wish to use.

loadBinaryFile(lastm_only=False)

Load a binary database, returning last modified, file count, database.

Parameters:lastm_only – if true, the database itself is not read. Returns:database object, or None, if lastm_only == True.

loadDatabase()

if no binary file is available, then load the database and create the binary file. if binary file is available, then check if it needs update, create new binary file, in case it does need an update.

loadTextDatabase()

simply loads the textdabase

needsUpdate()

does the binary db file need an update?

updateBinaryFile()

write a binar db file, but only if necessary.

class experiment.databaseObj.ExpResultList(expResList)

Bases: object

Holds a list of ExpResult objects for printout.

Variables:expResultList – list of ExpResult objects

experiment.datasetObj module

class experiment.datasetObj.CombinedDataSet(expResult)

Bases: object

Holds the information for a combined dataset (used for combining multiple datasets).

combinedLikelihood(nsig, marginalize=False, deltas_rel=0.2)

Computes the (combined) likelihood to observe nobs events, given a predicted signal “nsig”, with nsig being a vector with one entry per dataset. nsig has to obey the datasetOrder. Deltas is the error on the signal. :param nsig: predicted signal (list) :param deltas_rel: relative uncertainty in signal (float). Default value is 20%.

Returns:likelihood to observe nobs events (float)

getCombinedUpperLimitFor(nsig, expected=False, deltas_rel=0.2)

Get combined upper limit.

Parameters:

• nsig – list of signal events in each signal region/dataset. The list should obey the ordering in globalInfo.datasetOrder.
• expected – return expected, not observed value
• deltas_rel – relative uncertainty in signal (float). Default value is 20%.

Returns:

upper limit on sigma*eff

getDataSet(datasetID)

Returns the dataset with the corresponding dataset ID. If the dataset is not found, returns None.

Parameters:datasetID – dataset ID (string) Returns:DataSet object if found, otherwise None.

getID()

Return the ID for the combined dataset

getType()

Return the dataset type (combined)

sortDataSets()

Sort datasets according to globalInfo.datasetOrder.

totalChi2(nsig, marginalize=False, deltas_rel=0.2)

Computes the total chi2 for a given number of observed events, given a predicted signal “nsig”, with nsig being a vector with one entry per dataset. nsig has to obey the datasetOrder. Deltas is the error on the signal efficiency. :param nsig: predicted signal (list) :param deltas_rel: relative uncertainty in signal (float). Default value is 20%.

Returns:chi2 (float)

class experiment.datasetObj.DataSet(path=None, info=None, createInfo=True, discard_zeroes=True)

Bases: object

Holds the information to a data set folder (TxName objects, dataInfo,…)

checkForRedundancy()

In case of efficiency maps, check if any txnames have overlapping constraints. This would result in double counting, so we dont allow it.

chi2(nsig, deltas_rel=0.2, marginalize=False)

Computes the chi2 for a given number of observed events “nobs”, given number of signal events “nsig”, and error on signal “deltas”. nobs, expectedBG and bgError are part of dataInfo. :param nsig: predicted signal (float) :param deltas_rel: relative uncertainty in signal (float). Default value is 20%. :param marginalize: if true, marginalize nuisances. Else, profile them. :return: chi2 (float)

folderName()

Name of the folder in text database.

getAttributes(showPrivate=False)

Checks for all the fields/attributes it contains as well as the attributes of its objects if they belong to smodels.experiment.

Parameters:showPrivate – if True, also returns the protected fields (_field) Returns:list of field names (strings)

getEfficiencyFor(txname, mass)

convenience function. same as self.getTxName(txname).getEfficiencyFor(m)

getID()

Return the dataset ID

getSRUpperLimit(alpha=0.05, expected=False, compute=False, deltas_rel=0.2)

Computes the 95% upper limit on the signal*efficiency for a given dataset (signal region). Only to be used for efficiency map type results.

Parameters:

• alpha – Can be used to change the C.L. value. The default value is 0.05 (= 95% C.L.)
• expected – Compute expected limit ( i.e. Nobserved = NexpectedBG )
• deltas_rel – relative uncertainty in signal (float). Default value is 20%.
• compute – If True, the upper limit will be computed from expected and observed number of events. If False, the value listed in the database will be used instead.

Returns:

upper limit value

getTxName(txname)

get one specific txName object.

getType()

Return the dataset type (EM/UL)

getUpperLimitFor(mass=None, expected=False, txnames=None, compute=False, alpha=0.05, deltas_rel=0.2)

Returns the upper limit for a given mass and txname. If the dataset hold an EM map result the upper limit is independent of the input txname or mass.

Parameters:

• txname – TxName object or txname string (only for UL-type results)
• mass – Mass array with units (only for UL-type results)
• alpha – Can be used to change the C.L. value. The default value is 0.05 (= 95% C.L.) (only for efficiency-map results)
• deltas_rel – relative uncertainty in signal (float). Default value is 20%.
• expected – Compute expected limit, i.e. Nobserved = NexpectedBG (only for efficiency-map results)
• compute – If True, the upper limit will be computed from expected and observed number of events. If False, the value listed in the database will be used instead.

Returns:

upper limit (Unum object)

getValuesFor(attribute=None)

Returns a list for the possible values appearing in the DataSet for the required attribute.

Parameters:attribute – name of a field in the database (string). If not defined it will return a dictionary with all fields and their respective values Returns:list of values

likelihood(nsig, deltas_rel=0.2, marginalize=False)

Computes the likelihood to observe nobs events, given a predicted signal “nsig”, assuming “deltas” error on the signal efficiency. The values observedN, expectedBG, and bgError are part of dataInfo. :param nsig: predicted signal (float) :param deltas_rel: relative uncertainty in signal (float). Default value is 20%. :param marginalize: if true, marginalize nuisances. Else, profile them. :returns: likelihood to observe nobs events (float)

experiment.exceptions module

exception experiment.exceptions.DatabaseNotFoundException(value)

Bases: Exception

This exception is used when the database cannot be found.

exception experiment.exceptions.SModelSExperimentError(value=None)

Bases: Exception

Class to define SModelS specific errors

experiment.expResultObj module

class experiment.expResultObj.ExpResult(path=None, discard_zeroes=True)

Bases: object

Object containing the information and data corresponding to an experimental result (experimental conference note or publication).

Variables:

• path – path to the experimental result folder (i.e. ATLAS-CONF-2013-047)
• globalInfo – Info object holding the data in <path>/globalInfo.txt
• datasets – List of DataSet objects corresponding to the dataset folders in <path>

getAttributes(showPrivate=False)

Checks for all the fields/attributes it contains as well as the attributes of its objects if they belong to smodels.experiment.

Parameters:showPrivate – if True, also returns the protected fields (_field) Returns:list of field names (strings)

getDataset(dataId)

retrieve dataset by dataId

getEfficiencyFor(txname, mass, dataset=None)

Convenience function. Get the efficiency for a specific dataset for a a specific txname. Equivalent to: self.getDataset ( dataset ).getEfficiencyFor ( txname, mass )

getTxNames()

Returns a list of all TxName objects appearing in all datasets.

getTxnameWith(restrDict={})

Returns a list of TxName objects satisfying the restrictions. The restrictions specified as a dictionary.

Parameters:restrDict – dictionary containing the fields and their allowed values. E.g. {‘txname’ : ‘T1’, ‘axes’ : ….} The dictionary values can be single entries or a list of values. For the fields not listed, all values are assumed to be allowed. Returns:list of TxName objects if more than one txname matches the selection criteria or a single TxName object, if only one matches the selection.

getUpperLimitFor(dataID=None, alpha=0.05, expected=False, txname=None, mass=None, compute=False)

Computes the 95% upper limit (UL) on the signal cross section according to the type of result. For an Efficiency Map type, returns the UL for the signal*efficiency for the given dataSet ID (signal region). For an Upper Limit type, returns the UL for the signal*BR for the given mass array and Txname.

Parameters:

• dataID – dataset ID (string) (only for efficiency-map type results)
• alpha – Can be used to change the C.L. value. The default value is 0.05 (= 95% C.L.) (only for efficiency-map results)
• expected – Compute expected limit, i.e. Nobserved = NexpectedBG (only for efficiency-map results)
• txname – TxName object or txname string (only for UL-type results)
• mass – Mass array with units (only for UL-type results)
• compute – If True, the upper limit will be computed from expected and observed number of events. If False, the value listed in the database will be used instead.

Returns:

upper limit (Unum object)

getValuesFor(attribute=None)

Returns a list for the possible values appearing in the ExpResult for the required attribute (sqrts,id,constraint,…). If there is a single value, returns the value itself.

Parameters:attribute – name of a field in the database (string). If not defined it will return a dictionary with all fields and their respective values Returns:list of values or value

hasCovarianceMatrix()

id()

writePickle(dbVersion)

write the pickle file

experiment.infoObj module

class experiment.infoObj.Info(path=None)

Bases: object

Holds the meta data information contained in a .txt file (luminosity, sqrts, experimentID,…). Its attributes are generated according to the lines in the .txt file which contain “info_tag: value”.

Variables:path – path to the .txt file

addInfo(tag, value)

Adds the info field labeled by tag with value value to the object.

Parameters:

• tag – information label (string)
• value – value for the field in string format

dirName(up=0)

directory name of path. If up>0, we step up ‘up’ directory levels.

getInfo(infoLabel)

Returns the value of info field.

Parameters:infoLabel – label of the info field (string). It must be an attribute of the GlobalInfo object

experiment.metaObj module

class experiment.metaObj.Meta(pathname, discard_zeroes=None, mtime=None, filecount=None, hasFastLim=None, databaseVersion=None, format_version=201, python='3.6.2 (default, Apr 5 2018, 00:59:42) n[GCC 5.4.0 20160609]')

Bases: object

cTime()

current_version = 201

The Meta object holds all meta information regarding the database, like number of analyses, last time of modification, … This info is needed to understand if we have to re-pickle.

determineLastModified(force=False)

compute the last modified timestamp, plus count number of files. Only if text db

getPickleFileName()

get canonical pickle file name

isPickle()

is this meta info from a pickle file?

lastModifiedSubDir(subdir)

Return the last modified timestamp of subdir (working recursively) plus the number of files.

Parameters:

• subdir – directory name that is checked
• lastm – the most recent timestamp so far, plus number of files

Returns:

the most recent timestamp, and the number of files

needsUpdate(current)

do we need an update, with respect to <current>. so <current> is the text database, <self> the pcl.

printFastlimBanner()

check if fastlim appears in data. If yes, print a statement to stdout.

sameAs(other)

check if it is the same database version

versionFromFile()

Retrieves the version of the database using the version file.

experiment.txnameObj module

class experiment.txnameObj.Delaunay1D(data)

Bases: object

Uses a 1D data array to interpolate the data. The attribute simplices is a list of N-1 pair of ints with the indices of the points forming the simplices (e.g. [[0,1],[1,2],[3,4],…]).

checkData(data)

Define the simplices according to data. Compute and store the transformation matrix and simplices self.point.

find_index(xlist, x)

Efficient way to find x in a list. Returns the index (i) of xlist such that xlist[i] < x <= xlist[i+1]. If x > max(xlist), returns the length of the list. If x < min(xlist), returns 0. vertices = np.take(self.tri.simplices, simplex, axis=0) temp = np.take(self.tri.transform, simplex, axis=0) d=temp.shape[2] delta = uvw - temp[:, d]

Parameters:

• xlist – List of x-type objects
• x – object to be searched for.

Returns:

Index of the list such that xlist[i] < x <= xlist[i+1].

find_simplex(x, tol=0.0)

Find 1D data interval (simplex) to which x belongs

Parameters:

• x – Point (float) without units
• tol – Tolerance. If x is outside the data range with distance < tol, extrapolate.

Returns:

simplex index (int)

class experiment.txnameObj.TxName(path, globalObj, infoObj)

Bases: object

Holds the information related to one txname in the Txname.txt file (constraint, condition,…) as well as the data.

addInfo(tag, value)

Adds the info field labeled by tag with value value to the object.

Parameters:

• tag – information label (string)
• value – value for the field in string format

getEfficiencyFor(mass)

For upper limit results, checks if the input mass falls inside the upper limit grid. If it does, returns efficiency = 1, else returns efficiency = 0. For efficiency map results, checks if the mass falls inside the efficiency map grid. If it does, returns the corresponding efficiency value, else returns efficiency = 0.

Parameters:element – Element object Returns:efficiency (float)

getInfo(infoLabel)

Returns the value of info field.

Parameters:infoLabel – label of the info field (string). It must be an attribute of the TxNameInfo object

getValueFor(massarray, expected=False)

Access txnameData and txnameDataExp to get value for massarray.

Parameters:

• massarray – mass array values (with units), i.e. [[100*GeV,10*GeV],[100*GeV,10*GeV]]
• expected – query self.txnameDataExp

hasElementAs(element)

Verify if the conditions or constraint in Txname contains the element. Check both branch orderings.

Parameters:element – Element object Returns:A copy of the element on the correct branch ordering appearing in the Txname constraint or condition.

hasLikelihood()

can I construct a likelihood for this map? True for all efficiency maps, and for upper limits maps with expected Values.

hasOnlyZeroes()

class experiment.txnameObj.TxNameData(value, datatag, Id, accept_errors_upto=0.05)

Bases: object

Holds the data for the Txname object. It holds Upper limit values or efficiencies.

computeV(values)

Compute rotation matrix _V, and triangulation self.tri

Parameters:values – Nested array with the data values

countNonZeros(mp)

count the nonzeros in a vector

evaluateString(value)

Evaluate string.

Parameters:value – String expression.

flattenArray(objList)

Flatten any nested list to a 1D list.

Parameters:objList – Any list or nested list of objects (e.g. [[[100.,100.],1.],[[200.,200.],2.],..] Returns:1D list (e.g. [100.,100.,1.,200.,200.,2.,..])

formatInput(value, shapeArray)

Format value according to the shape in shapeArray. If shapeArray contains entries = *, the corresponding entries in value will be ignored.

Parameters:

• value – Array to be formatted (e.g. [[200.,100.],[200.,100.]])
• shapeArray – Array with format info (e.f. [‘*’,[float,float]])

Returns:

formatted array [[200.,100.]]

getDataShape(value)

Stores the data format (mass shape) and store it for future use. If there are wildcards (mass or branch = None), store their positions.

Parameters:value – list of data points

getUnits(value)

Get standard units for the input object. Uses the units defined in physicsUnits.standardUnits. (e.g. [[100*GeV,100.*GeV],3.*pb] -> returns [[GeV,GeV],fb] [[100*GeV,3.],[200.*GeV,2.*pb]] -> returns [[GeV,1.],[GeV,fb]] )

Parameters:value – Object containing units (e.g. [[100*GeV,100.*GeV],3.*pb]) Returns:Object with same structure containing the standard units used to normalize the data.

getValueFor(massarray)

Interpolates the value and returns the UL or efficiency for the respective massarray

Parameters:massarray – mass array values (with units), i.e. [[100*GeV,10*GeV],[100*GeV,10*GeV]]

interpolate(point, fill_value=nan)

loadData(value)

Uses the information in value to generate the data grid used for interpolation.

onlyZeroValues()

check if the map is zeroes only

removeUnits(value)

Remove units from unum objects. Uses the units defined in physicsUnits.standard units to normalize the data.

Parameters:value – Object containing units (e.g. [[100*GeV,100.*GeV],3.*pb]) Returns:Object normalized to standard units (e.g. [[100,100],3000])

removeWildCards(value)

Remove all entries = ‘*’ from value.

Parameters:value – usually a list containing floats and ‘*’ (e.g. [[200.,’*’],’*’],0.4],..)

round_to_n(x, n)

Module contents