Skip to content

Latest commit

 

History

History
775 lines (477 loc) · 24.7 KB

tree_api.md

File metadata and controls

775 lines (477 loc) · 24.7 KB

Warning: as of 22/April of 2018 there have been some serious changes in the tree management, which are still not documented. We hope, although that, as an API they are full compatible

Table of Contents

General preview

We have developed the classes here to be models for the Qt Model-View Programming.

They are tailored for the case we have to deal with two dimensional data, and one of the dimensions can be also viewed as a tree

o ------> row 
      |-> row 
           |
           |--> row 
           |--> row

Each row is

    row head + data col1 + data col2 + ....

To represent the tree we have developed the GuideItemModel (from QStandardItemModel)

And to handle each item individually we created the GuideItem (from QStandardItem

Column 0 of each row is the row header and has two data inside it:

  • the internal key accessed via the Qt.Role (Qt.QUserRole +1) and
  • an human readable value. Accessd via the Qt.Role Qt.QDisplayRole

When we navigate the tree we access for each row the row header.

The other columns of the row is what we call payload, and can be accessed thru the header individually or as a block

General Auxiliary functions

searchStandardItem(item,value,role):

Implement a binary search inside a QStandardItemModel.

Extremely faster than the .find method.

Only perform the search in one hierarchical level

currently REQUIRES that the children be ordered by default. As is the case in danacube

Actual binary search code was retrieved from StackOverflow. Sadly i lost the reference

  • Input Parameters

    • item (QStandardItem) parent of the level we are searching for
    • value value we are searching for
    • role the Qt Role for the value. (An item can contain different data associated to different roles)

  • Returns

    the item which has that value or None

  • Programming notes.

    Look for GuideModel.searchHierarchy for usage

TODO Make a variant which do not requires previous order

class TreeFormat(object):

This class mantains the format definition (for a display or background roles) for the value contents of an item in GuideItemModel (see the .data method)

NOTES The current implementation is pretty lame, and compatibility forced from previous code, and the parameters needed by util.numeros.fmtNumber

Properties

stats

Boolean. IF the tree will keep statistics for each row

__format __

A dictionary containing following entries

  • yellowoutliers Boolean. If statistic ooutliers in a row will be backgrounded in yellow. Default True
  • rednegatives Boolean. If negative values will be backgrounded in red- Default True
  • thousandseparator a Char. character for thousands separation. Default "." (spanish style)
  • decimalmarker a Char. decimal character. Default "," (spanish style)
  • decimalplaces number of default decimal places for floating point values. Default 2

Methods

init(self,**opciones):

  • Input parameters. A kwparms dict of
    • stats Boolean. IF the tree will keep statistics for each row
    • __format __ A dictionary containing following entries
      • yellowoutliers Boolean. If statistic outliers in a row will be backgrounded in yellow. Default True
      • rednegatives Boolean. If negative values will be backgrounded in red- Default True
      • thousandseparator a Char. character for thousands separation. Default "." (spanish style)
      • decimalmarker a Char. decimal character. Default "," (spanish style)
      • decimalplaces number of default decimal places for floating point values. Default 2

class GuideItem(QStandardItem):

This class is an specialization of QStandardItem for use with GuideItemModel

In our current implementation GuideItem instantiates every element of the row. We view a row as containing:

    row head + payload  (the equivalent of row head + data col1 + data col2 + ....

Column 0 of each row is the row header and has two data inside it:

  • the internal key accessed via the Qt.Role (Qt.QUserRole +1) and
  • an human readable value. Accessed via the Qt.Role Qt.QDisplayRole

When we navigate the tree we get the row header. for each row

The other columns of the row is what we call payload, and can be accessed thru the header individually or as a block

Attributes

both are given value during the application workflow

originalValue

A value for the item which to return

stats

a dict with following entries (excluding null values during computation)

  • avg average
  • std standard deviation
  • max max value
  • median median value
  • min minimum value
  • out_low upper threshold for low outliers
  • out_hig lower threshold for upper outliers

Methods reimplemented from QStandardItem.

See documentation there

init(self,*args):

type(self):

We define three different types of items (really of the rows)

  • TOTAL which correspond to the grand total
  • BRANCH rows which have children
  • LEAF rows without children

data(self,role):

We make Qt.UserRole +1 as default role if Qt.DisplayRole returns None

__str__(self):

__repr__(self):

__getitem__(self,campo):

We define three types of fields

  • 'key' the UserRole content
  • 'value' the DisplaRole content
  • a number: the playload column

General methods QStandardItem should have

depth(self):

hierachical level of the row

  • returns
    • an integer with the current level, starting from 0

searchChildren(self,value,role=None):

We seach for a specific value inside the children of the current row

  • Input parameters

    • value the value to be searched for. Type has to be QStandardItem.setData compatible
    • role Qt.Role, if not specified Qt.UserRole +1

  • Returns

    • An item which contains this value or None

  • Implementation notes

    Wide faster and hierachical level specific than model().find Currently the values of the rows have to be sorted in advance

getColumn(self,col):

Returns the item at column col for the current row

  • Input parameters

    • col the index of the column requested. 0 is not allowed

  • Returns

    • The item at that column or None if it doesn't exist

  • Implementation notes

    Surprisingly such a method does not exist at QStandardItem

setColumn(self,col,value,role=None):

Method to set , for the current row, a payload entry at position col with value value . It's a destructive function: if the column already exist is replaced with the new version Can only be executed on GuideItems with column 0 (the head of the row)

  • Input parameters

    • col the index of the column to be set. 0 is not allowed
    • value the value to be inserted. Type be QStandardItem.setData compatible
    • role Qt.Role, if not specified Qt.UserRole +1

  • returns

    • a reference to the new column item or None (if 0 happens to be specified at col)

  • Implementation notes.

    QtStandardItem.insertColumn and QStandardItem.insertColumns are most probably faulty, and, even if worked, the need to create a list of QStandardItem (or derived) before using QStandardItem.insertColumns is unconfortable to program. Column 0 is not allowed because (as it is the head of the row,i.e. self) it would destroy self

setUpdateColumn(self,col,value,role=None):

Method to set , for the current row, a payload entry at position col with value value . If the item already exist it is updated, else is created Can only be executed on GuideItems with column 0 (the head of the row)

  • Input parameters

    • col the index of the column to be set. 0 is not allowed
    • value the value to be inserted. Type be QStandardItem.setData compatible
    • role Qt.Role, if not specified Qt.UserRole +1

  • returns

    • a reference to the new column item or None (if 0 happens to be specified at col)

rowTraverse(self):

EXPERIMENTAL

generator to navigate the payload elements inside a row

  • Implementation note:

If executed over an arbitrary item it starts on the following column

If the corresponding column is still not defined it returns a QStandardItem, not a real column

If we use expanded functionality of the class we must code like

    for item in self.rowTraverse():
        if type(item) == QStandardItem:
            pass
        else:
            ...

General methods which make up the API for user functions

getPayload(self):

Returns a list with the values of the payload (columns 1 .. of the current row

setPayload(self,lista):

Updates in one step all the values of the payload

If len(lista) < len(vector) only those values get updated; otherwise the vector will be expanded as apropiate

  • Input parameters
    • lista a list with the new values for the vector

lenPayload(self):

Length of the payload for this row

  • Returns

    • Length of the payload

  • implementation notes

QStandardItem.columnCount() does not seem to work, it should be lenPayload = columnCount -1. In this case the use of the generator seems not to be that performant

getPayloadItem(self,idx):

Returns the data at payload position idx for the current row It corresponds to column idx + 1

  • Input parameters

    • idx the index inside the payload.

  • Returns

    • The item at that column or None if it doesn't exist

setPayloadItem(self,idx,valor):

Method to set , for the current row, a payload column at index idx with value value . If the item already exist it is updated, else is created It will correspond to column = idx +1

  • Input parameters

    • idx the index of the payload to be set.
    • value the value to be inserted.

  • returns

    • a reference to the new column item or None

getKey(self):

returns the internal key of the current row

getLabel(self):

returns the display value for the current row

Application specific methods

getColumnData(self,idx,role=None):

Returns data from an specified index and role inside the payload.

  • Input parameter

    • idx the index in the payload
    • role data associated to that role. Default is Qt.UserRole +1

  • Implementation notes For compatibility with other tree implementations

getFullHeadInfo(self,**parms):

  • parms
    • content = ('key','value')
    • format = ('simple','string','array')
    • delimiter (only if format == string)
    • sparse = boolean. Default False

getFullKey(self):

obtains the key of the row in string format (i.e each hierachical step separated by DELIMITER ).

Key is the DB internal key and Qt.UserRole + 1 data

getFullDesc(self):

obtains the description of the row in array format (i.e each hierachical step in an occurrence of a list ).

Description is the presentation values of the keys and Qt.DisplayRole data

getStatistics(self):

returns the statistics gathered for this row

isTotal(self):

Boolean to check if the row is a totalizer row

isBranch(self):

Boolean to check if the row is a branch row (has children of its own)

isLeaf(self):

Boolean to check if the row is a leaf row (has no children of its own)

restoreBackup(self):

Returns the payload item to the original value

If executed at the header restores the full row

setBackup(self):

Sets the current value as backup value (self.originalValue) for a payload column If executed at the header restores the full row

For orthogonality sake, we allow to process the full row if executed at the header

setStatistics(self):

Executes the statistic gathering routine at the row

Only executable at the row header

The statistics routine is util.numeros.stats and creates a dict with following entries (excluidn null values)

  • avg average
  • std standard deviation
  • max max value
  • median median value
  • min minimum value
  • out_low upper threshold for low outliers
  • out_hig lower threshold for upper outliers

simplify(self):

We get the value of the elements in the payload which are not Null (None)

Only to be executed at the head

  • return
    • A list with the values of the non null payload
    • A list with the indexes of those elements

simplifyHierarchical(self):

We get the value of the elements in the payload at this level and its parents. Only return those which are not Null at this level

Only to be executed at the head

  • return
    • A list for each hiearchical level of lists of values of the non null payload
    • A list with the indexes of those elements

class GuideItemModel(QStandardItemModel):

GuideItemModel is an extension of Qt's QStandardItemModel, specially crafted for holding trees with vector like items (defined as GuideItem, see below). They also have some attributes and methods specific for the handling of danacube

In danacube the guides of a view (row_hdr_idx, col_hdr_idx) are implemented with this class

Implementation details

Of the methods, following are reimplemetation of base methods, and are task tailored * __init__ * data

Those are provided as extensions to the general model I miss * traverse * numRecords * lenPayload * __searchHierarchy

The rest is task tailored for danacube, but the as* methods can be used as a frame for less specific work

Attributes

datos

A TreeFormat object. Holds the formating info for the tree. THe name is merely historical

name

Holds the name of the guide on which the tree is based. Is not mandatory

vista

En que vista se utiliza (activado en core.vista.toNewArray*

orthogonal

Cual es el otro modelo que complementa la vista (activado en core.vista.toNewArray*)

init(self,parent=None):

traverse(self,base=None):

Generator to navigate the tree. It only reads the head items of each rows

  • Input parameter
    • base initial item to process. default is .invisibleRootItem, but this element is not yielded
  • returns
    • next item in sequence

TODO

It would be nice to put it functionally on par to util.treebasic.TreeModel

numRecords(self,type=None):

Returns the number of items in the tree. The standard model rowCount method only gives the number of direct children of the first level

  • Input Parameters

    • type integer the type to be searched for (an integer above 1000. See Qt.StandardItem.type())

  • returns The number of items in the tree

asDict(self):

returns a dictionary view of the tree. The purpose is to allow a quasi direct access to the item based on the key values

*Returns

A dictionary whose keys are the fullkeys of the items. The fullkey is a string which concatenates the keys of the tree hierarchy of the item. To each item is attached, as value a dict with following entries

  • idx the ordinal of the item in the tree traversal

  • objid a reference to the item itself

  • Implementation notes Trees are meant to be navigated via traversal, but are complex to use as direct access (via the key). This function generates a dictionary which can be accessed directly via the key. Python didn't allowed a QStandardItem as a dict key, so we had to use the key as index. It is the full hierachical key to ensure unicity

asHdr(self,**parms):

Returns a list with an element for every item in the tree. Each element contains data which can be used as headers

  • Input parms.

A kwparm whith can contain any of the following entries

  • parms used by GuideItem.getFullHeadInfo

    • content = ('key','value'). Default 'value'
    • format = ('simple','string','array'). Default 'simple'
    • delimiter (only if format == string). Concatenation delimiter
    • sparse = boolean. Default False

  • specific parms

    • offset. a number of blank elements to precede the list
    • normArray. a number only if format = array, all elements will have at least this length (num of entries)

  • returns A list with one element for each item in the tree. What is offered in each element varies according to the input parameters:

    • Content
      • if 'key' we return the internal key of the item (the domain's code of the guide).
      • if 'value' its display text (the domain's desc of the guide
    • format
      • if 'simple' just the value for this entry (production rule in guide)
      • if 'string' the hierarchy of keys concatenated in one string
      • if 'array' an array with the hierarchy of keys

  • Implementation notes In danacbue the need for header structures related to the guides has been an usual task. This function is a generic one to generate such headers

asDictFilter(self,filter):

returns a dictionary view of the tree, but only those items for which the filter applies.

  • Input parameter
    • filter a function which returns a boolean and accept a QStandardItem as parameter

*Returns

A dictionary whose keys are the fullkeys of the items. The fullkey is a string which concatenates the keys of the tree hierarchy of the item. To each item is attached, as value a dict with following entries

  • idx the ordinal of the item in the filtered tree traversal

  • objid a reference to the item itself

  • oidx the original index without filter

  • Implementation notes Enhanced version of the asDict method. Allows any filtering

asHdrFilter(self,filter,**parms):

Returns a list with an element for every item in the tree for which the filter is True. Each element contains data which can be used as headers

  • Input parms.

  • filter a function which returns a boolean and accept a QStandardItem as parameter

  • A kwparm whith can contain any of the following entries

    • parms for getFullHeadInfo
      • content = ('key','value')
      • format = ('simple','string','array')
      • delimiter (only if format == string)
      • sparse = boolean. Default False
    • specific parms
      • offset. a number
      • normArray. a number only if format = array

  • returns A list with one element for each item in the filtered tree. What is offered in each element varies according to the input parameters:

    • Content
      • if 'key' we return the internal key of the item (the domain's code of the guide).
      • if 'value' its display text (the domain's desc of the guide
    • format
      • if 'simple' just the value for this entry (production rule in guide)
      • if 'string' the hierarchy of keys concatenated in one string
      • if 'array' an array with the hierarchy of keys

  • Implementation notes Enhanced version of the asDict method. Allows any filtering

lenPayload(self,leafOnly=False):

Returns the maximum lenght of the payload for each item. (in fact the length of the orthogonal model) QStandardItem.columnCount() give some incorrect results

  • Input parameters

    • leafOnly. Boolean. counting is done only for leaf elements

  • returns

    • the number of columns expected

  • Programming notes Implementation will vary, most probably

searchHierarchy(self,valueList,role=None):

Does a search thru all the hierarchy given the key/value data

  • Input parameters

    • valueList an array with the hierachy data to be searched
    • role the Qt.Role the data is associated with

  • Returns An Item of the tree which mastches the valueList

  • Programming notes Which data is searched depends on the role. Qt.UserRole +1 searches for internal keys; Qt.DisplayRole for description

setStats(self,switch):

Sets (and evaluate) the per row statistics. While they are per row(item) are calculated once per tree

  • Input parameters:

    • switch a Boolean, activates or deactivates the statistics

  • returns none. If activated loads the items with the corresponding statistics

data(self,index,role):

Reimplementation of QStandardItemModel.data for the needs of danacube. It will be invoked when a view associated with the model is redrawn

  • Input parameters

    • index a QIndexModel which identifies the item
    • role which Qt.Role are we handling

  • Programming notes We define special actions for following cases

    • Qt.TextAlignmentRole. For column 0 -the value- left aligned, else -the vector- right aligned (they are numbers)
    • Qt.BackgroundRole. Color if the conditions in TreeFormat are met
    • Qt.DisplayRole. Formats the numeric vector

MonkeyPatch section

Definition of synonyms of methods of the previous classes, used either for simplicity or compatibility

  • GuideItemModel.len = GuideItemModel.numRecords
  • GuideItem.childCount = GuideItem.rowCount
  • GuideItem.gpi = GuideItem.getPayloadItem
  • GuideItem.spi = GuideItem.setPayloadItem
  • GuideItem.getLevel = GuideItem.depth
  • GuideItem.getHeader = GuideItem.getFullHeadInfo