Union Pooler Experiment Results
The idea of this experiment is to compare the overlap between the Union SDRs created by the Union Pooler in the case where the Temporal Memory has learned and in the case where it has not.
Generate a set of sequences of unique SDRs. Create a networking consisting of a Temporal Memory feeding into a Union Pooler.
For the training phase there are two conditions:
-
Control condition: No training at all
-
Experimental condition: Only train the Temporal Memory
Note that this implies the Union Pooler is never trained.
For the test phase, the network is run once on each sequence with learning off. We verified that there are no bursting columns in the Temporal Memory during this phase, i.e. the TM learned each sequence completely. During this phase a trace of the Union SDRs was recorded. The control and experimental conditions were run separately, then we computed the overlap (or intersection) between the two Union SDR traces.
To obtain a baseline we used the simplest set of parameters. Specifically predictedActiveOverlapWeight was set to 0.0 to have no effect and the decay of cell poolingActivation was turned off. The full parameter set used for the 2048 cell baseline can be found here.
Python script implementation of experiment is here.
Minimum union size is 40 since it is equivalent to the size of one SDR output from a Spatial Pooler. The maximum union size is 410 (20% of 2048). The learning condition’s union size (left) mirrors that of the non-learning condition's (middle). Both conditions oscillate between the min and max bounds growing linearly. Overlap between the two (right) reaches 126 on average, which is about 30% of the maximum possible overlap.
Direct link to plot image here
This experiment is the same as the previous except with 32 cells per column. Overlap (right) reaches 112.5 on average, which is about 27% of maximum overlap.
Direct link to plot image here
The idea of this experiment is to explore the limits of the distinctness (in terms of overlap) of the Union Pooler's union SDR. How much do these pooled representations overlap for different sequences? We also want to see, on average, if the overlap between the Union SDRs of two different learned sequences is less than the overlap between the Union SDR of a learned sequence and that of an unlearned sequence. This would demonstrate that Union SDRs of different sequences are farther apart than the learned and unlearned Union SDRs of the same sequence (on average).
Generate a set of sequences of unique SDRs. Create a networking consisting of a Temporal Memory feeding into a Union Pooler.
For training run only the Temporal Memory on each sequence with learning on. Perform two training passes training on each sequence once per pass with resets after each sequence presentation.
For the test phase, run the network once on each sequence with learning off. At the end of each sequence, we record the current Union SDR from the Union Pooler. Afterwards we compute the distinctness confusion of the recorded set of Union SDRs. This is the average, across every unique 2-tuple (pair) of Union SDRs in the recorded set, of the overlap between a pair of Union SDRs from the recorded set.
We checked that two conditions held during the test phase: 1) the number of bursting columns in TM was 0 and 2) the number of active cells in TM (on average) equaled to the number of active columns (i.e., there was one cell active per column)
The parameter sets were based on those of Experiment 1. The sequence length was 13 to ensure that the Union Pooler capacity limit of 20% was reached. The exact parameter files can be found here.
Python script implementation of experiment is here.
Recall in experiment 1, the max overlap was 126 on average. We found that the distinctness values for different sequences were lower that this on average. Note that the x-axises in the plots below are on a logarithmic scale and the apparent growth is slower than if it were on a linear scale. From the 1024-column Temporal Memory condition to the 2048 condition the overlap essentially doubles. Although the max distinctness grows with increased amounts of sequences, the confidence intervals about the mean remain tight.
Direct link to plot image here
Direct link to plot image here
The goal of this experiment is to characterize the robustness of the Union Pooler's Union SDR to various amount of variations or perturbations in the input data stream. We also want to explore the trade-off between Union SDR stability to variations and responsiveness to actual changes from one sequence to another, e.g., from A-B-C to X-Y-Z.
Data: Sequences generated from an alphabet. Each pattern is different across all sequences.
Training Phase: Train TM-UP network on sequences over 3 repetitions of each sequence. Next, train a KNN Classifier on the output of Union Pooler during the presentation of a single sequence. Here, the classifier learns to associate each Union SDR output during the sequence presentation with the same class label of the particular sequence.
Testing phase: For a number of test presentations, select one of the sequences randomly. For each presentation of a pattern in the sequence, there is a chance of a perturbation in which the expected pattern is not shown. Specifically, the following types of perturbation may occur:
-
Substitution of the expected pattern with another pattern randomly selected from all patterns in the alphabet
-
Skipping over the expected pattern and presenting the next pattern in the sequence
-
Adding in a pattern randomly selected from all alphabet patterns that delays the presentation of the expected pattern by one time step
In the standard version of the experiment, given that a perturbation is to occur for a given pattern presentation, the perturbation type is randomly selected with equal likelihood among the above 3 options.
During the test phase, for each pattern presented to the network, we record the KNN Classifier's classification of the Union SDR. The ground truth is taken to be the label of the sequence currently presented to network. This implies that for perturbations, the correct classification is still the category of the current sequence. To summarize the results, the average rate of correct classifications during the test phase was computed.
There are two parameter sets: one with Union SDR size at 20% and another at 2%. There were a range of perturbation rates from 0-90% in steps of 10% for these two sets.
The full parameter set can be found here.
Python script implementation of experiment is here.
Here we plot the classification accuracy as a function of increasing perturbation rate. The red and purple conditions are the "baselines" for perturbations of the 3 types with equal probability. The 20% condition boasts better performance. The other three conditions explore having only one type of perturbation at a time with a 20% Union SDR limit.
In this variation we more strictly controlled for the number of perturbations that occurred. Rather than adding perturbations randomly, we tested an exact number of substitution-only perturbations occurring at random locations in the sequence. The conditions for the number of perturbations ranged from 0 to the length of the sequences tested (13). The classification scoring method was the same as in the original test. We found similar classification performance to that of the original test.
