Rework to flat entries for cache, saves space.

CHANGELOG.md

@@ -1,9 +1,15 @@
 # Changelog
 
-## 1.1.55
+## 1.2.57
 
 - add predicate memento.core/none-cache? that checks if cache is of none type
-- new
+- new function memento.core/caches-by-tag
+- important improvement of atomicity for invalidations by tag or function
+- important fix for thread synchronization when adding tagged entries
+- important fix for secondary indexes clearing
+- reduced memory use
+- improving performance on evictions when an eviction listener isn't used
+- *BREAKING CHANGE FOR IMPLEMENTATIONS* `invalidateId` is now `invalidateIds` and takes an iterable of tag ids, the implementations are expected to take care to block loads until invalidations are complete. Use the `memento.base/lockout-map` for this purpose.
 
 ## 1.1.54
 

README.md

@@ -264,6 +264,12 @@ You can add tags to the caches. Tags enable that you:
 
 This is a very powerful feature, [read more here.](doc/tags.md)
 
+## Loads and invalidations
+
+Cache only has a single ongoing load for a key going at any one time. For Caffeine cache, if a key is invalidated
+during the load, the load is repeated. This is the only way you can get multiple function invocations happen for a single
+cached function call.
+
 ## Namespace scan
 
 You can scan loaded namespaces for annotated vars and automatically create caches.
@@ -315,6 +321,9 @@ Set `-Dmemento.reloadable=false` JVM option (or change `memento.config/reload-gu
 
 ## Breaking changes
 
+Patch versions are compatible. Minor version change breaks API for implementation authors, but not for users,
+major version change breaks user API.
+
 Version 1.0.x changed implementation from Guava to Caffeine
 Version 0.9.0 introduced many breaking changes.
 

build.clj

@@ -4,7 +4,7 @@
             [org.corfield.build :as bb]))
 
 (def lib 'org.clojars.roklenarcic/memento)
-(def version (format "1.1.%s" (b/git-count-revs nil)))
+(def version (format "1.2.%s" (b/git-count-revs nil)))
 (def java-src-dir "java")
 
 (defn add-defaults [opts]

doc/cache-invalidation-concurrency.md

@@ -0,0 +1,44 @@
+# Invalidation and loads problems
+
+## 1. Accessing a cached value while a bulk invalidation is in progress
+
+You might have a cached function that calls another cached function and they operate on same data, but the invalidation
+is not atomic so the top function is invalidated and it immediately reconstructs an entry with invalid, but still
+not removed, data from the other cached function.
+
+This applies to tagged invalidation. 
+
+#### Solution
+
+Lockout/remove all the access to tagged entries of a specific tag until the invalidation is complete.
+
+Achieved by having a map with tag IDs that are under invalidation that is atomically updated to contain the set under
+invalidation, and a there are CountDownLatch objects that you can await for the invalidation to be over.
+
+#### What to do when cached value is invalidated
+
+If cached value is subject of ongoing invalidation pass, we need to recurse into a load, discarding the value. But 
+we don't want to recurse into a load, that still starts in the middle of an invalidation pass, as that will be also discarded.
+
+In order to minimize looping of the load function calls, we want to at least try to await the current invalidation pass.
+
+## 2. Load completes, but an invalidation has happened during that load
+
+Usually an invalidation in come in after a DB update or something like that. Any ongoing load will create some objects
+that will have the old data, but they won't be invalidated since the invalidation happened during load.
+
+For normal invalidation this is a simple problem, we can mark the Promise of the ongoing load as invalid, remove it from cache and retry.
+But for tagged invalidations, the tag IDs of the ongoing loads are not known before the result is calculated, so we need to detect
+all the tag id invalidations that happen after the load start.
+
+So each cache instance can have a map of ongoing loads to tag IDs that were invalidated during the load. This is combined with the
+lockout map to provide full coverage. 
+
+It gets even harder when we consider caches outside the JVM like Redis.
+
+#### Solution
+
+On a high level we need a history of recent tag IDs invalidated and a way to tell if they pertain to the ongoing load.
+
+We need to link those facts. So a part of the invalidation sequence would be somehow tagging every ongoing load with
+invalidated tag IDs that happen during the load. The lockout map is a source of such a snapshot.

doc/performance.md

@@ -109,54 +109,51 @@ Found 2 outliers in 60 samples (3,3333 %)
 ```text
 (cc/bench (f-memento 1))
 
-Evaluation count : 1665773940 in 60 samples of 27762899 calls.
-             Execution time mean : 34,272602 ns
-    Execution time std-deviation : 1,482470 ns
-   Execution time lower quantile : 33,454154 ns ( 2,5%)
-   Execution time upper quantile : 36,648990 ns (97,5%)
-                   Overhead used : 2,005368 ns
+Evaluation count : 854138220 in 60 samples of 14235637 calls.
+             Execution time mean : 70,745055 ns
+    Execution time std-deviation : 2,570125 ns
+   Execution time lower quantile : 68,659819 ns ( 2,5%)
+   Execution time upper quantile : 74,128774 ns (97,5%)
+                   Overhead used : 1,970580 ns
+
+Found 2 outliers in 60 samples (3,3333 %)
+	low-severe	 1 (1,6667 %)
+	low-mild	 1 (1,6667 %)
+ Variance from outliers : 22,2591 % Variance is moderately inflated by outliers
 
-Found 6 outliers in 60 samples (10,0000 %)
-	low-severe	 4 (6,6667 %)
-	low-mild	 2 (3,3333 %)
- Variance from outliers : 30,2710 % Variance is moderately inflated by outliers
 
 ```
 
-#### 1M misses (213 ns per miss)
+#### 1M misses (474 ns per miss)
 
 ```text
 (cc/bench (let [f-memento (m/memo identity {::m/type ::m/caffeine})]
             (reduce #(f-memento %2) (range 1000000))))
-Evaluation count : 360 in 60 samples of 6 calls.
-             Execution time mean : 213,842661 ms
-    Execution time std-deviation : 41,455030 ms
-   Execution time lower quantile : 142,937165 ms ( 2,5%)
-   Execution time upper quantile : 316,129642 ms (97,5%)
-                   Overhead used : 2,005368 ns
-
-Found 2 outliers in 60 samples (3,3333 %)
-	low-severe	 2 (3,3333 %)
- Variance from outliers : 89,4387 % Variance is severely inflated by outliers
+Evaluation count : 120 in 60 samples of 2 calls.
+             Execution time mean : 474,650866 ms
+    Execution time std-deviation : 76,082064 ms
+   Execution time lower quantile : 365,465019 ms ( 2,5%)
+   Execution time upper quantile : 641,739223 ms (97,5%)
+                   Overhead used : 1,992837 ns
 
 ```
 
-#### 1M misses for size 100 LRU cache (519 ns per miss)
+#### 1M misses for size 100 LRU cache (338 ns per miss)
 
 ```text
 (cc/bench (let [f-memento (m/memo identity {::m/size< 100 ::m/type ::m/caffeine})]
             (reduce #(f-memento %2) (range 1000000))))
-Evaluation count : 120 in 60 samples of 2 calls.
-             Execution time mean : 519,001949 ms
-    Execution time std-deviation : 22,003978 ms
-   Execution time lower quantile : 477,177477 ms ( 2,5%)
-   Execution time upper quantile : 549,082399 ms (97,5%)
-                   Overhead used : 2,005368 ns
+Evaluation count : 180 in 60 samples of 3 calls.
+             Execution time mean : 338,339882 ms
+    Execution time std-deviation : 15,865012 ms
+   Execution time lower quantile : 321,764748 ms ( 2,5%)
+   Execution time upper quantile : 370,249429 ms (97,5%)
+                   Overhead used : 1,970580 ns
 
-Found 5 outliers in 60 samples (8,3333 %)
-	low-severe	 4 (6,6667 %)
+Found 4 outliers in 60 samples (6,6667 %)
+	low-severe	 3 (5,0000 %)
 	low-mild	 1 (1,6667 %)
- Variance from outliers : 28,6961 % Variance is moderately inflated by outliers
+ Variance from outliers : 33,5491 % Variance is moderately inflated by outliers
 
 
 ```

doc/tags.md

@@ -124,3 +124,9 @@ Later you can invalidate tagged entries:
 
 (m/memo-clear-tags! [:person 1] [:user 33])
 ```
+
+## Invalidation atomicity
+
+```
+
+```

java/memento/base/CacheKey.java

@@ -1,14 +1,19 @@
 package memento.base;
 
+import clojure.lang.Util;
+
 import java.util.Objects;
 
 public class CacheKey {
     private final Object id;
     private final Object args;
 
+    private final int _hq;
+
     public CacheKey(Object id, Object args) {
         this.id = id;
         this.args = args;
+        this._hq = 31 * id.hashCode() + Util.hasheq(args);
     }
 
     public Object getId() {
@@ -24,12 +29,12 @@ public boolean equals(Object o) {
         if (this == o) return true;
         if (o == null || getClass() != o.getClass()) return false;
         CacheKey cacheKey = (CacheKey) o;
-        return Objects.equals(id, cacheKey.id) && Objects.equals(args, cacheKey.args);
+        return Objects.equals(id, cacheKey.id) && Util.equiv(args, cacheKey.args);
     }
 
     @Override
     public int hashCode() {
-        return Objects.hash(id, args);
+        return _hq;
     }
 
     @Override

java/memento/base/EntryMeta.java

@@ -1,6 +1,5 @@
 package memento.base;
 
-import clojure.lang.APersistentSet;
 import clojure.lang.IPersistentSet;
 import clojure.lang.PersistentHashSet;
 
@@ -9,10 +8,12 @@
 public class EntryMeta {
 
     public static final Object absent = new Object();
+    public static final EntryMeta NIL = new EntryMeta(null, false, null);
 
     public static Object unwrap(Object o) {
-        return o instanceof EntryMeta ? ((EntryMeta)o).getV() : o;
+        return o instanceof EntryMeta ? ((EntryMeta) o).getV() : o;
     }
+
     private Object v;
     private boolean noCache;
     private IPersistentSet tagIdents;

java/memento/base/ICache.java

@@ -5,7 +5,7 @@
 
 /**
  * Protocol for Cache. It houses entries for multiple functions.
- *
+ * <p>
  * Most functions receive a Segment object that should be used to partition for different functions
  * and using other :
  * - id: use for separating caches, it is either name specified by user's config, or var name or function object
@@ -15,15 +15,17 @@
 public interface ICache {
     /**
      * Return the conf for this cache.
+     *
      * @return
      */
     IPersistentMap conf();
 
     /**
      * Return the cache value.
-     *
+     * <p>
      * - segment is Segment record provided by the mount point, it contains information that allows Cache
      * to separate caches for different functions
+     *
      * @param segment
      * @param args
      * @return
@@ -32,6 +34,7 @@ public interface ICache {
 
     /**
      * Return cached value if present (and available immediately) in cache or memento.base/absent otherwise.
+     *
      * @param segment
      * @param args
      * @return
@@ -40,13 +43,15 @@ public interface ICache {
 
     /**
      * Invalidate all the entries linked a mount's single arg list, return Cache
+     *
      * @param segment
      * @return
      */
     ICache invalidate(Segment segment);
 
     /**
      * Invalidate all the entries linked to a mount, return Cache
+     *
      * @param segment
      * @param args
      * @return
@@ -55,19 +60,22 @@ public interface ICache {
 
     /**
      * Invalidate all entries, returns Cache
+     *
      * @return
      */
     ICache invalidateAll();
 
     /**
      * Invalidate entries with these secondary IDs, returns Cache. Each ID is a pair of tag and object
+     *
      * @param id
      * @return
      */
     ICache invalidateIds(Iterable<Object> id);
 
     /**
      * Add entries as for a function
+     *
      * @param segment
      * @param argsToVals
      * @return
@@ -76,12 +84,14 @@ public interface ICache {
 
     /**
      * Return all entries in the cache with keys shaped like as per cache implementation.
+     *
      * @return
      */
     IPersistentMap asMap();
 
     /**
      * Return all entries in the cache for a mount with keys shaped like as per cache implementation.
+     *
      * @param segment
      * @return
      */