Caching, Part 3: how Smritea uses smartcache

smartcache is an open-source Go library that gives one generic Cache[T] both read-through and write-through behavior over a pluggable store. Smritea uses it for every cache that sits in front of a database row. This post covers four choices Smritea made on top of it:

1. Write-then-evict.
2. Tenant-safe reads.
3. Fail-open versus fail-closed.
4. Bulk eviction for data erasure.

Parts 1 and 2 of this series introduce the library: Part 1 and Part 2. The source is at github.com/Bytonomics/smartcache.

The library in one example

Smritea registers one entity cache per kind of row. Every cache needs a TTL, so a missed invalidation heals itself within a bounded window.

mgr, err := smartcache.NewManager(
	redisstore.New(conn),
	smartcache.WithDefaultJitterFraction(0.10),
)
if err != nil {
	return err
}
ttl := time.Hour
users, err := smartcache.Register[User](mgr, "user",
	&smartcache.EntityOptions{TTL: &ttl})
if err != nil {
	return err
}

// Read: cache first, loader only on a miss.
u, outcome, err := users.GetByKey(ctx, id,
	func(ctx context.Context) (*User, error) {
		return db.LoadUser(ctx, id)
	})

GetByKey returns the value and an outcome such as Hit, Loaded or LoadedNotCached. Each Smritea service builds one manager, backed by Redis, and gives every key a service prefix.

Write-then-evict

A write updates the database first. Only after the write succeeds does it evict the cache key.

// Write: database first, then evict.
if err := db.UpdateUser(ctx, u); err != nil {
	return err // nothing evicted, cache untouched
}
if err := users.EvictByKey(ctx, u.ID); err != nil {
	log.Warn("evict failed; TTL bounds staleness", err)
}

Eviction replaces an in-place update on purpose. The library's README says delete-on-write avoids the concurrent stale-set race. Two writers can otherwise finish in the wrong order and leave an old value in the cache.

Two details matter:

• The code builds and validates the key before the write. A blank ID means the write never runs.
• For IDs that the database generates during the write, the code builds the key after the write instead.

Reads that stay inside one tenant

Smritea has two kinds of tenants. In the control plane, the tenant is an organization. In the data plane, the tenant is an app.

In the control plane, the cache key is the entity ID alone. Every organization shares the entry, so the loader must not limit itself to one organization. The reason is the negative cache: smartcache can cache a "not found" for 45 seconds. A loader limited to the wrong organization would cache "not found" for the real owner.

The read goes through an unscoped loader. Then the code compares the record's owning organization with the caller's organization on every read, hit or miss. A record from another organization reads as not found.

In the data plane, keys carry the tenant. Each key has the form app-<appId>::<hash of the ID>, behind the service prefix. The scope check rejects empty IDs. It also rejects whitespace, control characters and Redis glob characters, because a glob character could match another tenant's keys in a pattern delete.

Fail-open and fail-closed

The library reads fail-open. If the store errors on a read, the library treats it as a miss and calls the loader. The comment in the source says a backend read error must not fail the request more than the database does. If the write-back after a load fails, the value still returns with the outcome LoadedNotCached.

Smritea adds its own choices on top. The table says what Smritea does in each failure case and why.

A Redis outage alone does not fail the sign-up gate. The library treats the outage as a miss, so the read falls through to the database.

Erasing an app's cache

Erasure needs a bulk delete. smartcache offers EvictByKeyPrefix, which scans the keyspace and deletes every key under a prefix.

// Bulk erase, rare, never on the hot path:
n, err := users.EvictByKeyPrefix(ctx, "app-123::")
// err may be non-nil with n > 0: repeat the call,
// or let the TTL backstop expire the rest.

End the prefix with your own separator. Then app-12:: cannot match app-123::. The library rejects an empty prefix and a prefix with *.

When Smritea deletes an app, the data plane first deletes the database rows. Then it sweeps the cache, so nothing repopulates from the database. For each smartcache namespace it evicts the app's prefix and the app's own key. It also deletes nine groups of raw Redis keys by pattern. Every step runs even if an earlier one fails, and the errors come back joined. An eviction error fails the erasure request. That makes this path fail-closed on cache errors.

A startup check guards the sweep. The service panics at construction if a smartcache namespace has no evictor, because a missing namespace would leave app data behind.

When not to use smartcache

smartcache is for caching a database row. If Redis is the source of truth and no row backs the value, use the raw store. Smritea does this for revoked refresh tokens and CLI login codes. Hot quota counters also live in raw Redis.

Summary

• Evict after a committed write, not before.
• Share a cache entry across tenants only if the loader and the post-read check are tenant-safe.
• Choose fail-open or fail-closed per path, and write the reason in the code.
• Keep a TTL on every entry, so a lost eviction heals itself.