Update:Remora Database/Object Caching

In bug 425315/r18173, a new object-based memcaching framework landed. This page outlines the basic concepts and use instructions.

Basic concepts

The basic idea behind the caching framework is the fact that instead of letting the controllers do a lot of data manipulation, we push most of this work into the models (where it belongs, arguably). So we have, or need to make, model methods along the lines of:

function getMyObject($id, $other, $stuff) {
    // switch off automatic query caching (1)
    $this->caching = false;
    
    // build unique (but arbitrary) identifier for this object (2)
    $identifier = array("addon:$id", $other, $stuff);
    
    // return cached object if it exists (3)
    if ($c = $this->Cache->readCacheObject($identifier)) return $c;
    
    // build result array (4)
    $myobject = $this->findById($id);
    // ... do some other magic if needed
    
    // cache that object (5)
    $this->writeCacheObject($identifier, $myobject, $expirationlists, $seconds_to_cache);
    
    // the rest
    $this->caching = true;
    return $myobject;
}

This will cache data object-wise. An example of how this looks can be found in the Addon model, getAddon().

When we wrote the cache object (5), we defined one or more (arbitrary but unique) expiration list names similar to "addon:5" or "version:12". Each of these lists is going to contain a number of cached object IDs that are related and can be expired together. Now at write time, we need to mark one or more of these lists for instant expiration. This is usually done in the afterSave() callback of a model (example: Addon model), but can be done anywhere else if wanted:

function afterSave() {
    // put a list ID on the list of lists to delete off the cache (6)
    $this->Cache->markListForFlush("addon:{$this->id}");
    // as always, remember to call the parent's callback before returning (7)
    return parent::afterSave();
}

This marks the "addon:$id" list for flushing; the actual flushing of the objects is then performed automatically in the app_controller's afterFilter() callback. No additional action is needed.

Specifics

1: switch off automatic query caching

If we don't switch off the automatic query caching, the objects will be invalidated, but the queries to regenerate the object would come from cache, defeating the purpose of the expiration in the first place.

We should also switch it back on before we return from the method.

2: Identifier

Each object is identified by a unique string or array. APP and LANG are added automatically, so in the model we are most likely to use an identification string like "addon:$id" along with the parameters passed into this method together as an identification array.

3: readCacheObject

Pretty simple: Just pass in your identifier.

4: building result array

This is doing all the work, including post-fetch array manipulations, so that the resulting variable is ready-to-use for the controller/view and doesn't need (much) additional manipulation after being returned.

5: writeCacheObject

writeCacheObject takes the identifier, the object, one or more expiration lists, and optionally an expiration length in seconds (defaults to the CACHE_PAGES_FOR config setting).

expiration list(s) are a string or array of strings containing arbitrary but unique expiration list IDs. An expiration list is a bundle of objects that will be expired together: The list ID "addon:12" would contain all objects (e.g., addon result sets, in different languages etc.) that need to be expired when the add-on number 12 changes. These lists will be looked at later when a partial cache invalidation of this list ID is triggered.

Another example for an (as of yet unimplemented) invalidation list ID could be "categories", containing all objects related to category lists (for menu, search bar, etc.).

Using an array of list IDs allows us to put the same object on different lists, in case it needs to be flushed with different events.

6: instant cache invalidation

At write time, we use markListForFlush() to mark one or more of the expiration list IDs for flushing. Each of these lists will be looped through and each contained object ID will be deleted from memcache.

7: parent callback

Our app_model contains code for the translation framework. Failure to call the parent's afterSave() callback after it was overwritten in the Addon model caused translations not to be saved (bug 456090), so... remember to do that.