ThreadSafeDatabaseMetadata

CuneiFox stores data for different companies in different file paths. It is essential that the program is able to distinguish and choose correct database files to work on. This is not an issue if the client-side request is processed one-by-one and no internal processes dealing with any of the companies' data are ever needed. However, these conditions severely limit the functionality and latency of the program. Hence this modified Metadata class is developed.

Main Idea & Origin

The core essential of ThreadSafeMetadata is to somehow map each request to its proper database files, as well as initiate and teardown the database connections without affecting other requests running in parallel.

CuneiFox's flavour of ThreadSafeMetadata is derived from what written under Thread-Safety and Multiple Databases in Peewee 3.14.4 Documentation. (The documentation for this version does not exist on the main channel anymore. The link points to a copied version hosted on another site. The essential content is also copied here for safekeeping.)

# Content from Peewee 3.14.4 Documentation

import threading
from peewee import Metadata, Model
class ThreadSafeDatabaseMetadata(Metadata):
    def __init__(self, *args, **kwargs):
        # database attribute is stored in a thread-local.
        self._local = threading.local()
        super(ThreadSafeDatabaseMetadata, self).__init__(*args, **kwargs)
    def _get_db(self):
        return getattr(self._local, 'database', self._database)
    def _set_db(self, db):
        self._local.database = self._database = db
    database = property(_get_db, _set_db)
class BaseModel(Model):
    class Meta:
        # Instruct peewee to use our thread-safe metadata implementation.
        model_metadata_class = ThreadSafeDatabaseMetadata

In the source material, the Metadata class solves this problem by storing the database information in a thread-local. This thread-based solution does not work for CuneiFox which requires a storage that is specific to:

• Each client-side request with immediate return
• Each server-side initiated thread (using pseudo-sessions)
• Each client-side initiated thread (long processes, e.g. reports, multi-document printing)

Storage Format

To achieve specific storages as outlined in the previous section, CuneiFox employs a dictionary named 'db_dict' stored as an attribute of the main app object. The key-value pair within the dict is initiated for each request/thread and teardown (popped) once the request/thread finished.

The key for each items is a 2-tuple:

• For client-side requests: (str session_token, str request_id)
• For server-side threads: (str thread_name, None)
• For client-side threads: (str session_token, str thread_name)
• Fallback key: ('universal', None)

The reusable function 'establish_key' determines the case and key value for each database-related process.

The value of each dict item is yet another dict with Model ID as the key and the database object (the path to the file is embedded within) as the value.

app.db_dict = {tuple key0: {int model_id00: peewee.Database database_object00,
                            int model_id01: peewee.Database database_object01,
                            ...},
               tuple key1: {int model_id10: peewee.Database database_object10,
                            int model_id11: peewee.Database database_object11,
                            ...},
               ...}