Cache Strategy and Customization

The application employs a hierarchical configuration system to manage cache strategies across different environments. By leveraging Python's dataclasses and inheritance, the project balances the need for high performance in production with the flexibility and low resource footprint required for local development.

Configuration Hierarchy

The cache strategy is defined within app/config.py through a series of configuration classes. The system uses a "base-and-override" pattern where common defaults are established in a base class and then specialized for specific deployment scenarios.

Base Strategy

The BaseConfig class serves as the foundation for all environments. It provides a default cache configuration via the get_cache_config() method, which returns a dictionary containing standard settings:

@dataclass
class BaseConfig:
    # ... other settings ...

    def get_cache_config(self) -> Dict[str, Any]:
        """Return cache settings for this environment."""
        return _build_cache_config()

By default, _build_cache_config() (an internal helper) initializes the cache with a Time-To-Live (TTL) of 300 seconds and a maximum size of 1024 entries.

Development Tuning

In DevelopmentConfig, the strategy shifts toward visibility and rapid iteration. The cache limits are significantly reduced to ensure that developers see changes quickly and that the application remains lightweight on local machines:

@dataclass
class DevelopmentConfig(BaseConfig):
    DEBUG: bool = True
    PAGE_SIZE: int = 10

    def get_cache_config(self) -> Dict[str, Any]:
        return _build_cache_config(ttl=30, max_size=128)

With a TTL of only 30 seconds and a capacity of 128 entries, the development environment prioritizes data freshness over the performance gains of long-term caching.

Production Hardening

The ProductionConfig class is optimized for high-concurrency environments where performance is critical. It increases both the duration and the capacity of the cache to minimize database load:

@dataclass
class ProductionConfig(BaseConfig):
    # ...
    def get_cache_config(self) -> Dict[str, Any]:
        return _build_cache_config(ttl=600, max_size=4096)

In production, the cache persists for 10 minutes (600s) and can hold up to 4096 entries, reflecting a design choice that favors memory consumption to achieve lower latency.

The Cache Configuration Helper

To maintain consistency in the structure of cache settings, the project uses an internal helper function, _build_cache_config. This ensures that regardless of the environment, the service layer receives a dictionary with a standardized schema:

def _build_cache_config(ttl: int = 300, max_size: int = 1024) -> Dict[str, Any]:
    """Build cache configuration dict.
    
    This is an internal helper — callers should use the config classes instead.
    """
    return {"ttl_seconds": ttl, "max_entries": max_size, "eviction": "lru"}

This helper enforces the use of the Least Recently Used (LRU) eviction policy across all environments, centralizing the logic for how cache metadata is structured.

Tradeoffs and Implementation Constraints

While the configuration classes provide a robust framework for environment-specific tuning, there are notable tradeoffs and constraints in the current implementation:

Configuration vs. Service Implementation

A significant constraint in the current codebase is the disconnect between the Config classes and the actual service initialization. In app/services/bookmark_service.py, the BookmarkService initializes its internal LRUCache with a hardcoded value, effectively bypassing the environment-specific settings defined in app/config.py:

# From app/services/bookmark_service.py
def _init_services(self) -> None:
    """Bootstrap repository, cache, and search index."""
    self._repo = BookmarkRepository()
    self._cache: LRUCache[Bookmark] = LRUCache(max_size=256) # Hardcoded size
    self._search = SearchIndex(self._repo)

This design choice means that while the ProductionConfig specifies a capacity of 4096, the BookmarkService will only ever utilize 256 entries unless the service's _init_services method is manually updated to consume the configuration object.

Memory vs. Performance

The project explicitly trades memory for performance in production. By quadrupling the max_size from the base default (1024 to 4096), the system acknowledges that production servers typically have more RAM available than development environments and that reducing database I/O is the primary optimization goal.

Strategy Summary

Environment	TTL (Seconds)	Max Entries	Primary Goal
Base	300	1024	Balanced defaults
Development	30	128	Data freshness & low footprint
Production	600	4096	High performance & reduced I/O
Testing	300 (Base)	1024 (Base)	Consistency with defaults