Python rich logging

Source: Notion | Last edited: 2024-10-23 | ID: 1282d2dc-3ef...

---

Standard Import for Logging Setup

This import statement is essential for setting up logging in our Python scripts. It imports the necessary components from our custom logger setup module.

The logger setup can be customized in several ways:

• Logger Level: Currently set to “INFO” by default. This can be modified to: • “DEBUG” for more verbose output • “WARNING”, “ERROR”, or “CRITICAL” for less verbose output Example: logger = get_logger(__name__, "DEBUG", show_path=False, rich_tracebacks=True)
• show_path: Currently set to False. If set to True, each log message will include the exact file and line number where the log was generated. Example: logger = get_logger(__name__, "INFO", show_path=True, rich_tracebacks=True)
• rich_tracebacks: Currently set to True. This enables detailed, formatted tracebacks for exceptions. Set to False to disable this feature. Example: logger = get_logger(__name__, "INFO", show_path=False, rich_tracebacks=False) Note: The show_path=True option is particularly useful for debugging, as it allows you to quickly identify the source of each log message in your code.

from ml_feature_set.utils.logger_setup import Console, get_logger, Table, Text, print; logger = get_logger(__name__, "INFO", show_path=False, rich_tracebacks=True)

Logger Setup Module

This module provides a robust and flexible logging setup for our Python scripts. It utilizes the Rich library to enhance log output with color, formatting, and rich tracebacks. The module includes functions for creating customized loggers, handling warnings, and setting up consistent logging across different parts of our application.

Key features of this logger setup include:

• Customizable log levels
• Rich console output with colored formatting
• Detailed tracebacks for better error diagnosis
• Warning interception and logging
• Consistent logging interface across modules

import logging, os
from rich.logging import RichHandler
from rich.console import Console
from rich.table import Table
from rich.text import Text
from rich.panel import Panel
from rich.prompt import Prompt, IntPrompt, FloatPrompt
from rich import print
from rich.traceback import install
import warnings
import traceback
install(show_locals=True)

console = Console()
logging.basicConfig(
    level=os.environ.get("LOG_LEVEL", "INFO").upper(),
    format="%(message)s",
    datefmt="[%X]",
    handlers=[
        RichHandler(
            console=console,
            rich_tracebacks=True,
            markup=True,
            show_path=True,
            omit_repeated_times=True,
            log_time_format="[%X]"
        )
    ],
    force=True
)

def get_logger(name: str, level: str = None, show_path: bool = None, rich_tracebacks: bool = True) -> logging.Logger:
    logger = logging.getLogger(name)
    if level:
        logger.setLevel(level.upper())

    # Always use RichHandler with rich_tracebacks enabled
    handler = RichHandler(
        console=console,
        rich_tracebacks=True,
        markup=True,
        show_path=show_path if show_path is not None else True,
        omit_repeated_times=True,
        log_time_format="[%X]"
    )
    logger.handlers = [handler]  # Replace any existing handlers
    logger.propagate = False

    # Add some debug logging
    logger.debug(f"Logger '{name}' configured with level={logger.level}, show_path={show_path}, rich_tracebacks=True")

    return logger

logger = get_logger(__name__)

def setup_warning_logging():
    def custom_warning_handler(message, category, filename, lineno, file=None, line=None):
        tb = ''.join(traceback.format_stack(limit=5))
        logger.warning(f"{category.__name__}: {message} at {filename}:{lineno}\nStack trace:\n{tb}")
    warnings.showwarning = custom_warning_handler

# Test the logger setup
if __name__ == "__main__":
    test_logger = get_logger("test_logger", level="DEBUG")
    test_logger.debug("This is a debug message")
    test_logger.info("This is an info message")
    test_logger.warning("This is a warning message")
    test_logger.error("This is an error message")
    try:
        raise ValueError("This is a test exception")
    except Exception as e:
        test_logger.exception("An exception occurred:")