Omitting exc_info from Console Logs While Using Coloredlogs in Python

When you’re working with Python’s logging library, dealing with stack traces (exception info, known as exc_info) can sometimes clutter your console output. While detailed logs help debug issues, seeing lengthy tracebacks printed directly in your console can be distracting, especially when you already log those complete details into a separate log file. This balance between file-based detailed logs and cleaner console outputs is easier said than done, particularly when using the popular coloredlogs library.

A popular Stack Overflow question addresses this scenario for Python’s standard logging module. However, addressing this issue differs slightly when using coloredlogs. Let’s explore why that is and how you can neatly omit exc_info from your console when using coloredlogs alongside Python logging.

The Issue at Hand

Ideally, you want logs reflecting exceptions, including their stack traces, preserved in your log file. But you prefer a cleaner console where you can quickly grasp essential log messages without scrolling through volumes of stack traces. Python’s default logging framework provides formatter subclassing—a powerful choice to control what gets output to console versus file—but coloredlogs simplifies formatting in a different way that requires an adjusted approach.

Understanding coloredlogs

The coloredlogs library enhances Python’s regular logging mechanism by automatically formatting console logs with clear, colored outputs. Coloredlogs delivers visual clarity, displaying log levels like DEBUG, INFO, WARNING, or ERROR in distinct colors—making it straightforward to recognize critical problems at a glance.

Developers often prefer coloredlogs for its simplicity. It automates many formatting tasks you would manually configure with Python’s default logging, providing cleaner readable console outputs without extensive configuration.

How Does coloredlogs Compare to Standard Logging?

In standard Python logging, controlling the output across handlers (such as files, consoles, or network destinations) is achieved through Formatter objects, which you can subclass and customize extensively. This flexibility grants you considerable fine-tuning.

In contrast, coloredlogs internally manages its own formatters. It automatically assigns log messages colors, formats timestamps, and organizes information. You usually don’t directly subclass formatters with coloredlogs because that defeats part of its appeal—simplicity. However, the downside is reduced flexibility in selectively omitting exception information from console outputs.

Challenges with Hiding exc_info in coloredlogs

With coloredlogs, unlike with pure logging setups, directly subclassing a Formatter and tweaking the exc_info output is not straightforward. Coloredlogs doesn’t expose formatter subclassing explicitly; it relies upon predefined patterns and color codes to manage how your console logs look.

This limitation means you’ll need creative alternatives to handle exceptions neatly without printing stack traces every time an error occurs on-screen.

Potential Solutions and Workarounds

Given these limitations, you might consider several practical approaches:

Using multiple handlers: standard logging handlers for the file (rich with exc_info), and coloredlogs for the console output (without exc_info).
Adjusting logging levels explicitly: lower-level logs going to files and higher-level logs directed to coloredlogs’ console.
Utilizing Python filters to selectively strip out exception information before the logs are output to the console.

Among these approaches, leveraging logging filters is typically straightforward and effective when using coloredlogs. Let’s see how to implement this practically.

Implementation Steps: Omit exc_info from Console Output with coloredlogs

To set up clear console outputs without losing crucial stack trace details in your log files, follow these implementation steps:

Step 1: Configure Regular File Handler (Includes exc_info)

First, set up standard logging configured to log detailed outputs including stack traces into your log files:

import logging

# Create basic configuration for logging
logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)

# File handler will include exc_info
file_handler = logging.FileHandler('app.log')
file_formatter = logging.Formatter('%(asctime)s [%(levelname)s]: %(message)s\n%(exc_info)s')
file_handler.setFormatter(file_formatter)
logger.addHandler(file_handler)

Step 2: Configure coloredlogs Console Handler (Exclude exc_info)

Next, configure coloredlogs for console output without including exception stack traces. Here, you introduce a logging filter that omits exc_info from appearing in the coloredlogs output.

import coloredlogs

# Define custom filter that removes exc_info from console logs
class NoExceptionFilter(logging.Filter):
    def filter(self, record):
        record.exc_info = None  # Remove stack traces from console logs
        return True

console_fmt = "%(asctime)s %(name)s[%(process)d] %(levelname)s %(message)s"

coloredlogs.install(level='INFO', logger=logger, fmt=console_fmt)

# Add our custom filter to each console handler coloredlogs created
for handler in logger.handlers:
    if isinstance(handler, logging.StreamHandler):
        handler.addFilter(NoExceptionFilter())

In the snippet above, NoExceptionFilter ensures that exception details aren’t shown in your console output, keeping logs concise.

Testing and Validation

Let’s test our setup to validate that it works correctly. Here’s a snippet you can run:

try:
    result = 10 / 0
except ZeroDivisionError:
    logger.error("A divide-by-zero error occurred", exc_info=True)

After running the snippet:

Console Output will show clearly formatted, colored ERROR without lengthy tracebacks.
Log file (“app.log”) will include detailed exception tracebacks for debugging later.

Logging Best Practices with coloredlogs

To ensure your log configuration is clear and maintainable:

Clearly separate log handlers for different output streams (console vs file).
Use logging levels appropriately—DEBUG messages for general debugging, INFO for general operational updates, and higher levels like ERROR and CRITICAL sparingly to highlight important issues.
Regularly review logs, ensuring essential information isn’t buried beneath unnecessary verbosity.
Avoid logging sensitive user data or secrets directly to logs.

Optimizing how and what you log greatly enhances both debugging and readability—a critical part of Python application maintenance.

Real-world Use Cases

Consider these common scenarios where omitting exc_info from console logs could streamline your development workflow:

API development: Detailed exceptions cluttering console outputs overshadow key command-line feedback.
CI/CD pipelines: Limiting console logs allows quicker identification of deployment issues.
Large-scale backend systems: Reducing console noise aids faster comprehension of operational statuses without compromising full logs.

In each scenario, improving console log readability while retaining complete stack traces elsewhere adds efficiency and clarity, ultimately improving the developer experience.

Summing up Logging with coloredlogs

Balancing detailed logs and console readability is a crucial aspect of professional Python development. While coloredlogs makes logs more readable and pleasant, it introduces challenges to selectively hide exceptions like exc_info. Using logging filters alongside coloredlogs neatly overcomes this limitation, providing clearer console outputs without sacrificing detailed, traceable error logs stored in files.

Have you customized your logging setup in other creative ways? Let us know your insights or challenges with logging strategies below!