Skip to content

Filters

Log filters for controlling which log records are processed. Filters can include or exclude messages based on keywords or scrub sensitive context data.

Quick Reference

Python
from pylogshield import KeywordFilter, ContextScrubber

# Include only logs containing specific keywords
include_filter = KeywordFilter(["error", "critical"], include=True)

# Exclude logs containing debug info
exclude_filter = KeywordFilter(["debug", "trace"], include=False)

# Remove cloud credentials from log context
scrubber = ContextScrubber()

KeywordFilter

Filter log records based on keywords in the message.

Include Mode (Default)

Only allow messages containing specified keywords:

Python
from pylogshield import get_logger, KeywordFilter
import logging

# Using get_logger parameter
logger = get_logger("my_app", log_filter=["error", "warning", "failed"])

logger.info("Application started")    # Not logged (no keyword match)
logger.info("Operation failed")       # Logged (contains "failed")
logger.error("An error occurred")     # Logged (contains "error")

Exclude Mode

Block messages containing specified keywords:

Python
from pylogshield import KeywordFilter
import logging

# Create filter manually
exclude_filter = KeywordFilter(
    keywords=["debug", "trace", "verbose"],
    include=False,  # Exclude mode
    case_insensitive=True
)

# Add to handler
handler = logging.StreamHandler()
handler.addFilter(exclude_filter)

Case Sensitivity

Python
# Case-insensitive matching (default)
filter1 = KeywordFilter(["ERROR"], case_insensitive=True)
# Matches: "error", "ERROR", "Error"

# Case-sensitive matching
filter2 = KeywordFilter(["ERROR"], case_insensitive=False)
# Matches only: "ERROR"

ContextScrubber

Automatically remove cloud provider credentials and tokens from log records.

Default Prefixes

The scrubber removes attributes starting with:

Prefix Description
AWS_ Amazon Web Services credentials
AZURE_ Microsoft Azure credentials
GCP_ Google Cloud Platform credentials
GOOGLE_ Google services credentials
TOKEN Various token attributes

Basic Usage

Python
from pylogshield import get_logger

# Enabled by default
logger = get_logger("my_app", enable_context_scrubber=True)

# Disable if not needed
logger = get_logger("my_app", enable_context_scrubber=False)

Custom Prefixes

Python
from pylogshield import ContextScrubber
import logging

# Create scrubber with custom prefixes
scrubber = ContextScrubber(
    forbidden_prefixes=("SECRET_", "PRIVATE_", "INTERNAL_")
)

# Add to handler
handler = logging.StreamHandler()
handler.addFilter(scrubber)

How It Works

The scrubber inspects log record attributes and removes any that match the forbidden prefixes:

Python
import logging

# If code accidentally adds cloud credentials to the record:
record.AWS_SECRET_ACCESS_KEY = "..."  # Will be scrubbed
record.AZURE_CLIENT_SECRET = "..."    # Will be scrubbed

# Normal attributes are preserved:
record.user_id = "12345"              # Kept

API Reference

filters

CLASS DESCRIPTION
KeywordFilter

Filter log records based on keywords in the message.
ContextScrubber

Strip potentially sensitive environment-derived attributes from LogRecord.

KeywordFilter(keywords: Iterable[str], *, include: bool = True, case_insensitive: bool = True, name: str = '')

Bases: Filter

Filter log records based on keywords in the message.

This filter can either include only messages containing specific keywords or exclude messages containing those keywords.

PARAMETER DESCRIPTION

keywords

Iterable of keywords to test against the log message.

TYPE: Iterable[str]

include

If True, only records containing any keyword are allowed. If False, records containing any keyword are excluded. Default is True.

TYPE: bool DEFAULT: True

case_insensitive

Whether to match keywords case-insensitively. Default is True.

TYPE: bool DEFAULT: True

name

The filter name. Default is empty string.

TYPE: str DEFAULT: ''

Examples:

Python Console Session
>>> # Include only logs containing "error" or "critical"
>>> filter = KeywordFilter(["error", "critical"], include=True)
>>> handler.addFilter(filter)
Python Console Session
>>> # Exclude logs containing "debug" or "trace"
>>> filter = KeywordFilter(["debug", "trace"], include=False)
Source code in src/pylogshield/filters.py
Python
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
def __init__(
    self,
    keywords: Iterable[str],
    *,
    include: bool = True,
    case_insensitive: bool = True,
    name: str = "",
) -> None:
    super().__init__(name)
    self.include = include
    self.case_insensitive = case_insensitive
    kws: List[str] = [k for k in keywords if k]
    self.keywords: Sequence[str] = tuple(
        (k.lower() if case_insensitive else k) for k in kws
    )

ContextScrubber(forbidden_prefixes: Optional[Tuple[str, ...]] = None, name: str = '')

Bases: Filter

Strip potentially sensitive environment-derived attributes from LogRecord.

This filter removes attributes from log records that might contain cloud provider credentials or tokens that were inadvertently added to the logging context.

The filter always returns True (allowing the record through) after scrubbing the sensitive attributes.

PARAMETER DESCRIPTION

forbidden_prefixes

Tuple of attribute name prefixes to scrub. Default is None, which uses the default cloud provider prefixes.

TYPE: tuple of str or None DEFAULT: None

name

The filter name. Default is empty string.

TYPE: str DEFAULT: ''

ATTRIBUTE DESCRIPTION
DEFAULT_FORBIDDEN_PREFIXES

Default prefixes to scrub: AWS_, AZURE_, GCP_, GOOGLE_, TOKEN.

TYPE: tuple of str

Notes

The following prefixes are scrubbed by default:

  • AWS_ - Amazon Web Services credentials
  • AZURE_ - Microsoft Azure credentials
  • GCP_ - Google Cloud Platform credentials
  • GOOGLE_ - Google services credentials
  • TOKEN - Various token attributes

Examples:

Python Console Session
>>> scrubber = ContextScrubber()
>>> handler.addFilter(scrubber)
Python Console Session
>>> # Custom prefixes
>>> scrubber = ContextScrubber(forbidden_prefixes=("SECRET_", "PRIVATE_"))
Source code in src/pylogshield/filters.py
Python
107
108
109
110
111
def __init__(
    self, forbidden_prefixes: Optional[Tuple[str, ...]] = None, name: str = ""
) -> None:
    super().__init__(name)
    self._forbidden_prefixes = forbidden_prefixes or self.DEFAULT_FORBIDDEN_PREFIXES

ContextFilter

For context propagation (injecting request IDs and other structured fields into log records), see the dedicated Context Propagation reference.