Tokenizer

Exceptions for Tokenizer.

TokenizerError

Bases: FedRAGError

Base evals error for all tokenizer-related exceptions.

Source code in src/fed_rag/exceptions/tokenizer.py

class TokenizerError(FedRAGError):
    """Base evals error for all tokenizer-related exceptions."""

    pass

TokenizerWarning

Bases: FedRAGWarning

Base inspector warning for all tokenizer-related warnings.

Source code in src/fed_rag/exceptions/tokenizer.py

class TokenizerWarning(FedRAGWarning):
    """Base inspector warning for all tokenizer-related warnings."""

    pass