Configuration API

Programmatic configuration of scinr.newton.

Core Functions

scinr.newton.config.configure

configure(
    llm: Any | None = None,
    repair_llm: Any | None = None,
    neo4j_uri: str | None = None,
    neo4j_user: str | None = None,
    neo4j_password: str | None = None,
    neo4j_database: str | None = None,
    graph_backend: Literal["neo4j"] | None = None,
    enabled_base_themes: list[ThemePath | str]
    | None = None,
    enabled_user_themes: list[str] | None = None,
    extra_models_paths: list[str | Path] | None = None,
    storage_backend: Literal["none", "mongodb", "custom"]
    | None = None,
    mongodb_uri: str | None = None,
    mongodb_database: str | None = None,
    mongodb_raw_files_collection: str | None = None,
    mongodb_pages_collection: str | None = None,
    mongodb_gridfs_bucket: str | None = None,
    custom_storage: tuple | None = None,
    extra_converters: dict[str, type] | None = None,
    mistral_api_key: str | None = None,
    mistral_ocr_safe_max_pages: int | None = None,
    mistral_ocr_safe_max_bytes: int | None = None,
    mistral_ocr_max_retries: int | None = None,
    mistral_ocr_retry_backoff_seconds: float | None = None,
    mistral_ocr_chunk_concurrency: int | None = None,
    mistral_ocr_error_strategy: Literal[
        "fail_fast", "best_effort"
    ]
    | None = None,
    prompt_caching_enabled: bool | None = None,
    full_docstring: bool | None = None,
    extraction_batch_size: int | None = None,
    llm_concurrency: int | None = None,
    neo4j_concurrency: int | None = None,
    neo4j_sync_concurrency: int | None = None,
    consolidation_token_safety_margin: float | None = None,
    consolidation_max_output_tokens: int | None = None,
    consolidation_max_input_tokens: int | None = None,
    log_level: str = "INFO",
    prompt_family: PromptFamily
    | Literal["generic", "claude", "gpt_reasoning"]
    | None = None,
    normalization_enabled: bool | None = None,
    normalization_batch_size: int | None = None,
    normalization_llm: Any | None = None,
) -> ScinrConfig

Configure the scinr-ingest library.

Parameter resolution order: explicit argument > environment variable > default.

Parameters:

Name	Type	Description	Default
`llm`	`Any \| None`	LangChain BaseChatModel instance to use for all LLM calls.	`None`
`repair_llm`	`Any \| None`	LangChain BaseChatModel for the JSON repair loop. Falls back to `llm` if None.	`None`
`neo4j_uri`	`str \| None`	Neo4j connection URI. Env: `NEO4J_URI`. Default: `bolt://localhost:7687`.	`None`
`neo4j_user`	`str \| None`	Neo4j username. Env: `NEO4J_USER`. Required.	`None`
`neo4j_password`	`str \| None`	Neo4j password. Env: `NEO4J_PASSWORD`. Required.	`None`
`neo4j_database`	`str \| None`	Neo4j target database name. Env: `NEO4J_DATABASE`. Required.	`None`
`graph_backend`	`Literal['neo4j'] \| None`	Backend for the read-only graph-navigation API (`scinr.newton.navigation`). `'neo4j'` (default). Env: `GRAPH_BACKEND`. Validated like `storage_backend`; reserved for future engines.	`None`
`enabled_base_themes`	`list[ThemePath \| str] \| None`	Whitelist of built-in theme paths to activate (`ThemePath` values).	`None`
`enabled_user_themes`	`list[str] \| None`	Whitelist of user theme paths to activate.	`None`
`extra_models_paths`	`list[str \| Path] \| None`	Filesystem paths to scan for additional user-defined theme models.	`None`
`storage_backend`	`Literal['none', 'mongodb', 'custom'] \| None`	Storage type: `'none'` (default), `'mongodb'`, or `'custom'`.	`None`
`mongodb_uri`	`str \| None`	MongoDB connection URI. Env: `MONGODB_URI`.	`None`
`mongodb_database`	`str \| None`	MongoDB database name. Env: `MONGODB_DATABASE`.	`None`
`mongodb_raw_files_collection`	`str \| None`	Collection for raw file metadata.	`None`
`mongodb_pages_collection`	`str \| None`	Collection for converted pages.	`None`
`mongodb_gridfs_bucket`	`str \| None`	GridFS bucket name for binary files.	`None`
`custom_storage`	`tuple \| None`	Tuple `(RawFileRepository, PageRepository)` when `storage_backend='custom'`.	`None`
`extra_converters`	`dict[str, type] \| None`	Dict mapping file extensions to custom `BaseConverter` subclasses.	`None`
`mistral_api_key`	`str \| None`	Mistral API key for PDF OCR conversion.	`None`
`mistral_ocr_safe_max_pages`	`int \| None`	Máximo de páginas por chunk de PDF enviado a la API de Mistral OCR antes de dividirlo. Env: `MISTRAL_OCR_SAFE_MAX_PAGES`. Default: `900`.	`None`
`mistral_ocr_safe_max_bytes`	`int \| None`	Máximo de bytes por chunk de PDF (tamaño ya serializado) enviado a la API de Mistral OCR antes de dividirlo. Env: `MISTRAL_OCR_SAFE_MAX_BYTES`. Default: `45 * 1024 * 1024` (45 MiB).	`None`

mistral_ocr_max_retries: Número máximo de intentos por chunk ante errores de red o HTTP reintentables. Env: MISTRAL_OCR_MAX_RETRIES. Default: 15. mistral_ocr_retry_backoff_seconds: Base (en segundos) del backoff exponencial entre reintentos. Env: MISTRAL_OCR_RETRY_BACKOFF_SECONDS. Default: 2.0. mistral_ocr_chunk_concurrency: Reservado para paralelismo futuro entre chunks de PDF; actualmente no se usa (procesamiento siempre secuencial). Env: MISTRAL_OCR_CHUNK_CONCURRENCY. Default: 1. mistral_ocr_error_strategy: Estrategia de manejo de errores al convertir PDFs divididos en chunks: 'fail_fast' (default, aborta el documento completo si algún chunk falla) o 'best_effort' (omite los chunks que fallen y continúa con el resto). Env: MISTRAL_OCR_ERROR_STRATEGY. Default: 'fail_fast'. Nota: esta estrategia solo aplica a fallos de red/API por chunk (reintentos agotados, errores HTTP no reintentables) una vez que el PDF ya fue dividido en chunks. NO cubre el caso en que la propia partición inicial falla estructuralmente (PdfSplitError, una página individual excede mistral_ocr_safe_max_bytes incluso aislada) — en ese caso el documento aborta siempre, independientemente del valor de mistral_ocr_error_strategy. prompt_caching_enabled: Enable prompt caching for supported LLM providers. full_docstring: Use the full class docstring (True, default) or only its first non-empty line (False) when building the model catalog description shown to the LLM during annotation and stored as CatalogModel.description in Neo4j. Env: FULL_DOCSTRING. Default: True. extraction_batch_size: Pages per extraction chunk (default: 1). llm_concurrency: Max concurrent LLM calls (semaphore size, default: 4). neo4j_concurrency: Max concurrent Neo4j write sessions (default: 10). neo4j_sync_concurrency: Max concurrent Stage 2 (sync ingestion) dispatches to asyncio.to_thread() (default: 8). consolidation_token_safety_margin: Fraction of max_tokens used as the output-token ceiling for the Stage 1 fast_extraction consolidation LLM call, when consolidation_max_output_tokens is unset. Env: CONSOLIDATION_TOKEN_SAFETY_MARGIN. Default: 0.75. consolidation_max_output_tokens: Explicit output-token ceiling for the Stage 1 fast_extraction consolidation LLM call's decisions array. When None (default), derived as max_tokens * consolidation_token_safety_margin. Env: CONSOLIDATION_MAX_OUTPUT_TOKENS. consolidation_max_input_tokens: Input-token ceiling that governs the batch size of the Stage 1 fast_extraction consolidation's sliding-window algorithm (see structure_consolidation.consolidate_structure()) — each batch of consecutive chunks (plus its backward buffer from the immediately preceding batch) is kept under this ceiling. Default: 65536 (64k). Passing None explicitly to ScinrConfig directly (bypassing configure()) skips the ceiling check entirely as a defensive fallback — not the normal path. Env: CONSOLIDATION_MAX_INPUT_TOKENS. log_level: Logging level string ("DEBUG", "INFO", "WARNING", "ERROR"). prompt_family: Prompt family to use ("generic", "claude", or "gpt_reasoning"). normalization_enabled: Enable post-extraction normalization for tabular data. normalization_batch_size: Max normalization entries per LLM batch (default: 5). normalization_llm: Dedicated LLM model instance for tabular normalization.

Returns:

Type	Description
`ScinrConfig`	ScinrConfig singleton containing active library settings.

Raises:

Type	Description
`ConfigurationError`	If conflicting settings or invalid URIs are supplied.

scinr.newton.config.get_config

get_config() -> ScinrConfig

Return the current ScinrConfig singleton.

Raises:

Type	Description
`ConfigurationError`	If configure() has not been called yet.

scinr.newton.config.get_available_themes

get_available_themes() -> dict[str, list[str]]

Return all currently registered theme paths, grouped by origin.

Does not require :func:configure to have been called first — it initialises the registry with default settings if needed.

Returns:

Type	Description
`dict[str, list[str]]`	dict with two keys:
`dict[str, list[str]]`	`"builtin"` Theme paths that ship with the scinr-ingest package.
`dict[str, list[str]]`	`"user"` Theme paths loaded from `extra_models_paths`.

Examples::

from scinr.newton import get_available_themes

themes = get_available_themes()
print(themes["builtin"])   # ['default', 'equipment_qualification', ...]
print(themes["user"])      # ['my_custom_theme', ...]

Configuration Classes

scinr.newton.config.ScinrConfig `dataclass`

scinr.newton.config.ThemePath `module-attribute`

ThemePath = Literal[
    "default",
    "equipment_qualification",
    "pharmaceutical_quality",
    "structural_specs",
    "pharma_operations",
    "pharma_operations/product_master",
    "pharma_operations/commercial_sales",
    "pharma_operations/regulatory_portfolio",
    "pharma_operations/batch_manufacturing",
    "pharma_operations/serialization",
    "pharma_regulatory/qa",
    "pharma_regulatory/bpg",
    "pharma_regulatory/variation_guidelines",
]

Theme path identifiers for the built-in scinr-ingest theme library.

Use these values in enabled_base_themes to activate specific built-in themes. Extend with plain str values for user-defined themes added via extra_models_paths.

Examples::

configure(
    llm=...,
    enabled_base_themes=["pharmaceutical_quality", "pharma_operations/batch_manufacturing"],
)

scinr.newton.config.PromptFamily

Bases: str, Enum

Selects which prompt variant family to use for LLM calls.

GENERIC (default): Simplified, model-agnostic prompts. Work correctly across all LLM families (OpenAI, Kimi, GLM, Claude, Ollama, etc.). CLAUDE: Prompts optimized for Claude/Sonnet. Use XML-structured instructions, multi-step protocols, and internal checklists that leverage Claude's extended reasoning capabilities. GPT_REASONING: Prompts for OpenAI reasoning models (GPT-5.5, o3, o4-mini). Use Markdown section headers and goal-based language. No step-by-step protocols, no auto-checklists, no XML instruction wrappers. Use GENERIC for non-reasoning GPT models (GPT-4o, GPT-4.1, GPT-4.5).

To add support for a new model family in the future, add a new member here and create the corresponding prompt files (_newmodel.py) in each stage directory.