Configuration API

Programmatic configuration of scinr.newton.

Core Functions

scinr.newton.config.configure

configure(
    llm: Any | None = None,
    repair_llm: Any | None = None,
    neo4j_uri: str | None = None,
    neo4j_user: str | None = None,
    neo4j_password: str | None = None,
    enabled_base_themes: list[ThemePath | str]
    | None = None,
    enabled_user_themes: list[str] | None = None,
    extra_models_paths: list[str | Path] | None = None,
    storage_backend: Literal["none", "mongodb", "custom"]
    | None = None,
    mongodb_uri: str | None = None,
    mongodb_database: str | None = None,
    mongodb_raw_files_collection: str | None = None,
    mongodb_pages_collection: str | None = None,
    mongodb_gridfs_bucket: str | None = None,
    custom_storage: tuple | None = None,
    extra_converters: dict[str, type] | None = None,
    mistral_api_key: str | None = None,
    mistral_ocr_safe_max_pages: int | None = None,
    mistral_ocr_safe_max_bytes: int | None = None,
    mistral_ocr_max_retries: int | None = None,
    mistral_ocr_retry_backoff_seconds: float | None = None,
    mistral_ocr_chunk_concurrency: int | None = None,
    mistral_ocr_error_strategy: Literal[
        "fail_fast", "best_effort"
    ]
    | None = None,
    prompt_caching_enabled: bool | None = None,
    full_docstring: bool | None = None,
    extraction_batch_size: int | None = None,
    llm_concurrency: int | None = None,
    neo4j_concurrency: int | None = None,
    neo4j_sync_concurrency: int | None = None,
    log_level: str = "INFO",
    prompt_family: PromptFamily
    | Literal["generic", "claude", "gpt_reasoning"]
    | None = None,
    normalization_enabled: bool | None = None,
    normalization_batch_size: int | None = None,
    normalization_llm: Any | None = None,
) -> ScinrConfig

Configure the scinr-ingest library.

Parameter resolution order: explicit argument > environment variable > default.

Parameters:

Name	Type	Description	Default
`llm`	`Any \| None`	LangChain BaseChatModel instance to use for all LLM calls.	`None`
`repair_llm`	`Any \| None`	LangChain BaseChatModel for the JSON repair loop. Falls back to `llm` if None.	`None`
`neo4j_uri`	`str \| None`	Neo4j connection URI. Env: `NEO4J_URI`. Default: `bolt://localhost:7687`.	`None`
`neo4j_user`	`str \| None`	Neo4j username. Env: `NEO4J_USER`.	`None`
`neo4j_password`	`str \| None`	Neo4j password. Env: `NEO4J_PASSWORD`.	`None`
`enabled_base_themes`	`list[ThemePath \| str] \| None`	Whitelist of built-in theme paths to activate (`ThemePath` values).	`None`
`enabled_user_themes`	`list[str] \| None`	Whitelist of user theme paths to activate.	`None`
`extra_models_paths`	`list[str \| Path] \| None`	Filesystem paths to scan for additional user-defined theme models.	`None`
`storage_backend`	`Literal['none', 'mongodb', 'custom'] \| None`	Storage type: `'none'` (default), `'mongodb'`, or `'custom'`.	`None`
`mongodb_uri`	`str \| None`	MongoDB connection URI. Env: `MONGODB_URI`.	`None`
`mongodb_database`	`str \| None`	MongoDB database name. Env: `MONGODB_DATABASE`.	`None`
`mongodb_raw_files_collection`	`str \| None`	Collection for raw file metadata.	`None`
`mongodb_pages_collection`	`str \| None`	Collection for converted pages.	`None`
`mongodb_gridfs_bucket`	`str \| None`	GridFS bucket name for binary files.	`None`
`custom_storage`	`tuple \| None`	Tuple `(RawFileRepository, PageRepository)` when `storage_backend='custom'`.	`None`
`extra_converters`	`dict[str, type] \| None`	Dict mapping file extensions to custom `BaseConverter` subclasses.	`None`
`mistral_api_key`	`str \| None`	Mistral API key for PDF OCR conversion.	`None`
`mistral_ocr_safe_max_pages`	`int \| None`	Máximo de páginas por chunk de PDF enviado a la API de Mistral OCR antes de dividirlo. Env: `MISTRAL_OCR_SAFE_MAX_PAGES`. Default: `900`.	`None`
`mistral_ocr_safe_max_bytes`	`int \| None`	Máximo de bytes por chunk de PDF (tamaño ya serializado) enviado a la API de Mistral OCR antes de dividirlo. Env: `MISTRAL_OCR_SAFE_MAX_BYTES`. Default: `45 * 1024 * 1024` (45 MiB).	`None`
`mistral_ocr_max_retries`	`int \| None`	Número máximo de intentos por chunk ante errores de red o HTTP reintentables. Env: `MISTRAL_OCR_MAX_RETRIES`. Default: `3`.	`None`
`mistral_ocr_retry_backoff_seconds`	`float \| None`	Base (en segundos) del backoff exponencial entre reintentos. Env: `MISTRAL_OCR_RETRY_BACKOFF_SECONDS`. Default: `2.0`.	`None`
`mistral_ocr_chunk_concurrency`	`int \| None`	Reservado para paralelismo futuro entre chunks de PDF; actualmente no se usa (procesamiento siempre secuencial). Env: `MISTRAL_OCR_CHUNK_CONCURRENCY`. Default: `1`.	`None`
`mistral_ocr_error_strategy`	`Literal['fail_fast', 'best_effort'] \| None`	Estrategia de manejo de errores al convertir PDFs divididos en chunks: `'fail_fast'` (default, aborta el documento completo si algún chunk falla) o `'best_effort'` (omite los chunks que fallen y continúa con el resto). Env: `MISTRAL_OCR_ERROR_STRATEGY`. Default: `'fail_fast'`. Nota: esta estrategia solo aplica a fallos de red/API por chunk (reintentos agotados, errores HTTP no reintentables) una vez que el PDF ya fue dividido en chunks. NO cubre el caso en que la propia partición inicial falla estructuralmente (`PdfSplitError`, una página individual excede `mistral_ocr_safe_max_bytes` incluso aislada) — en ese caso el documento aborta siempre, independientemente del valor de `mistral_ocr_error_strategy`.	`None`
`prompt_caching_enabled`	`bool \| None`	Enable prompt caching for supported LLM providers.	`None`
`full_docstring`	`bool \| None`	Use the full class docstring (True, default) or only its first non-empty line (False) when building the model catalog description shown to the LLM during annotation and stored as `CatalogModel.description` in Neo4j. Env: `FULL_DOCSTRING`. Default: `True`.	`None`
`extraction_batch_size`	`int \| None`	Pages per extraction chunk (default: `1`).	`None`
`llm_concurrency`	`int \| None`	Max concurrent LLM calls (semaphore size, default: `4`).	`None`
`neo4j_concurrency`	`int \| None`	Max concurrent Neo4j write sessions (default: `10`).	`None`
`neo4j_sync_concurrency`	`int \| None`	Max concurrent Stage 2 (sync ingestion) dispatches to asyncio.to_thread() (default: `8`).	`None`
`log_level`	`str`	Logging level string (`"DEBUG"`, `"INFO"`, `"WARNING"`, `"ERROR"`).	`'INFO'`
`prompt_family`	`PromptFamily \| Literal['generic', 'claude', 'gpt_reasoning'] \| None`	Prompt family to use (`"generic"`, `"claude"`, or `"gpt_reasoning"`).	`None`
`normalization_enabled`	`bool \| None`	Enable post-extraction normalization for tabular data.	`None`
`normalization_batch_size`	`int \| None`	Max normalization entries per LLM batch (default: `5`).	`None`
`normalization_llm`	`Any \| None`	Dedicated LLM model instance for tabular normalization.	`None`

Returns:

Type	Description
`ScinrConfig`	ScinrConfig singleton containing active library settings.

Raises:

Type	Description
`ConfigurationError`	If conflicting settings or invalid URIs are supplied.

scinr.newton.config.get_config

get_config() -> ScinrConfig

Return the current ScinrConfig singleton.

Raises:

Type	Description
`ConfigurationError`	If configure() has not been called yet.

scinr.newton.config.get_available_themes

get_available_themes() -> dict[str, list[str]]

Return all currently registered theme paths, grouped by origin.

Does not require :func:configure to have been called first — it initialises the registry with default settings if needed.

Returns:

Type	Description
`dict[str, list[str]]`	dict with two keys:
`dict[str, list[str]]`	`"builtin"` Theme paths that ship with the scinr-ingest package.
`dict[str, list[str]]`	`"user"` Theme paths loaded from `extra_models_paths`.

Examples::

from scinr.newton import get_available_themes

themes = get_available_themes()
print(themes["builtin"])   # ['default', 'equipment_qualification', ...]
print(themes["user"])      # ['my_custom_theme', ...]

Configuration Classes

scinr.newton.config.ScinrConfig `dataclass`

scinr.newton.config.ThemePath `module-attribute`

ThemePath = Literal[
    "default",
    "equipment_qualification",
    "pharmaceutical_quality",
    "structural_specs",
    "pharma_operations",
    "pharma_operations/product_master",
    "pharma_operations/commercial_sales",
    "pharma_operations/regulatory_portfolio",
    "pharma_operations/batch_manufacturing",
    "pharma_operations/serialization",
    "pharma_regulatory/qa",
    "pharma_regulatory/bpg",
    "pharma_regulatory/variation_guidelines",
]

Theme path identifiers for the built-in scinr-ingest theme library.

Use these values in enabled_base_themes to activate specific built-in themes. Extend with plain str values for user-defined themes added via extra_models_paths.

Examples::

configure(
    llm=...,
    enabled_base_themes=["pharmaceutical_quality", "pharma_operations/batch_manufacturing"],
)

scinr.newton.config.PromptFamily

Bases: str, Enum

Selects which prompt variant family to use for LLM calls.

GENERIC (default): Simplified, model-agnostic prompts. Work correctly across all LLM families (OpenAI, Kimi, GLM, Claude, Ollama, etc.). CLAUDE: Prompts optimized for Claude/Sonnet. Use XML-structured instructions, multi-step protocols, and internal checklists that leverage Claude's extended reasoning capabilities. GPT_REASONING: Prompts for OpenAI reasoning models (GPT-5.5, o3, o4-mini). Use Markdown section headers and goal-based language. No step-by-step protocols, no auto-checklists, no XML instruction wrappers. Use GENERIC for non-reasoning GPT models (GPT-4o, GPT-4.1, GPT-4.5).

To add support for a new model family in the future, add a new member here and create the corresponding prompt files (_newmodel.py) in each stage directory.