Skip to content

Deletion API

scinr.newton.ingest.deletion.delete_document async 

delete_document(
    path: str | None = None,
    version: int | None = None,
    *,
    tenant_id: str | None,
    created_by_user_id: str | Sequence[str] | None = None,
    job_id: str | Sequence[str] | None = None,
) -> DeletionResult

Completely delete Document node(s), their entire cascade, and orphans.

Unlike delete_document_content() (which only wipes content for in-place re-ingestion and keeps the :Document node), this permanently removes every :Document node matching the selector below along with:

  • Every descendant reached via IS_COMPOSED_OF* downwards from the matched nodes: deleting a folder deletes its whole subtree, while deleting a leaf by path leaves its parent folder and siblings untouched (they only go too when they match the selector themselves, e.g. the same job_id).
  • All :StructureNode descendants (HAS_STRUCTURE / HAS_CHILD), their :InfoUnit, :ModelDecision, :ProposedModel, :ProposedField, and :ExtractionResult children.

Target selection

tenant_id is mandatory (keyword-only, no default — omitting it is a TypeError) and always applied: the deletion never leaves that tenant's documents. tenant_id=None explicitly means "public documents" (stored as "__public__", and tenant_id="__public__" means the same); there is no way to delete across tenants. Because every tenant has its own folder documents, the IS_COMPOSED_OF* cascade below stays within the tenant too.

On top of the tenant, exactly one of path or job_id must be provided (ValueError otherwise):

  • path: delete the tenant's Document at that path (all versions, or only version when given).
  • job_id: delete every Document of the tenant whose job_id property is any of these values (one str or several) — across all paths and versions of those ingestion runs.

created_by_user_id (one str or several, matched with IN), when given, is an additional AND filter applied on top of either selector. Left as None it means "do not filter on this property" (it does not mean "the property must be null"). version is also accepted as an extra filter in job_id mode.

Who may delete a public document is an authorization decision for the calling API layer; this function only enforces the scope.

Before any Neo4j deletion happens, this also deletes the documental storage records (raw binary + converted Markdown pages) for every non-empty raw_file_id found on the target Document(s) and their IS_COMPOSED_OF* descendants, via the configured storage backend (storage/factory.py::get_storage()). This step is fail-fast: if deleting storage for any raw_file_id raises an unexpected exception, it propagates immediately and the Neo4j cascade delete is never run.

The cascade runs in bounded transactions (ingest/_cascade.py), not in a single one: one transaction holding the subtree of hundreds of documents overruns the server's transaction memory. It is therefore not atomic — the :Document nodes are marked deletion_pending first and deleted last, so if it fails half-way, calling delete_document() again with the same selector deletes what is left. It also removes the :ComplementaryMatch and :SupplementaryField nodes of the annotations.

Before the :Document nodes go, the :Entity/:ModelInstance and :LabeledEntity nodes of the same tenant this deletion leaves orphaned are deleted too (ingest/_gc.py). Only what hung from the deleted :ExtractionResult nodes is checked, so the cost follows the size of the deletion, not of the tenant; orphans left by anything else stay until :func:collect_orphans is run. The exception is a call that finishes an interrupted delete or freeze: the candidates of the interrupted run are lost, so it sweeps the whole tenant (up to :data:GC_MAX_PASSES iterations per pass), as collect_orphans() does.

Opens and closes its own Neo4j driver — does not require the caller to manage one. The Neo4j-specific work (existence check, raw_file_id lookup, cascade delete, GC passes) uses the existing synchronous Neo4j driver under the hood, dispatched via asyncio.to_thread(); the storage deletion calls are awaited directly since storage repositories (Motor-backed) are natively async.

Returns

DeletionResult Structured counts of everything deleted. If no Document matches the selector, found is False and all counters are 0 (no storage, delete, or GC queries are executed in that case).

Raises

TypeError If tenant_id is not passed. ValueError If neither or both of path and job_id are provided, or tenant_id is empty, or job_id / created_by_user_id is an empty list.

scinr.newton.ingest.deletion.collect_orphans async 

collect_orphans(
    *, tenant_id: str | None
) -> OrphanCollectionResult

Delete every orphaned :Entity, :ModelInstance and :LabeledEntity node of one tenant.

delete_document() and freeze_document() only collect the orphans they cause. This maintenance call collects the rest: the nodes left behind by the write paths that delete :ExtractionResult nodes without collecting (re-ingestion with update_mode=True, re-extraction of a node), and by an interrupted delete or freeze that was never run again.

An :Entity / :ModelInstance is an orphan when no :ExtractionResult reaches it within 7 hops; a :LabeledEntity, when nothing points at it. Every such node of the tenant is checked, so the cost grows with the tenant, not with what there is to collect: run it from time to time, not after every operation.

Do not run it while the same tenant is being ingested: a node written just before the :ExtractionResult that will point at it is, for that instant, an orphan.

tenant_id is mandatory (keyword-only, no default); None and "__public__" both mean the public documents. Opens and closes its own Neo4j driver.

Returns

OrphanCollectionResult How many nodes were deleted, and in how many iterations.

Raises

TypeError If tenant_id is not passed. ValueError If tenant_id is empty.