Skip to content

Explore papis-scibrain: a papis plugin syncing papis libraries into sci-brain KBs #36

Description

@chenpeizhi

Background

sci-brain builds knowledge bases (.knowledge/ with rendered frontmattered Markdown, INDEX.md, references.bib, gitignored .raw//.figures/) via its own fetch/render pipeline. papis users already curate their literature in papis libraries (folder-per-document info.yaml + files) and get its ecosystem: publisher downloaders, papis doctor, papis-zotero, papis-ask. We rejected papis as sci-brain's storage backend (#30 records why). The inverse integration remains open: a papis-side plugin so papis-native users can produce/refresh sci-brain KBs from their libraries — papis owns acquisition and curation; sci-brain owns the agent-facing contract. Motivation and full papis analysis: #30.

Objective

Explore and, if the maintainers approve the repo, implement papis-scibrain: a pip-installable papis plugin providing one-way export/sync from a papis library into a sci-brain KB.

Interface (Input → Output)

In: papis -l <library> scibrain sync <kb-path> [--query <papis query>] — a papis library (or query subset) and a target KB directory.
Out: for each selected papis document: a rendered <id>_<slug>.md with sci-brain frontmatter (ids mapped from papis doi/eprint/isbn fields, full_text per available files), a references.bib entry using the document's papis ref as cite key, PDFs/sources copied into .raw/; INDEX.md regenerated. Re-runs are idempotent (unchanged docs produce no diff). One-way only: the plugin never writes back into the papis library.

Technical recommendations (non-binding)

Separate repo (papis-ecosystem convention: entry-point plugin, own PyPI package) — QuantumBFS/papis-scibrain or personal, maintainers' call. Vendor or import sci-brain's render/index helpers rather than reimplementing the contract; treat sci-brain's SKILL.md-documented KB layout as the normative spec. Decision points to settle in this issue before implementation: repo home, whether PDF→markdown rendering happens plugin-side (pymupdf4llm dep) or is delegated to a sci-brain checkout, and Zotero flow (papis-zotero → papis → this plugin could replace researchstyle's Zotero path).

Verification

Once implemented (staged roadmap — commands are concrete, runnability follows implementation):

  1. Scratch papis library with two documents added by papis add --from arxiv 1806.08734 --batch and papis add --from doi 10.22331/q-2018-08-06-79 --batch; run papis -l <lib> scibrain sync $KB → KB contains two frontmattered .md files, two bib entries keyed by the papis refs, and an INDEX.md listing both — and kb_doctor.py --kb $KB (companion issue) exits 0, which proves contract conformance by the standing linter rather than by eyeball.
  2. Idempotency: immediate re-run → git status --porcelain over the KB is empty.
  3. Negative control: a papis document with no files and no identifiers → the sync emits a WARN and renders a stub entry rather than crashing or silently skipping; the summary line counts it.

Assumes

  • Repo home and ownership (QuantumBFS vs personal) — settled in this issue's discussion before code.
  • kb_doctor (companion issue) lands first; it is the conformance oracle for verification step 1.

Dependencies

Part of #30. Verification depends on the kb_doctor child issue.

Out of scope

Two-way sync (KB→papis); making sci-brain's own pipeline depend on papis (see the optional-downloaders child instead); replacing sci-brain's fetch pipeline for non-papis users.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions