文献自动化入库流水线Literature ingestion pipeline
OWNER · 2025.10 – 至今 · 国家生物信息中心OWNER · Oct 2025 – present · China National Center for Bioinformation面向国家级基因组数据库（GSA / GSA Human / OMIX）。文献里提到某个数据集，到底是新提交、是复用、还是只引了一句——把这件事从人工判读变成可审计、可入库的结构化数据。 For the national genomics databases (GSA / GSA Human / OMIX). When a paper mentions a dataset, was it newly deposited, reused, or merely cited? This turns that judgement from manual reading into auditable, ingestible structured data.
1增量怎么发现Finding what's new
原来的做法是拿本地库里数十万条登录号，逐个去外部源反查，跑一次就是一整晚。把方向反过来：每天只拉 PMC 的 500 篇增量，用它们反查本地库。The old way took hundreds of thousands of accession numbers from our own database and looked each one up against external sources — one run took all night. Reverse the direction: pull only the ~500 new PMC papers each day, and look those up against our database.
单次耗时 14 小时以上 → 3 分钟，从月度批量改成每日可跑；审核员单次待审量由数百篇降到每日 10–20 篇。全量比对留作离线兜底，实测补 3% 覆盖，不占线上链路。One run went from 14+ hours to 3 minutes, and from a monthly batch to a daily job; a curator's queue dropped from hundreds of papers per run to 10–20 per day. The full comparison stays as an offline backstop — it adds about 3% coverage and never touches the live path.
2文献怎么进来How papers get in
邮件、PubMed、PMC、Google Scholar、PDF 五个来源统一收口，抽出登录号与它周围的上下文，再补齐 DOI 与期刊信息。Five sources — email, PubMed, PMC, Google Scholar and PDFs — funnel into one intake. It pulls out accession numbers together with the sentences around them, then fills in DOI and journal metadata.
抽的不只是编号，还有编号周围的句子——同一个登录号出现在方法学段落里还是引文列表里，结论完全不同。It isn't only the number that matters. The same accession in a methods paragraph and in a reference list mean entirely different things.
3引用怎么判Classifying the citation
三分类：deposited（自己提交）、reused（复用他人）、reference-only（只是提及）。规则引擎按 YAML 配置的关键词与上下文判一遍；自部署模型用句子相似度动态检索 Few-shot 例子，再判一遍。Three classes: deposited, reused, reference-only. A rule engine judges it once from keywords and context defined in YAML; a self-hosted model judges it again, retrieving few-shot examples by sentence similarity.
在 500 条人工标注的验证集上，三分类准确率约 90%。About 90% three-class accuracy on a 500-item hand-labelled validation set.
4怎么敢自动落库Why auto-ingest is safe
写入的是国家库，误判的代价远高于漏判。所以两路结论一致才自动落库，不一致的一律转人工复核。This writes into a national database, where a false positive costs far more than a miss. So a record is ingested automatically only when both paths agree; anything else goes to human review.
用覆盖率换自动化部分的零误判：自动落库覆盖 74.8%，其余进人工队列。Coverage traded for zero false positives on the automated share: 74.8% auto-ingested, the rest queued for a human.