文献自动化入库流水线Literature ingestion pipeline
OWNER · 2025.10 – 至今 · 国家生物信息中心OWNER · Oct 2025 – present · China National Center for Bioinformation面向国家级基因组数据库（GSA / GSA Human / OMIX）。文献里提到某个数据集，到底是新提交、是复用、还是只引了一句——把这件事从人工判读变成可审计、可入库的结构化数据。 For the national genomics databases (GSA / GSA Human / OMIX). When a paper mentions a dataset, was it newly deposited, reused, or merely cited? This turns that judgement from manual reading into auditable, ingestible structured data.
1增量怎么发现Finding what's new
原来是拿本地库数十万条登录号逐个反查外部源，跑一次一整晚。把方向反过来：每天只拉 PMC 的 500 篇增量，用它们反查本地库。The old way looked up hundreds of thousands of our own accession numbers against external sources — one run took all night. Reverse it: pull the ~500 new PMC papers each day and look those up against our database.
单次 14 小时以上 → 3 分钟，月度批量变每日可跑；审核员单次待审从数百篇降到每日 10–20 篇。全量比对留作离线兜底，实测补 3% 覆盖。One run went from 14+ hours to 3 minutes, a monthly batch to a daily job; a curator's queue from hundreds of papers to 10–20 per day. The full comparison stays an offline backstop, adding about 3% coverage.
2文献怎么进来How papers get in
邮件、PubMed、PMC、Google Scholar、PDF 五源统一收口，抽出登录号和它周围的句子，再补齐 DOI 与期刊。Five sources — email, PubMed, PMC, Google Scholar, PDFs — funnel into one intake: accession numbers plus the sentences around them, then DOI and journal metadata.
关键是那些句子——同一个登录号出现在方法学段落还是引文列表里，结论完全不同。The sentences are the point: the same accession in a methods paragraph and in a reference list mean entirely different things.
3引用怎么判Classifying the citation
三分类：deposited（自己提交）、reused（复用他人）、reference-only（只是提及）。规则引擎按 YAML 的关键词与上下文判一遍，自部署模型用句子相似度动态检索 Few-shot 例子再判一遍。Three classes: deposited, reused, reference-only. A rule engine judges from keywords and context defined in YAML; a self-hosted model judges again, retrieving few-shot examples by sentence similarity.
500 条人工标注验证集上，准确率约 90%。About 90% three-class accuracy on a 500-item hand-labelled validation set.
4怎么敢自动落库Why auto-ingest is safe
写入的是国家库，误判代价远高于漏判。所以两路一致才自动落库，不一致一律转人工。This writes into a national database, where a false positive costs far more than a miss. Only when both paths agree does a record land automatically; anything else goes to a human.
用覆盖率换自动化部分的零误判：自动落库覆盖 74.8%，其余进人工队列。Coverage traded for zero false positives on the automated share: 74.8% auto-ingested, the rest queued for a human.