Recipe: fine_tune_graph · Theory: representation learning on graphs — node2vec / contrastive (Stanković et al. 2020, Part III; Hamilton 2017); citation- graph contrastive supervision (SPECTER, Cohan et al. 2020); nearest-neighbor-of- own-representations (NNCLR, Dwibedi et al. 2021) · Rail: measurement (declared-edge gain, and two negative controls).
Tier 02 propagated fixed embeddings; tier 03 learns new ones. fine_tune_graph samples random walks over a graph and fine-tunes the encoder contrastively: papers that co-occur on a walk are pulled together, others pushed apart. The graph supervises the metric — but which graph, and how well the objective is trained, both matter to how much it helps; this chapter measures both axes honestly.
The keystone ran this on the GPU server with edge_provenance="declared", over the declared citation graph:
db.fine_tune_graph(
node_source=papers, id_column="paper_id", text_column="abstract",
edge_source=cite_edges, src_column="src", dst_column="dst",
base_model="answerdotai/ModernBERT-base", edge_provenance="declared",
epochs=15,
)and committed the resulting checkpoint id. The requested epochs is honored exactly by the engine. Graph-supervised contrastive fine-tunes in the literature (SPECTER trains for tens of epochs) converge slowly — a single epoch under-trains the objective and this same recipe turns in a small negative gain over the base encoder on this subset; 15 epochs is the value-demonstrating point on the curve (still rising toward the field’s ~20-epoch norm). This chapter reads the cache on CPU.
from jammi_cookbook import contracts, rails
ft = contracts.load_artifact("arxiv.ft_model")
print("fine-tuned model id:", ft["model_id"])
print("edge provenance: ", ft["edge_provenance"])
print("base model: ", ft["base_model"])fine-tuned model id: jammi:fine-tuned:f41e7b50-b737-4b82-b4a5-a3455f3079cd
edge provenance: declared
base model: answerdotai/ModernBERT-baseThe honest test of graph supervision is whether the fine-tuned encoder retrieves same-subject papers better than the base encoder. The keystone measured recall@10 for the declared-edge fine-tune and froze it; the gain over the tier-01 base is the learned signal.
base_recall = contracts.golden("arxiv.tier01.recall_at_10").value
ft_recall = contracts.golden("arxiv.tier03.recall_at_10").value
gain = contracts.golden("arxiv.tier03.recall_gain_vs_base").value
print(f"recall@10 base encoder: {base_recall:.3f}")
print(f"recall@10 declared-edge FT: {ft_recall:.3f}")
print(f"Δ (graph-supervised gain): {gain:+.3f}")recall@10 base encoder: 0.538
recall@10 declared-edge FT: 0.594
Δ (graph-supervised gain): +0.056edge_provenance distinguishes an edge drawn independently of the base embedding metric ("declared" — who-cited-whom, a signal the embeddings cannot reconstruct) from one drawn by it ("similarity" — the tier-01 k-NN-of-own- embeddings graph). The keystone ran the identical recipe over three graphs at a matched, cheaper epoch budget, so any recall difference is attributable to the graph alone:
edge_provenance="similarity"): a smaller but real, positive gain. This is not a purely circular no-op: training against a node’s own nearest neighbors pulls same-subject papers closer through the same homophily mechanism nearest-neighbor contrastive learning exploits on other modalities (NNCLR, Dwibedi et al. 2021) — the self-similarity graph carries partial signal, not none, echoing SciNCL’s citation-embedding k-NN augmentation (Ostendorff et al. 2022).edge_provenance="declared" since it is external to the metric, just uninformative): harmful — a negative gain. Training against structureless pairs degrades the metric rather than merely failing to improve it.sim_recall = contracts.golden("arxiv.tier03.control_similarity_recall_at_10").value
sim_gain = contracts.golden("arxiv.tier03.control_similarity_recall_gain_vs_base").value
rand_recall = contracts.golden("arxiv.tier03.control_random_recall_at_10").value
rand_gain = contracts.golden("arxiv.tier03.control_random_recall_gain_vs_base").value
control_epochs = int(contracts.golden("arxiv.tier03.control_epochs").value)
print(f"recall@10 similarity-graph FT ({control_epochs} epochs): {sim_recall:.3f} "
f"(Δ {sim_gain:+.3f} vs base)")
print(f"recall@10 random-graph FT ({control_epochs} epochs): {rand_recall:.3f} "
f"(Δ {rand_gain:+.3f} vs base)")recall@10 similarity-graph FT (5 epochs): 0.561 (Δ +0.023 vs base)
recall@10 random-graph FT (5 epochs): 0.524 (Δ -0.014 vs base)The ordering — random (harmful) < base < similarity (real, partial) < declared (largest) — is the honest circularity finding: a similarity graph is a weak bootstrap, never a substitute for external supervision, but it is not “nothing.” Only a declared graph, trained to convergence, delivers the largest gain; a random graph actively hurts; the field’s literature (SPECTER for declared-edge supervision, NNCLR / SciNCL for the partial self-similarity signal) predicts exactly this ordering.
Graph-supervised metric learning = contrastive fine-tune over walk-sampled graph edges. node2vec / DeepWalk’s skip-gram over random walks (Grover & Leskovec 2016; the monograph’s Part III graph embeddings; Hamilton et al. 2017) ↔︎
fine_tune_graph. The walk sampler turns a graph into positive pairs; the contrastive loss is the learning objective;edge_provenancenames which graph — declared external structure (SPECTER, Cohan et al. 2020) supervises the largest gain, self-similarity (NNCLR, Dwibedi et al. 2021; SciNCL, Ostendorff et al. 2022) a smaller real one, and a random graph none at all.