Two local models completed all 12 collaborative tasks over three delivery variants. Delivery succeeded, but the model made a factual error. This experiment does not show a speed advantage of IPvX over IP.
The table scrolls horizontally: answer assessment and model time are on the right.
Four different questions per variant; one infrastructure, no WAN or independent operators. Direct IP has fewer hops. The rubric column is an exact label match: the speed question expected insufficient in advance, while the model answered no; both reject the unsubstantiated advantage. This label difference is not the same as a factual error. The model incorrectly rejects the 384/384 fact, contradicts its own label when explaining duplicates, and invents a latency comparison for the speed question.
Download comparison dataIPvX · via R1 · delivery · saved run
A · draft · 7.1 s
{"decision": "no", "evidence": ["A1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
B · review · 9.3 s
{"decision": "no", "evidence": ["A1", "B1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
A · final · 9.3 s
{"decision": "no", "evidence": ["A1", "B1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
A → R1 → B → R1 → A
IPvX · via R1 · speed · saved run
A · draft · 10.2 s
{"decision": "no", "evidence": ["A4: Accepted document-task latency p50 was 459 ms, p95 was 1321 ms."], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one."}
B · review · 14.5 s
{"decision": "no", "evidence": ["A4", "B4"], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one. Evidence B4 also indicates the experiment did not include language models or an ordinary-IP baseline, making it impossible to establish a comparative speed result."}
A · final · 14.3 s
{"decision": "no", "evidence": ["A4", "B4"], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one. Evidence B4 also indicates the experiment did not include language models or an ordinary-IP baseline, making it impossible to establish a comparative speed result."}
A → R1 → B → R1 → A
IP · direct · delivery · saved run
A · draft · 7.2 s
{"decision": "no", "evidence": ["A1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
B · review · 9.4 s
{"decision": "no", "evidence": ["A1", "B1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
A · final · 9.5 s
{"decision": "no", "evidence": ["A1", "B1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
A → B → A
IP · direct · speed · saved run
A · draft · 10.4 s
{"decision": "no", "evidence": ["A4: Accepted document-task latency p50 was 459 ms, p95 was 1321 ms."], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one."}
B · review · 13.6 s
{"decision": "no", "evidence": ["A4", "B4"], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one. Evidence B4 also indicates the experiment did not include language models or an ordinary-IP baseline, making it impossible to establish a comparative speed result."}
A · final · 14.1 s
{"decision": "no", "evidence": ["A4", "B4"], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one. Evidence B4 also indicates the experiment did not include language models or an ordinary-IP baseline, making it impossible to establish a comparative speed result."}
A → B → A
IP · via R1 · delivery · saved run
A · draft · 7.1 s
{"decision": "no", "evidence": ["A1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
B · review · 9.3 s
{"decision": "no", "evidence": ["A1", "B1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
A · final · 9.5 s
{"decision": "no", "evidence": ["A1", "B1"], "explanation": "The evidence provided does not confirm that all 384 document jobs returned correct results. Only one run completed 384/384 tasks correctly."}
A → R1 → B → R1 → A
IP · via R1 · speed · saved run
A · draft · 9.8 s
{"decision": "no", "evidence": ["A4: Accepted document-task latency p50 was 459 ms, p95 was 1321 ms."], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one."}
B · review · 13.5 s
{"decision": "no", "evidence": ["A4", "B4"], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one. Evidence B4 also indicates the experiment did not include language models or an ordinary-IP baseline, making it impossible to establish a comparative speed result."}
A · final · 14.0 s
{"decision": "no", "evidence": ["A4", "B4"], "explanation": "The provided evidence does not support the claim that native IPvX is faster than ordinary IP for language-model agents. The data shows a higher latency for IPvX, not a faster one. Evidence B4 also indicates the experiment did not include language models or an ordinary-IP baseline, making it impossible to establish a comparative speed result."}
A → R1 → B → R1 → A
