Ishrat Jahan Ananya
AI & ML interests
Recent Activity
Organizations
mDenseOn with the mLateOn: Open Multilingual, Long-Context, and Code Retrieval Models
Thanks for the careful read. Your understanding is correct and we flag it in the post: the ordering among negatives is rank distillation from the miner, biases included.
One reason we suspect it's not mainly bias reproduction: the gains transfer outside the miner's distribution. The miners are GTE-ModernBERT / Nomic, yet the lead holds when we swap the first stage to Qwen3-VL-Embedding, and the model transfers zero-shot from EN to FR. A model that had mostly memorized miner quirks should be more brittle across those shifts. But that's indirect evidence, not the ablation you're asking for.
You're right that we haven't ablated the supervision source directly, though shuffled negatives and a pos-vs-neg-only partial order are cheap to run and would cleanly answer this. Adding them to the list, thanks for the suggestion!
Thank you! We actually did evaluate llama-nemotron-rerank-vl-1b-v2 while preparing the post, under the exact same protocol. Here are the details:
ViDoRe V3 (ColQwen2.5 first-stage, NDCG@10): 58.97 EN / 56.02 FR → 57.50 overall. Just below the 2B pack (MonoQwen 57.76, Qwen3-VL-2B 59.18, jina-m0 59.40, ours 62.66), which is a very good result for the smallest model in the comparison.
BEIR text (clean mean NDCG@10): 53.79 with BM25 which is essentially tied with jina-reranker-m0 as the best text score of any multimodal reranker we measured, ahead of our own models.
So it mirrors the post's theme from the other side: excellent on text for its size, trailing the 2B class on visual documents. We'll edit it in to the comparison tables, thanks for the nudge.