MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
Paper
•
2411.15296
•
Published
•
21
AI & ML interests
None defined yet.
Recent Activity
View all activity
luodian
authored
4
papers
2 months ago
Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos
Paper
•
2501.13826
•
Published
•
24
LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training
Paper
•
2509.23661
•
Published
•
46
Visual Jigsaw Post-Training Improves MLLMs
Paper
•
2509.25190
•
Published
•
36
yuna0x0
authored
a
paper
2 months ago
osanseviero
authored
a
paper
2 months ago
mikonvergence
posted
an
update
4 months ago
Post
426
𝐌𝐚𝐣𝐨𝐫 𝐓𝐎𝐌 ➕ 𝐆𝐨𝐨𝐠𝐥𝐞 𝐃𝐞𝐞𝐩𝐌𝐢𝐧𝐝'𝐬 𝐀𝐥𝐩𝐡𝐚𝐄𝐚𝐫𝐭𝐡 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠𝐬 𝐚𝐫𝐞 𝐧𝐨𝐰 𝐚𝐯𝐚𝐢𝐥𝐚𝐛𝐥𝐞 ‼️
This is a tiny (about 6 TB of data, but only 62,489 grid cells of ~100 sqkm) prototype dataset that allows to instantly connect existing Major TOM data with AlphaEarth embeddings.
Major-TOM/Core-AlphaEarth-Embeddings
I curated it to support several relevant research projects, but I figured it could help more people in the community to experiment and explore new applications of AlphaEarth.
𝐃𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧𝐬 𝐟𝐨𝐫 𝐔𝐬𝐞
Each embedding sample comes from the original annual dataset produced by Google DeepMind. It means that, unlike samples from Sentinel-2 or Sentinel-1, it contains aggregated annual information from a particular year and is not linked to one particular observation. The existing Major TOM samples from physical sensors provide information potentially (and likely) contained in the AlphaEarth embedding sample, but they miss the temporal component represented within AEF embedding fields.
For more information, please check the dataset card on HuggingFace.
⚠️ 𝐖𝐀𝐑𝐍𝐈𝐍𝐆: 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠𝐬 𝐢𝐧 𝐭𝐡𝐢𝐬 𝐝𝐚𝐭𝐚𝐬𝐞𝐭 𝐝𝐨 𝐧𝐨𝐭 𝐫𝐞𝐩𝐫𝐞𝐬𝐞𝐧𝐭 𝐢𝐧𝐝𝐢𝐯𝐢𝐝𝐮𝐚𝐥 𝐬𝐚𝐦𝐩𝐥𝐞𝐬, 𝐛𝐮𝐭 𝐚 𝐰𝐡𝐨𝐥𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐦𝐮𝐥𝐭𝐢-𝐦𝐨𝐝𝐚𝐥 𝐨𝐛𝐬𝐞𝐫𝐯𝐚𝐭𝐢𝐨𝐧𝐬. 𝐇𝐚𝐯𝐞 𝐟𝐮𝐧!
🙏 Built on top of fantastic work of
Christopher Brown, Michal Kazmierski, Valerie Pasquarella, Emily Schechter and others at Google DeepMind.
This is a tiny (about 6 TB of data, but only 62,489 grid cells of ~100 sqkm) prototype dataset that allows to instantly connect existing Major TOM data with AlphaEarth embeddings.
Major-TOM/Core-AlphaEarth-Embeddings
I curated it to support several relevant research projects, but I figured it could help more people in the community to experiment and explore new applications of AlphaEarth.
𝐃𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧𝐬 𝐟𝐨𝐫 𝐔𝐬𝐞
Each embedding sample comes from the original annual dataset produced by Google DeepMind. It means that, unlike samples from Sentinel-2 or Sentinel-1, it contains aggregated annual information from a particular year and is not linked to one particular observation. The existing Major TOM samples from physical sensors provide information potentially (and likely) contained in the AlphaEarth embedding sample, but they miss the temporal component represented within AEF embedding fields.
For more information, please check the dataset card on HuggingFace.
⚠️ 𝐖𝐀𝐑𝐍𝐈𝐍𝐆: 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠𝐬 𝐢𝐧 𝐭𝐡𝐢𝐬 𝐝𝐚𝐭𝐚𝐬𝐞𝐭 𝐝𝐨 𝐧𝐨𝐭 𝐫𝐞𝐩𝐫𝐞𝐬𝐞𝐧𝐭 𝐢𝐧𝐝𝐢𝐯𝐢𝐝𝐮𝐚𝐥 𝐬𝐚𝐦𝐩𝐥𝐞𝐬, 𝐛𝐮𝐭 𝐚 𝐰𝐡𝐨𝐥𝐞 𝐲𝐞𝐚𝐫 𝐨𝐟 𝐦𝐮𝐥𝐭𝐢-𝐦𝐨𝐝𝐚𝐥 𝐨𝐛𝐬𝐞𝐫𝐯𝐚𝐭𝐢𝐨𝐧𝐬. 𝐇𝐚𝐯𝐞 𝐟𝐮𝐧!
🙏 Built on top of fantastic work of
Christopher Brown, Michal Kazmierski, Valerie Pasquarella, Emily Schechter and others at Google DeepMind.
Post
4366
YAML engineering becomes more and more important than ever from infra provisioning to model training (recipes).
Here, I built a simple editor first for @dstackai , and I will share the live endpoint this week. Let me know what you think about this approach.
Based on this approach, if people think this is useful, I am going to do the same thing for the LLM training recipes for popular frameworks such as Hugging Face open-r1, Axolotl, and so on. Let me hear.
Here, I built a simple editor first for @dstackai , and I will share the live endpoint this week. Let me know what you think about this approach.
Based on this approach, if people think this is useful, I am going to do the same thing for the LLM training recipes for popular frameworks such as Hugging Face open-r1, Axolotl, and so on. Let me hear.
luodian
authored
2
papers
5 months ago
Use Space Secrets
#2 opened 7 months ago
by
freddyaboulton
mikonvergence
authored
2
papers
8 months ago
mikonvergence
posted
an
update
8 months ago
Post
838
🔵 𝐂𝐎𝐏-𝐆𝐄𝐍-𝐁𝐞𝐭𝐚: 𝐔𝐧𝐢𝐟𝐢𝐞𝐝 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐌𝐨𝐝𝐞𝐥𝐥𝐢𝐧𝐠 𝐨𝐟 𝐂𝐎𝐏𝐞𝐫𝐧𝐢𝐜𝐮𝐬 𝐈𝐦𝐚𝐠𝐞𝐫𝐲 𝐓𝐡𝐮𝐦𝐛𝐧𝐚𝐢𝐥𝐬
Today we release a prototype of COP-GEN - a universal generative model for Copernicus data. 𝐂𝐎𝐏-𝐆𝐄𝐍-𝐁𝐞𝐭𝐚 is a model trained globally on the thumbnails of the Major TOM Core datasets, including Sentinel-2 L1C, Sentinel-2 L2A, Sentinel-1 RTC, and COP-DEM GLO-30.
⚖️ 𝐌𝐨𝐝𝐞𝐥 mespinosami/COP-GEN-Beta
📱 𝐃𝐞𝐦𝐨 mikonvergence/COP-GEN-Beta
How is it universal? COP-GEN learns a joint generative process of all modalities, which means that it can reconstruct data from any subset of present observations. 𝐖𝐢𝐭𝐡𝐨𝐮𝐭 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐬𝐩𝐞𝐜𝐢𝐟𝐢𝐜𝐚𝐥𝐥𝐲 to perform any of these tasks it can be used to approximate:
✅ Sentinel-1 to Sentinel-2 translation
✅ Elevation estimation from Sentinel-2 or Sentinel-1
✅ Atmospheric Correction (L1C to L2A pipeline)
✅ Atmospheric Generation (L2A to L1C)
✅ ...and any other task involving translation between the supported modalities
On its own, the model can be used as a useful prior for estimating the data likelihood distribution for Copernicus data. COP-GEN-Beta learns joint, conditional, and marginal distributions within a single unified backbone, allowing to flexibly sample any modality given any condition.
Why is it Beta? Because thumbnails are a low-cost representation of the data that scales well and we managed to develop this prototype quite fast. We are currently developing the more costly COP-GEN model that supports the original data. For now, we wanted to showcase the prototype and make it available to the community for a test!
🌐 𝐖𝐞𝐛𝐬𝐢𝐭𝐞 https://miquel-espinosa.github.io/cop-gen
💻 𝐂𝐨𝐝𝐞 https://github.com/miquel-espinosa/COP-GEN-Beta
📄 𝐏𝐚𝐩𝐞𝐫 https://arxiv.org/pdf/2504.08548
Today we release a prototype of COP-GEN - a universal generative model for Copernicus data. 𝐂𝐎𝐏-𝐆𝐄𝐍-𝐁𝐞𝐭𝐚 is a model trained globally on the thumbnails of the Major TOM Core datasets, including Sentinel-2 L1C, Sentinel-2 L2A, Sentinel-1 RTC, and COP-DEM GLO-30.
⚖️ 𝐌𝐨𝐝𝐞𝐥 mespinosami/COP-GEN-Beta
📱 𝐃𝐞𝐦𝐨 mikonvergence/COP-GEN-Beta
How is it universal? COP-GEN learns a joint generative process of all modalities, which means that it can reconstruct data from any subset of present observations. 𝐖𝐢𝐭𝐡𝐨𝐮𝐭 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐬𝐩𝐞𝐜𝐢𝐟𝐢𝐜𝐚𝐥𝐥𝐲 to perform any of these tasks it can be used to approximate:
✅ Sentinel-1 to Sentinel-2 translation
✅ Elevation estimation from Sentinel-2 or Sentinel-1
✅ Atmospheric Correction (L1C to L2A pipeline)
✅ Atmospheric Generation (L2A to L1C)
✅ ...and any other task involving translation between the supported modalities
On its own, the model can be used as a useful prior for estimating the data likelihood distribution for Copernicus data. COP-GEN-Beta learns joint, conditional, and marginal distributions within a single unified backbone, allowing to flexibly sample any modality given any condition.
Why is it Beta? Because thumbnails are a low-cost representation of the data that scales well and we managed to develop this prototype quite fast. We are currently developing the more costly COP-GEN model that supports the original data. For now, we wanted to showcase the prototype and make it available to the community for a test!
🌐 𝐖𝐞𝐛𝐬𝐢𝐭𝐞 https://miquel-espinosa.github.io/cop-gen
💻 𝐂𝐨𝐝𝐞 https://github.com/miquel-espinosa/COP-GEN-Beta
📄 𝐏𝐚𝐩𝐞𝐫 https://arxiv.org/pdf/2504.08548
-
- 1 reply
mikonvergence
posted
an
update
8 months ago
Post
1696
𝐌𝐄𝐒𝐀 🏔️ 𝐓𝐞𝐱𝐭-𝐛𝐚𝐬𝐞𝐝 𝐭𝐞𝐫𝐫𝐚𝐢𝐧 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐦𝐨𝐝𝐞𝐥
MESA is a novel generative model based on latent denoising diffusion capable of generating 2.5D representations (co-registered colour and depth maps) of terrains based on text prompt conditioning.
Work developed by Paul Borne–Pons ( @NewtNewt ) during his joint internship at
Adobe & ESA, and in collaboration with asterisk labs.
🏔️ 𝐏𝐫𝐨𝐣𝐞𝐜𝐭 𝐏𝐚𝐠𝐞 : https://paulbornep.github.io/mesa-terrain/
📝 𝐏𝐫𝐞𝐩𝐫𝐢𝐧𝐭 : https://arxiv.org/abs/2504.07210
🤗 𝐌𝐨𝐝𝐞𝐥 𝐖𝐞𝐢𝐠𝐡𝐭𝐬 : NewtNewt/MESA
💾 𝐃𝐚𝐭𝐚𝐬𝐞𝐭 : Major-TOM/Core-DEM
🧑🏻💻𝐂𝐨𝐝𝐞 : https://github.com/PaulBorneP/MESA
𝐇𝐅 𝐒𝐩𝐚𝐜𝐞: mikonvergence/MESA
MESA is a novel generative model based on latent denoising diffusion capable of generating 2.5D representations (co-registered colour and depth maps) of terrains based on text prompt conditioning.
Work developed by Paul Borne–Pons ( @NewtNewt ) during his joint internship at
Adobe & ESA, and in collaboration with asterisk labs.
🏔️ 𝐏𝐫𝐨𝐣𝐞𝐜𝐭 𝐏𝐚𝐠𝐞 : https://paulbornep.github.io/mesa-terrain/
📝 𝐏𝐫𝐞𝐩𝐫𝐢𝐧𝐭 : https://arxiv.org/abs/2504.07210
🤗 𝐌𝐨𝐝𝐞𝐥 𝐖𝐞𝐢𝐠𝐡𝐭𝐬 : NewtNewt/MESA
💾 𝐃𝐚𝐭𝐚𝐬𝐞𝐭 : Major-TOM/Core-DEM
🧑🏻💻𝐂𝐨𝐝𝐞 : https://github.com/PaulBorneP/MESA
𝐇𝐅 𝐒𝐩𝐚𝐜𝐞: mikonvergence/MESA
- 2 replies
mikonvergence
authored
a
paper
8 months ago
osanseviero
authored
a
paper
8 months ago
Post
3947
simple guide on the recipe for GRPO on Open-R1 which is built on top of TRL
I think FastAPI wrapper of vLLM with WeightSyncWorker is pretty cool feature. Also, we have many predefined reward functions out of the box!
I think FastAPI wrapper of vLLM with WeightSyncWorker is pretty cool feature. Also, we have many predefined reward functions out of the box!
- 5 replies
Post
2675
Mistral AI Small 3.1 24B is not only commercial free but also the best model in a single GPU deployment.
I packed up all the information you need to know in a single picture. Hope this helps! :)
I packed up all the information you need to know in a single picture. Hope this helps! :)
- 1 reply
Post
1604
Gemma 3 Release in a nutshell
(seems like function calling is not supported whereas the announcement said so)
(seems like function calling is not supported whereas the announcement said so)