Improve model card: Add pipeline tag, library name, paper link, and expanded usage
#1
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,70 +1,111 @@
|
|
| 1 |
---
|
| 2 |
-
tags:
|
| 3 |
-
- model_hub_mixin
|
| 4 |
-
- pytorch_model_hub_mixin
|
| 5 |
-
license: mit
|
| 6 |
datasets:
|
| 7 |
- MultimodalUniverse/legacysurvey
|
| 8 |
- MultimodalUniverse/hsc
|
| 9 |
- MultimodalUniverse/gaia
|
| 10 |
- MultimodalUniverse/sdss
|
| 11 |
- MultimodalUniverse/desi
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
---
|
|
|
|
| 13 |
# AION-1: Astronomical Omnimodal Network
|
| 14 |
|
| 15 |
[](https://opensource.org/licenses/MIT)
|
| 16 |
[](https://github.com/PolymathicAI/AION)
|
|
|
|
| 17 |
[](https://arxiv.org/abs/2510.17960)
|
| 18 |
[](https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb)
|
| 19 |
|
| 20 |
-
**AION-base** is a 300M parameter large omnimodal model specifically designed for astronomical surveys. It
|
| 21 |
-
integrates 39 distinct astronomical data types and enables adaptation to a wide range of astronomical tasks
|
| 22 |
-
through multimodal masked modeling.
|
| 23 |
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
- **Architecture**: Encoder-Decoder Transformer (12 blocks each, 768 dim, 12 heads)
|
| 27 |
-
- **Parameters**: 300M
|
| 28 |
-
- **Training**: Multimodal Masked Modeling (4M) on astronomical survey data
|
| 29 |
-
- **Modalities**: 39 data types including imaging, spectra, catalogs, and photometry
|
| 30 |
|
| 31 |
-
##
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
|
|
|
|
|
|
| 35 |
|
| 36 |
-
|
| 37 |
-
model = AION.from_pretrained('polymathic-ai/aion-base')
|
| 38 |
|
| 39 |
-
|
|
|
|
|
|
|
| 40 |
```
|
|
|
|
| 41 |
|
| 42 |
-
##
|
| 43 |
-
|
| 44 |
-
AION-Base processes data from major astronomical surveys:
|
| 45 |
|
| 46 |
-
|
| 47 |
-
- Spectra: SDSS, DESI
|
| 48 |
-
- Catalog: Legacy Survey entries
|
| 49 |
-
- Gaia: BP/RP spectra, parallax, coordinates, photometry
|
| 50 |
-
- Photometry: Legacy Survey (g,r,i,z + WISE), HSC (g,r,i,z,y)
|
| 51 |
-
- Shape: Ellipticity and morphological parameters
|
| 52 |
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
```
|
| 58 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
## Resources
|
| 61 |
|
| 62 |
-
-
|
| 63 |
-
- Tutorial: https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb
|
| 64 |
|
| 65 |
## License
|
| 66 |
|
| 67 |
-
MIT License
|
| 68 |
|
| 69 |
---
|
| 70 |
-
Built with
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
datasets:
|
| 3 |
- MultimodalUniverse/legacysurvey
|
| 4 |
- MultimodalUniverse/hsc
|
| 5 |
- MultimodalUniverse/gaia
|
| 6 |
- MultimodalUniverse/sdss
|
| 7 |
- MultimodalUniverse/desi
|
| 8 |
+
license: mit
|
| 9 |
+
tags:
|
| 10 |
+
- model_hub_mixin
|
| 11 |
+
- pytorch_model_hub_mixin
|
| 12 |
+
pipeline_tag: any-to-any
|
| 13 |
+
library_name: aion
|
| 14 |
---
|
| 15 |
+
|
| 16 |
# AION-1: Astronomical Omnimodal Network
|
| 17 |
|
| 18 |
[](https://opensource.org/licenses/MIT)
|
| 19 |
[](https://github.com/PolymathicAI/AION)
|
| 20 |
+
[](https://huggingface.co/papers/2510.17960)
|
| 21 |
[](https://arxiv.org/abs/2510.17960)
|
| 22 |
[](https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb)
|
| 23 |
|
| 24 |
+
**AION-base** is a 300M parameter large omnimodal model specifically designed for astronomical surveys, presented in the paper [AION-1: Omnimodal Foundation Model for Astronomical Sciences](https://huggingface.co/papers/2510.17960). It integrates 39 distinct astronomical data types and enables adaptation to a wide range of astronomical tasks through multimodal masked modeling.
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
Project Homepage: https://polymathic-ai.org/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
+
## Model Details
|
| 29 |
|
| 30 |
+
- **Architecture**: Encoder-Decoder Transformer (12 blocks each, 768 dim, 12 heads)
|
| 31 |
+
- **Parameters**: 300M
|
| 32 |
+
- **Training**: Multimodal Masked Modeling (4M) on astronomical survey data
|
| 33 |
+
- **Modalities**: 39 data types including imaging, spectra, catalogs, and photometry
|
| 34 |
|
| 35 |
+
## Installation
|
|
|
|
| 36 |
|
| 37 |
+
Assuming you have PyTorch installed, you can install AION trivially with:
|
| 38 |
+
```bash
|
| 39 |
+
pip install polymathic-aion
|
| 40 |
```
|
| 41 |
+
For advanced installation options, including specific PyTorch versions or developer installations, refer to the [GitHub repository](https://github.com/PolymathicAI/AION).
|
| 42 |
|
| 43 |
+
## Usage
|
|
|
|
|
|
|
| 44 |
|
| 45 |
+
After installation, you can load the pretrained model and start analyzing astronomical data.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
+
```python
|
| 48 |
+
import torch
|
| 49 |
+
from aion import AION
|
| 50 |
+
from aion.codecs import CodecManager
|
| 51 |
+
from aion.modalities import LegacySurveyImage, Z
|
| 52 |
+
|
| 53 |
+
# Load model and codec manager
|
| 54 |
+
model = AION.from_pretrained('polymathic-ai/aion-base').to('cuda') # or 'aion-large', 'aion-xlarge'
|
| 55 |
+
codec_manager = CodecManager(device='cuda')
|
| 56 |
+
|
| 57 |
+
# Example: Prepare your astronomical data (e.g., a dummy Legacy Survey image)
|
| 58 |
+
# In a real scenario, 'your_image_tensor' would come from your dataset.
|
| 59 |
+
your_image_tensor = torch.randn(1, 4, 96, 96) # Example: batch_size=1, 4 bands, 96x96 resolution
|
| 60 |
+
image = LegacySurveyImage(
|
| 61 |
+
flux=your_image_tensor,
|
| 62 |
+
bands=['DES-G', 'DES-R', 'DES-I', 'DES-Z']
|
| 63 |
+
)
|
| 64 |
+
|
| 65 |
+
# Encode data to tokens
|
| 66 |
+
tokens = codec_manager.encode(image)
|
| 67 |
+
|
| 68 |
+
# Option 1: Extract embeddings for downstream tasks
|
| 69 |
+
embeddings = model.encode(tokens, num_encoder_tokens=600)
|
| 70 |
+
print(f"Extracted embeddings shape: {embeddings.shape}")
|
| 71 |
+
|
| 72 |
+
# Option 2: Generate predictions (e.g., redshift)
|
| 73 |
+
# For this example, we predict redshift (Z) from the image.
|
| 74 |
+
# The target_mask tells the model which modality to generate.
|
| 75 |
+
preds = model(
|
| 76 |
+
codec_manager.encode(image),
|
| 77 |
+
target_modality=Z,
|
| 78 |
+
)
|
| 79 |
+
print(f"Predicted redshift logits shape: {preds['tok_z'].shape}")
|
| 80 |
```
|
| 81 |
|
| 82 |
+
### Supported Data Types
|
| 83 |
+
AION-Base processes data from major astronomical surveys. Here's an overview of the supported categories:
|
| 84 |
+
|
| 85 |
+
| **Category** | **Description** | **Token Name(s)** |
|
| 86 |
+
|:------------------------|:----------------------------------------|:-------------------------|
|
| 87 |
+
| **Imaging (2)** | Legacy Survey, HSC Wide | `tok_image_ls`, `tok_image_hsc` |
|
| 88 |
+
| **Catalog (1)** | Legacy Survey catalog entries | `catalog` |
|
| 89 |
+
| **Spectra (2)** | SDSS, DESI | `tok_spectrum_sdss`, `tok_spectrum_desi` |
|
| 90 |
+
| **Gaia (4)** | BP/RP spectra, parallax, sky coords | `tok_xp_bp`, `tok_xp_rp`, `tok_parallax`, `tok_ra`, `tok_dec` |
|
| 91 |
+
| **Gaia Photometry (3)** | G/BP/RP flux | `tok_flux_g_gaia`, `tok_flux_bp_gaia`, `tok_flux_rp_gaia` |
|
| 92 |
+
| **Legacy Survey (9)** | g,r,i,z bands & WISE W1–W4 flux, E(B–V) | `tok_flux_g`,…,`tok_flux_w4`, `tok_ebv` |
|
| 93 |
+
| **Legacy Shape (3)** | Ellipticity components & effective radius | `tok_shape_e1`, `tok_shape_e2`, `tok_shape_r` |
|
| 94 |
+
| **HSC Photometry (5)** | g,r,i,z,y magnitudes | `tok_mag_g`,…,`tok_mag_y` |
|
| 95 |
+
| **HSC Extinction (5)** | g,r,i,z,y extinctions | `tok_a_g`,…,`tok_a_y` |
|
| 96 |
+
| **HSC Shape (3)** | Shape components 11,22,12 | `tok_shape11`, `tok_shape22`, `tok_shape12` |
|
| 97 |
+
| **Other (1)** | Spectroscopic redshift | `tok_z` |
|
| 98 |
+
|
| 99 |
+
More details and interactive examples are available in the [Colab Tutorial](https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb).
|
| 100 |
|
| 101 |
## Resources
|
| 102 |
|
| 103 |
+
- GitHub Repository: https://github.com/PolymathicAI/AION
|
| 104 |
+
- Interactive Tutorial: https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb
|
| 105 |
|
| 106 |
## License
|
| 107 |
|
| 108 |
+
This project is licensed under the MIT License. See the [LICENSE](https://github.com/PolymathicAI/AION/blob/main/LICENSE) file in the GitHub repository for full details.
|
| 109 |
|
| 110 |
---
|
| 111 |
+
Built with ❤️ for the astronomical community by https://polymathic-ai.org/
|