Improve model card: Add pipeline tag, library name, paper link, and expanded usage

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +77 -36
README.md CHANGED
@@ -1,70 +1,111 @@
1
  ---
2
- tags:
3
- - model_hub_mixin
4
- - pytorch_model_hub_mixin
5
- license: mit
6
  datasets:
7
  - MultimodalUniverse/legacysurvey
8
  - MultimodalUniverse/hsc
9
  - MultimodalUniverse/gaia
10
  - MultimodalUniverse/sdss
11
  - MultimodalUniverse/desi
 
 
 
 
 
 
12
  ---
 
13
  # AION-1: Astronomical Omnimodal Network
14
 
15
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
16
  [![GitHub Repo](https://img.shields.io/badge/GitHub-Repo-blue?logo=github)](https://github.com/PolymathicAI/AION)
 
17
  [![arXiv](https://img.shields.io/badge/arXiv-2510.17960-b31b1b.svg)](https://arxiv.org/abs/2510.17960)
18
  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb)
19
 
20
- **AION-base** is a 300M parameter large omnimodal model specifically designed for astronomical surveys. It
21
- integrates 39 distinct astronomical data types and enables adaptation to a wide range of astronomical tasks
22
- through multimodal masked modeling.
23
 
24
- ## Model Details
25
-
26
- - **Architecture**: Encoder-Decoder Transformer (12 blocks each, 768 dim, 12 heads)
27
- - **Parameters**: 300M
28
- - **Training**: Multimodal Masked Modeling (4M) on astronomical survey data
29
- - **Modalities**: 39 data types including imaging, spectra, catalogs, and photometry
30
 
31
- ## Quick Start
32
 
33
- ```python
34
- from aion import AION
 
 
35
 
36
- # Load the pretrained model
37
- model = AION.from_pretrained('polymathic-ai/aion-base')
38
 
39
- # Your astronomical analysis begins here!
 
 
40
  ```
 
41
 
42
- ## Supported Data Types
43
-
44
- AION-Base processes data from major astronomical surveys:
45
 
46
- - Imaging: Legacy Survey, HSC Wide
47
- - Spectra: SDSS, DESI
48
- - Catalog: Legacy Survey entries
49
- - Gaia: BP/RP spectra, parallax, coordinates, photometry
50
- - Photometry: Legacy Survey (g,r,i,z + WISE), HSC (g,r,i,z,y)
51
- - Shape: Ellipticity and morphological parameters
52
 
53
- ## Installation
54
-
55
- ```bash
56
- pip install polymathic-aion
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  ```
58
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
 
60
  ## Resources
61
 
62
- - GitHub: https://github.com/PolymathicAI/AION
63
- - Tutorial: https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb
64
 
65
  ## License
66
 
67
- MIT License - see https://github.com/PolymathicAI/AION/blob/main/LICENSE for details.
68
 
69
  ---
70
- Built with for the astronomical community by https://polymathic-ai.org/
 
1
  ---
 
 
 
 
2
  datasets:
3
  - MultimodalUniverse/legacysurvey
4
  - MultimodalUniverse/hsc
5
  - MultimodalUniverse/gaia
6
  - MultimodalUniverse/sdss
7
  - MultimodalUniverse/desi
8
+ license: mit
9
+ tags:
10
+ - model_hub_mixin
11
+ - pytorch_model_hub_mixin
12
+ pipeline_tag: any-to-any
13
+ library_name: aion
14
  ---
15
+
16
  # AION-1: Astronomical Omnimodal Network
17
 
18
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
19
  [![GitHub Repo](https://img.shields.io/badge/GitHub-Repo-blue?logo=github)](https://github.com/PolymathicAI/AION)
20
+ [![Paper](https://img.shields.io/badge/Paper-2510.17960-b31b1b.svg)](https://huggingface.co/papers/2510.17960)
21
  [![arXiv](https://img.shields.io/badge/arXiv-2510.17960-b31b1b.svg)](https://arxiv.org/abs/2510.17960)
22
  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb)
23
 
24
+ **AION-base** is a 300M parameter large omnimodal model specifically designed for astronomical surveys, presented in the paper [AION-1: Omnimodal Foundation Model for Astronomical Sciences](https://huggingface.co/papers/2510.17960). It integrates 39 distinct astronomical data types and enables adaptation to a wide range of astronomical tasks through multimodal masked modeling.
 
 
25
 
26
+ Project Homepage: https://polymathic-ai.org/
 
 
 
 
 
27
 
28
+ ## Model Details
29
 
30
+ - **Architecture**: Encoder-Decoder Transformer (12 blocks each, 768 dim, 12 heads)
31
+ - **Parameters**: 300M
32
+ - **Training**: Multimodal Masked Modeling (4M) on astronomical survey data
33
+ - **Modalities**: 39 data types including imaging, spectra, catalogs, and photometry
34
 
35
+ ## Installation
 
36
 
37
+ Assuming you have PyTorch installed, you can install AION trivially with:
38
+ ```bash
39
+ pip install polymathic-aion
40
  ```
41
+ For advanced installation options, including specific PyTorch versions or developer installations, refer to the [GitHub repository](https://github.com/PolymathicAI/AION).
42
 
43
+ ## Usage
 
 
44
 
45
+ After installation, you can load the pretrained model and start analyzing astronomical data.
 
 
 
 
 
46
 
47
+ ```python
48
+ import torch
49
+ from aion import AION
50
+ from aion.codecs import CodecManager
51
+ from aion.modalities import LegacySurveyImage, Z
52
+
53
+ # Load model and codec manager
54
+ model = AION.from_pretrained('polymathic-ai/aion-base').to('cuda') # or 'aion-large', 'aion-xlarge'
55
+ codec_manager = CodecManager(device='cuda')
56
+
57
+ # Example: Prepare your astronomical data (e.g., a dummy Legacy Survey image)
58
+ # In a real scenario, 'your_image_tensor' would come from your dataset.
59
+ your_image_tensor = torch.randn(1, 4, 96, 96) # Example: batch_size=1, 4 bands, 96x96 resolution
60
+ image = LegacySurveyImage(
61
+ flux=your_image_tensor,
62
+ bands=['DES-G', 'DES-R', 'DES-I', 'DES-Z']
63
+ )
64
+
65
+ # Encode data to tokens
66
+ tokens = codec_manager.encode(image)
67
+
68
+ # Option 1: Extract embeddings for downstream tasks
69
+ embeddings = model.encode(tokens, num_encoder_tokens=600)
70
+ print(f"Extracted embeddings shape: {embeddings.shape}")
71
+
72
+ # Option 2: Generate predictions (e.g., redshift)
73
+ # For this example, we predict redshift (Z) from the image.
74
+ # The target_mask tells the model which modality to generate.
75
+ preds = model(
76
+ codec_manager.encode(image),
77
+ target_modality=Z,
78
+ )
79
+ print(f"Predicted redshift logits shape: {preds['tok_z'].shape}")
80
  ```
81
 
82
+ ### Supported Data Types
83
+ AION-Base processes data from major astronomical surveys. Here's an overview of the supported categories:
84
+
85
+ | **Category** | **Description** | **Token Name(s)** |
86
+ |:------------------------|:----------------------------------------|:-------------------------|
87
+ | **Imaging (2)** | Legacy Survey, HSC Wide | `tok_image_ls`, `tok_image_hsc` |
88
+ | **Catalog (1)** | Legacy Survey catalog entries | `catalog` |
89
+ | **Spectra (2)** | SDSS, DESI | `tok_spectrum_sdss`, `tok_spectrum_desi` |
90
+ | **Gaia (4)** | BP/RP spectra, parallax, sky coords | `tok_xp_bp`, `tok_xp_rp`, `tok_parallax`, `tok_ra`, `tok_dec` |
91
+ | **Gaia Photometry (3)** | G/BP/RP flux | `tok_flux_g_gaia`, `tok_flux_bp_gaia`, `tok_flux_rp_gaia` |
92
+ | **Legacy Survey (9)** | g,r,i,z bands & WISE W1–W4 flux, E(B–V) | `tok_flux_g`,…,`tok_flux_w4`, `tok_ebv` |
93
+ | **Legacy Shape (3)** | Ellipticity components & effective radius | `tok_shape_e1`, `tok_shape_e2`, `tok_shape_r` |
94
+ | **HSC Photometry (5)** | g,r,i,z,y magnitudes | `tok_mag_g`,…,`tok_mag_y` |
95
+ | **HSC Extinction (5)** | g,r,i,z,y extinctions | `tok_a_g`,…,`tok_a_y` |
96
+ | **HSC Shape (3)** | Shape components 11,22,12 | `tok_shape11`, `tok_shape22`, `tok_shape12` |
97
+ | **Other (1)** | Spectroscopic redshift | `tok_z` |
98
+
99
+ More details and interactive examples are available in the [Colab Tutorial](https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb).
100
 
101
  ## Resources
102
 
103
+ - GitHub Repository: https://github.com/PolymathicAI/AION
104
+ - Interactive Tutorial: https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb
105
 
106
  ## License
107
 
108
+ This project is licensed under the MIT License. See the [LICENSE](https://github.com/PolymathicAI/AION/blob/main/LICENSE) file in the GitHub repository for full details.
109
 
110
  ---
111
+ Built with ❤️ for the astronomical community by https://polymathic-ai.org/