SpecEncoder for Galaxy spectra🌕

GitHub: GalaxySVLM
Weight file (recommended): specformer-epoch=440-val.ckpt
Weight file (last): last.ckpt

After finishing the project, we will share all the code on GitHub.

Hyperparameters

Parameters	Value
Epoch	500
Batch	512
LR	1e-4
Scheduler	CosineAnnealingWithWarmupLR
Warmup_step	1000
Optimizer	AdamW

2 NVIDIA RTX PRO 6000

*Data Preprocessing

1. Flux (spectrum) Overlapping

def preprocess_flux(self, b_wave, r_wave, z_wave, b_flux, r_flux, z_flux):
    '''
    simple overlapping (kyuiin method.)
    '''

    # indices to keep
    r_start = np.searchsorted(r_wave, b_wave[-1], side="right")
    z_start = np.searchsorted(z_wave, r_wave[-1], side="right")

    #print("Remove from R:", r_start)
    #print("Remove from Z:", z_start)

    wave = np.concatenate([
        b_wave,
        r_wave[r_start:],
        z_wave[z_start:]
    ])

    flux = np.concatenate([
        b_flux,
        r_flux[r_start:],
        z_flux[z_start:]
    ])

    #print("Final length:", len(wave))
    assert len(wave)==7781, 'manually spectrum length 7781'

    return wave, flux

2. Each Standardization

std, mean = x.std(1, keepdim=True).clip_(0.2), x.mean(1, keepdim=True)
x = (x - mean) / std

3. slicing

   • Overlap length = 10
   • window = 20

4. Add standardization information (mean/std)

x = F.pad(x, pad=(2, 0, 1, 0), mode="constant", value=0)
x[:, 0, 0] = (mean.squeeze() - 2) / 2
x[:, 0, 1] = (std.squeeze() - 2) / 8
#print(x.shape) # (B, 778, 22)

5. masking (for self-supervised learning)
   • mask_chunk_width: 50
   • mask_num_chunks: 6

Reference

• AstroCLIP