BioHackathon_Lipids-tapt_ulmfit-LR_2e-05

This model is a fine-tuned version of microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0089
• Accuracy: 0.7719

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 3407
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.06
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.1173	1.0	29	1.0719	0.7634
1.1305	2.0	58	1.0794	0.7616
1.1273	3.0	87	1.0206	0.7735
1.1235	4.0	116	1.0500	0.7668
1.1148	5.0	145	1.0672	0.7624
1.0937	6.0	174	1.0456	0.7704
1.0873	7.0	203	1.0653	0.7674
1.0914	8.0	232	1.0253	0.7688
1.1042	9.0	261	1.0706	0.7635
1.0698	10.0	290	1.0236	0.7744
1.0837	11.0	319	1.0653	0.7632
1.0862	12.0	348	1.0166	0.7745
1.0699	13.0	377	1.0374	0.7684
1.0775	14.0	406	1.0149	0.7752
1.0512	15.0	435	1.0393	0.7689
1.0781	16.0	464	1.0280	0.7692
1.0683	17.0	493	1.0156	0.7726
1.0821	18.0	522	1.0366	0.7750
1.0543	19.0	551	1.0215	0.7729
1.0674	20.0	580	1.0010	0.7736
1.0628	21.0	609	1.0159	0.7682
1.0622	22.0	638	1.0183	0.7691
1.0399	23.0	667	1.0685	0.7654
1.0301	24.0	696	1.0431	0.7718
1.0517	25.0	725	1.0105	0.7746
1.0529	26.0	754	0.9985	0.7732
1.0448	27.0	783	1.0250	0.7685
1.036	28.0	812	1.0129	0.7674
1.0334	29.0	841	1.0039	0.7743
1.0279	30.0	870	1.0235	0.7717
1.031	31.0	899	1.0073	0.7719
1.0038	32.0	928	1.0215	0.7662
1.0077	33.0	957	1.0471	0.7674
1.0056	34.0	986	1.0235	0.7707
1.0017	35.0	1015	0.9899	0.7729
0.9885	36.0	1044	1.0062	0.7753
1.0057	37.0	1073	1.0204	0.7703
1.0062	38.0	1102	1.0028	0.7709
0.9956	39.0	1131	1.0575	0.7602
1.0014	40.0	1160	1.0122	0.7745
0.997	41.0	1189	1.0124	0.7721
0.9741	42.0	1218	1.0455	0.7637
0.9816	43.0	1247	1.0390	0.7677
0.9899	44.0	1276	1.0734	0.7593
0.9707	45.0	1305	1.0267	0.7700
0.9705	46.0	1334	1.0408	0.7726
0.9663	47.0	1363	1.0350	0.7679
0.9914	48.0	1392	1.0182	0.7686
0.9909	49.0	1421	1.0218	0.7664
0.9678	50.0	1450	1.0417	0.7709
0.964	51.0	1479	1.0112	0.7733
0.9655	52.0	1508	1.0314	0.7666
0.9735	53.0	1537	1.0200	0.7702
0.9624	54.0	1566	1.0394	0.7682
0.951	55.0	1595	1.0211	0.7697
0.9664	56.0	1624	1.0353	0.7706
0.9449	57.0	1653	1.0195	0.7726
0.9464	58.0	1682	1.0432	0.7644
0.9477	59.0	1711	1.0151	0.7701
0.9381	60.0	1740	0.9901	0.7765
0.9442	61.0	1769	1.0175	0.7673
0.9364	62.0	1798	1.0653	0.7639
0.9405	63.0	1827	1.0308	0.7651
0.9498	64.0	1856	1.0132	0.7727
0.9372	65.0	1885	1.0252	0.7729
0.9356	66.0	1914	1.0254	0.7664
0.9383	67.0	1943	0.9812	0.7742
0.9336	68.0	1972	1.0125	0.7712
0.9362	69.0	2001	1.0397	0.7631
0.9273	70.0	2030	1.0128	0.7671
0.933	71.0	2059	1.0381	0.7659
0.9418	72.0	2088	1.0013	0.7722
0.952	73.0	2117	1.0078	0.7724
0.9204	74.0	2146	1.0435	0.7721
0.9186	75.0	2175	1.0336	0.7704
0.9303	76.0	2204	1.0415	0.7704
0.9376	77.0	2233	1.0153	0.7772
0.9168	78.0	2262	1.0177	0.7689
0.9211	79.0	2291	1.0094	0.7752
0.9527	80.0	2320	0.9772	0.7762
0.9262	81.0	2349	1.0733	0.7675
0.9366	82.0	2378	1.0236	0.7707
0.9174	83.0	2407	1.0249	0.7679
0.9156	84.0	2436	1.0120	0.7689
0.933	85.0	2465	1.0519	0.7647
0.9294	86.0	2494	1.0183	0.7701
0.928	87.0	2523	1.0252	0.7660
0.9202	88.0	2552	1.0469	0.7644
0.9081	89.0	2581	1.0259	0.7707
0.9164	90.0	2610	1.0298	0.7666
0.9215	91.0	2639	1.0293	0.7710
0.9258	92.0	2668	1.0461	0.7634
0.9052	93.0	2697	1.0126	0.7721
0.9322	94.0	2726	1.0077	0.7741
0.9144	95.0	2755	1.0300	0.7671
0.9204	96.0	2784	1.0631	0.7628
0.9323	96.5614	2800	1.0089	0.7719

Framework versions

• Transformers 4.48.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.2
• Tokenizers 0.21.0