Many pancreatic neuroendocrine tumors (PanNETs) fall into 2 major prognostic subtypes based on DAXX/ATRX-induced alternative lengthening of telomerase phenotype and alpha- and beta-cell-like epigenomic profiles. However, these PanNETs are still flanked by other PanNETs that do not fit into either subtype. Furthermore, despite advanced genotyping, PanNETs are generally not well-characterized in terms of their histologic and hormonal phenotypes. We aimed to identify new subgroups of PanNETs by extending the currently used transcription factor signatures and investigating their correlation with histologic, hormonal, molecular, and prognostic findings. One hundred eighty-five PanNETs (nonfunctioning 165 and functioning 20), resected between 1996 and 2023, were classified into 5 subgroups (A1, A2, B, C, and D) by cluster analysis based on ARX, PDX1, islet-1 (ISL1), and CDX2 expressions and correlated with trabecular vs solid histology, expression of insulin, glucagon, polypeptide (PP), somatostatin, serotonin, gastrin, calcitonin, adrenocorticotropic hormone (ACTH), DAXX/ATRX, MEN1, and alternative lengthening of telomerase status by fluorescence in situ hybridization, and disease-free survival. A1 (46%, ARX+/ISL1+/PDX1-/CDX2-) and A2 (15%, ARX+/ISL1+/PDX1+/CDX2-) showed trabecular histology and glucagon/PP expression, with A2 also showing gastrin expression. B (18%, PDX1+/ISL1+/ARX-/CDX2-) showed solid histology, insulin, and somatostatin expression (P < .001). It included all insulinomas and had the best outcome (P < .01). C (15%, ARX-/PDX1-/ISL1-/CDX2-) showed solid histology and frequent expression of serotonin, calcitonin, and ACTH. D (5%, PDX1+/CDX2+/ISL1-/ARX-) showed solid histology, expressed ACTH/serotonin, and was an independent poor prognosticator (P < .01). Differential expressions of ARX, PDX1, ISL1, and CDX2 stratified PanNETs into 5 subgroups with different histologies, hormone expressions, and outcomes. Subgroups A1 and A2 resembled the alpha-cell-like type, and subgroup B, the beta-cell-like type. Subgroup C with almost no transcription factor signature was unclear in cell lineage, whereas the PDX+/CDX2+ signature of subgroup D suggested a pancreatic/intestinal cell lineage. Assigning PanNETs to the subgroups may help establish the diagnosis, predict the outcome, and guide the treatment.