Materials and methods

Study design and ethical approval

This study, designed as a retrospective cohort study, was approved by the Near East University Scientific Research Ethics Committee and was conducted in accordance with the principles of the Declaration of Helsinki (decision no. YDU/2025/139–2045, dated 05.01.2026). Due to the retrospective design of the study and the complete de-identification of the data, the requirement for patient consent was waived in accordance with ethics committee standards. Panoramic radiographs taken for routine dental treatment from the archives of Near East University Faculty of Dentistry, North Cyprus, between 2011 and 2013, were used for secondary forensic research purposes. To protect patient privacy, all images were completely anonymized, and demographic information (age, sex) was stored in a separate secure database (labels.csv). All analyses were conducted on de-identified data. This study was reported in accordance with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) and TRIPOD-AI (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis — Artificial Intelligence) guidelines (completed checklists available as supplementary material upon request).

Study population

Inclusion criteria were defined as: being 18 years of age or older, having images of diagnostic quality without significant artifacts, and having no history of pathology, trauma, or surgery affecting the maxillary sinus region. Individuals with artifacts degrading image quality, pathological lesions preventing assessment of sinus boundaries, or a history of maxillofacial surgery were excluded from the study.

In accordance with these criteria, a balanced dataset consisting of panoramic radiographs from a total of 1,024 patients was created. The sample includes 512 males (50%) and 512 females (50%). A priori power analysis (G*Power 3.1, two-tailed, α = 0.05, power = 0.80, medium effect size d = 0.50) indicated a minimum of 128 subjects per group; the present sample (512 per group) provides > 99% statistical power. The age range is 18–81, with a mean age of 49.5 ± 18.5 years (Table 1). The study population represents a North Cyprus-based university hospital population; it consists predominantly of Mediterranean-origin (Turkish-Cypriot, 78%) adult patients. Chronological age estimation (hereafter referred to as “age estimation”) was used in the study; the standard practice in forensic anthropology for biological profiling is the training and validation of models with chronological age based on registry documents [3, 6]. To eliminate inter-device variability, all images were obtained with the same Planmeca Proline CC (Helsinki, Finland) panoramic X-ray device using a standard protocol (70–77 kVp, 8–10 mA, 12–18 s, focal point-detector distance: 1.5 m).

Characteristic | Total (n = 1024) | Training (n = 717) | Validation (n = 154) | Test (n = 153)
Number of patients | 1024 | 717 (70.0%) | 154 (15.0%) | 153 (15.0%)
Sex - Male | 512 (50.0%) | 358 (49.9%) | 77 (50.0%) | 77 (50.3%)
Sex - Female | 512 (50.0%) | 359 (50.1%) | 77 (50.0%) | 76 (49.7%)
Age Mean ± SD | 49.5 ± 18.5 | 49.3 ± 18.4 | 49.8 ± 18.7 | 50.1 ± 18.6
Age Median [IQR] | 48 [33–65] | 48 [33–65] | 49 [34–66] | 49 [34–65]
Age Min - Max | 18–81 | 18–81 | 19–81 | 18–80
Age 18–33 years | 256 (25.0%) | 179 (25.0%) | 39 (25.3%) | 38 (24.8%)
Age 34–49 years | 256 (25.0%) | 179 (25.0%) | 38 (24.7%) | 39 (25.5%)
Age 50–65 years | 256 (25.0%) | 180 (25.1%) | 38 (24.7%) | 38 (24.8%)
Age 66–81 years | 256 (25.0%) | 179 (25.0%) | 39 (25.3%) | 38 (24.8%)

Image preprocessing and ground truth generation

To optimize the performance of deep learning models, a standard preprocessing pipeline was applied to all images.

Physical Calibration and Spatial Resolution

Images in original BMP format are 16-bit depth and average 2000 × 1000 pixels in size. The Planmeca Proline CC device has a pixel size of 0.079 mm/pixel, with a corresponding pixel area of (0.079 mm)² ≈ 0.006241 mm²/pixel. This calibration was used in converting all morphometric measurements (area, perimeter, height, width) from pixel values to millimetric physical dimensions [45]. Images were not resized; original spatial resolution was preserved.

Intensity Normalization

Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm was applied to make the faint boundaries of sinuses more distinct (clipLimit = 2.0, tileGridSize=(8, 8)). CLAHE provides local contrast enhancement while maintaining global histogram balance. Subsequently, images were converted to 8-bit PNG format (min-max normalization in the 0–255 range) to provide a standard input format for model training. For all models, intensity values were scaled to the [0,1] range using min-max normalization rather than z-score standardization.

Radiomic Feature Quantization Parameters: For radiomic feature extraction with PyRadiomics (version 3.0.1) [61, 62], the following standardization parameters were used: gray-level binWidth = 25 (10 bins in 8-bit intensity range), no intensity rescaling (already in [0,255] range), no voxel spacing resampling (2D image, non-isotropic panoramic geometry). All radiomic calculations were performed only within the segmented sinus ROI.

Ground truth annotation and inter-observer reliability

After preprocessing, the right and left maxillary sinuses in the entire dataset (1,024 images) were meticulously hand-labeled as polygons using the LabelMe v5.0.1 tool by a board-certified oral and maxillofacial radiologist with more than 15 years of experience (Observer 1: S.A.). These expert labels were accepted as “ground truth” for training and evaluation of models.

To assess inter-observer reliability, a randomly selected subset of 100 images (stratified by sex and age group, random seed = 42) was independently re-annotated by a second experienced observer (Observer 2: K.O., Professor of Oral Radiology with > 20 years of experience). The two observers were blinded to each other’s annotations. Inter-observer agreement was evaluated using the following metrics:

(a) Intraclass Correlation Coefficient (ICC): Two-way random-effects model, single measures, absolute agreement (ICC (2,1)) was calculated for morphometric measurements (area, perimeter, height, width) extracted from annotations of both observers(b) Dice Similarity Coefficient (DSC): Pixel-level overlap between Observer 1 and Observer 2 annotations was computed for each sinus(c) Bland-Altman Analysis: Systematic bias and 95% limits of agreement between observers were assessed for sinus area measurements.

Results demonstrated excellent inter-observer reliability (Table 8):

Sinus area: ICC = 0.968 [95% CI: 0.954–0.978], mean DSC = 0.934 ± 0.028.Sinus perimeter: ICC = 0.951 [95% CI: 0.932–0.966].Sinus height: ICC = 0.943 [95% CI: 0.921–0.960].Sinus width: ICC = 0.957 [95% CI: 0.940–0.970].

All ICC values exceeded 0.90, indicating “excellent” agreement according to the Cicchetti (1994) classification [50]. Bland-Altman analysis revealed no systematic bias (mean difference: −0.8 mm² [95% LoA: −22.1, 20.5 mm²] for sinus area). These results confirm the reliability of the ground truth annotations used in this study.

Experimental setup and methodological framework

Data Leakage Prevention: Preventing data leakage is critical to accurately evaluating the model’s generalization performance. For this purpose, the dataset was split according to patient identification numbers. This patient-level split guarantees that all data from one patient is in only one set (training, validation, or test). The dataset was divided into 70% training (717 patients), 15% validation (154 patients), and 15% test (153 patients) using the StratifiedGroupKFold method, which ensures that age and sex are preserved in each set. Split validation: It was verified that there was no patient-level overlap (no data leakage) between training, validation, and test sets.

Data Augmentation

To prevent overfitting and increase the robustness of models to anatomical variations, the following data augmentation techniques were applied to the training set.

For YOLOv8 and U-Net

Random rotation (± 15°), random scaling (0.90–1.10), random brightness/contrast adjustment (± 20%), horizontal flip disabled (laterality preservation), elastic deformation (σ = 10, α = 150).

For nnU-Net: The nnU-Net v2 automatic augmentation policy was used (rotation: ±15°, scaling: 0.7–1.4, Gaussian noise: σ = 0.1, Gaussian blur: σ = 0.5–1.5). For laterality preservation, mirror_axes=[] was set in nnUNetPlans.json.

Transfer Learning

Domain-specific pre-training has been shown to significantly improve performance in medical imaging tasks [25, 26]. Accordingly, segmentation models were pre-trained on approximately 5,300 panoramic dental radiographs in the “Teeth Detection” dataset [44] from the Roboflow Universe platform. These images were only used to improve anatomical recognition capacity and contain no demographic information, posing no data leakage risk.

Pipeline 1: YOLOv8-hybrid approach

This pipeline consists of four stages: coarse segmentation, boundary refinement, multi-modal feature extraction, and ensemble-based prediction.

Stage 1: Coarse Segmentation: The YOLOv8n-seg model (3.2 M parameters) was trained with 5-fold cross-validation, for each fold up to 300 epochs with SGD (momentum = 0.937, weight_decay = 5 × 10⁻⁴), initial learning rate of 0.01 and CosineLR scheduling, using Dice + BCE (1:1) composite loss. Early stopping was applied when there was no improvement in validation mAP@50 for 50 epochs.

Stage 2: Boundary Refinement: A lightweight 2D U-Net [24] (encoder/decoder: 5 blocks each, concatenation-type skip connections) refined coarse masks. Trained with AdamW (lr = 10⁻³, weight_decay = 3 × 10⁻⁴), batch size 8, Dice loss for up to 300 epochs with early stopping (patience = 30).

Stage 3: Feature Extraction: >120 features extracted in three groups: morphometric (area, perimeter, shape indices, Hu moments, height/width; ≥28 features), radiomic texture (GLCM, GLRLM, GLSZM, GLDM; ≥80 features with PyRadiomics v3.0.1), and bilateral asymmetry (area differences, ratios, indices; ≥12 features).

Stage 4: Prediction: CatBoost [32] for sex classification (depth = 6, learning_rate = 0.05, iterations = 500, l2_leaf_reg = 3.5, bootstrap_type=’Bayesian’) optimized with Optuna [34] (300 trials). XGBoost [33] for age regression (max_depth = 8, learning_rate = 0.03, n_estimators = 800).

Feature Selection Strategy and Overfitting Control: Given the high-dimensional feature space (> 120 features relative to sample size), multiple complementary strategies were employed to mitigate overfitting. First, both CatBoost and XGBoost incorporate built-in L2 regularization (l2_leaf_reg = 3.5 and reg_lambda = 1.0, respectively) and tree-depth constraints (max_depth = 6 and 8) that inherently perform implicit feature selection by penalizing complex splits on non-informative features [32, 33]. Second, Bayesian hyperparameter optimization via Optuna (300 trials) with 5-fold cross-validation was used to identify the regularization strength that minimizes validation loss, thereby controlling model complexity [34]. Third, SHAP-based post-hoc feature importance analysis [38] was conducted across all five folds to assess feature stability: the top-15 features for sex classification showed 87% overlap (13/15 features consistent) across folds, and for age estimation 80% overlap (12/15), indicating robust feature utilization rather than fold-specific overfitting. Fourth, the nested cross-validation design (outer 5-fold for evaluation, inner Optuna loop for tuning) ensures that hyperparameter selection does not leak into performance estimates. No explicit dimensionality reduction (e.g., PCA) was applied, as tree-based ensemble methods are inherently robust to irrelevant features and benefit from access to the full feature space for interaction detection [32, 33, 56].

Pipeline 2: nnU-Net v2 approach

The nnU-Net v2 framework [22] was deliberately used in its default auto-configuration mode (“fingerprinting”) for architecture selection, preprocessing, and training schedule, which is a principled methodological choice to ensure full reproducibility and to leverage the framework’s self-configuring strengths as intended by its developers [52]. The automatically determined architecture was a 2D full-resolution U-Net (~ 31.2 M parameters). Task-specific modifications were limited to: (a) disabling mirror augmentation (mirror_axes=[] in nnUNetPlans.json) to preserve sinus laterality information, and (b) domain-specific transfer learning from a dental radiograph corpus (see Transfer Learning above). All other configurations — including 5-fold cross-validation, 1000 epochs per fold, SGD (momentum = 0.99, nesterov=True), PolyLR scheduler (power = 0.9), batch size 2, 512 × 512 patch size, and Dice + CE (1:1) loss with deep supervision — were retained from nnU-Net v2’s automatic configuration. The same feature set was extracted and input to XGBoost [33] models. The comparative architecture of both pipelines is illustrated in Fig. 1.

Fig. 1: Comparative deep learning pipeline architecture. Upper panel: YOLOv8-Hybrid pipeline (coarse segmentation → U-Net refinement → feature extraction → CatBoost/XGBoost prediction). Lower panel: nnU-Net v2 pipeline (auto-configured 2D U-Net → feature extraction → XGBoost prediction). Linear measurements and maxillary sinus labeling (a1, a2 = sinus height; b1, b2 = sinus width) are shown in the preprocessing stage

External validation

To evaluate the generalizability of the developed pipelines beyond the primary dataset, an external validation was conducted on an independent dataset. Fifty panoramic radiographs (25 males, 25 females; age range: 18–81 years, stratified by age decade) were randomly selected (seed = 42) from the same institutional archive but a temporally distinct patient cohort (Near East University, 2014–2016 cohort; temporal external validation) that was not included in the training, validation, or test sets. These images were acquired with the same Planmeca Proline CC device, ensuring methodological consistency while maintaining temporal independence from the primary dataset (2011–2013).

Both trained pipelines (YOLOv8-hybrid and nnU-Net v2) were applied to these 50 images in fully automated inference mode without any retraining or fine-tuning. The predicted segmentation masks were visually reviewed by both observers, and the predicted sex and age values were compared against ground truth records. External validation results are reported in the Results section.

Performance evaluation and statistical analysis

Segmentation: mAP@50 and mAP@50–95 for YOLOv8; DSC and HD95 (calibrated to 0.079 mm/pixel) for nnU-Net. Sex classification: accuracy, AUC, precision, sensitivity, F1-score, Cohen’s kappa, Matthews correlation coefficient. Age estimation: MAE, RMSE, R², MAPE. Uncertainty: patient-level stratified bootstrap with BCa 95% CI over 10,000 iterations [35]. AUC comparison: DeLong test [36]. Dependent metrics: Wilcoxon signed-rank test. Independent groups: Mann-Whitney U test. Effect size: Cohen’s d [37]. Age bias: Bland-Altman analysis [39]. Inter-observer reliability: ICC(2,1) [50].

Model interpretability (SHAP analysis)

SHAP (SHapley Additive exPlanations) [38] TreeExplainer was applied to the test set (153 patients, 306 sinuses). Global importance was summarized by mean absolute SHAP value, and the top 15 features were identified for each task.

Software and hardware

Python 3.11.7, PyTorch 2.2.1, scikit-learn 1.4.2, Optuna 3.6.0, CatBoost 1.2.3, XGBoost 2.0.3, PyRadiomics 3.0.1, nnU-Net v2.3.1, pingouin 0.5.4 (ICC). Fixed random seed = 42. GPU: NVIDIA RTX 4090 (24 GB), CPU: AMD Ryzen 9 7950X, RAM: 64 GB DDR5, Storage: 2 TB NVMe SSD.