Results

Model building and classification results

The classification models were trained using different combinations of extracted features, including AAC, DDE, UniRep, and their fused feature sets (Table 1). The RF classifier, configured with 100 decision trees and a fixed random seed (42), demonstrated superior performance among the evaluated models. Using feature bagging and parallel processing, the RF model achieved an accuracy of 77.4% and the highest MCC of 0.5509 when trained on the fused AAC+DDE+UniRep feature set.

Features name | Concatenate features | Final fused feature
AAC features + DDE feature | F1 + F2 | F4
AAC features + UniRep embeddings | F1 + F3 | F5
DDE features + UniRep embeddings | F2 + F3 | F6
AAC features + DDE features + UniRep embeddings | F1 +F2 + F3 | F7

To evaluate the predictive performance of the models, we assessed precision, recall, F-measure (F1-score), Matthews correlation coefficient (MCC), accuracy, and area under the curve (AUC) for each classification algorithm. Among the evaluated classifiers, RF achieved the highest F1-score and MCC, indicating strong predictive capability for distinguishing virulent from non-virulent proteins across different class sizes. Precision and recall analyses indicated a low false-positive rate, thereby reducing the likelihood of incorrectly classifying non-virulent proteins as virulent and minimizing unnecessary downstream experimental validation. Furthermore, comparison of individual and fused feature sets demonstrated that combining AAC, DDE, and UniRep features improved the sensitivity (recall) and specificity of virulence prediction, highlighting the potential value of integrating complementary compositional, sequential, and learned protein representations. Collectively, these evaluation metrics provide a comprehensive assessment of model performance and support the utility of FunVFPred for prioritizing candidate VFs in fungi.

The ANN model, composed of two hidden layers with 64 and 32 neurons, respectively, employed ReLU activation functions and dropout regularization (30% and 20%) to reduce overfitting. With a sigmoid activation function in the output layer, the ANN model achieved an accuracy of 75% when trained using the combined AAC+UniRep feature set.

The MLP model, incorporating three hidden layers with 64, 32, and 16 neurons, respectively, followed a similar activation strategy and achieved an accuracy of 75% when trained using the DDE+UniRep feature set.

The DNN model extended the MLP architecture by incorporating four fully connected layers, batch normalization, and progressively decreasing dropout rates. This configuration yielded an accuracy of 75.81% when trained using the AAC+UniRep feature set, highlighting the model’s ability to capture complex feature interactions. Despite its deeper architecture, the DNN model did not surpass the RF model in terms of overall accuracy and MCC.

Overall, the RF model consistently outperformed the other ML and DL classifiers evaluated in this study. It demonstrated robust performance in classifying virulent and non-virulent fungal proteins, achieving the highest accuracy of 77.4% (Figure 2; Table 2) and the highest MCC of 0.5509 (Figure 3; Table 2).

Figure 2: Comparison of accuracies of different machine learning and deep learning modelsThis figure shows a comparison of validation accuracies across different machine learning and deep learning models used in the study. The models include random forest (RF), artificial neural network (ANN), multi-layer perceptron (MLP), and deep neural network (DNN). The comparison highlights how each model performed in terms of accuracy, helping to identify which approach was most effective for the prediction task.

Classifier | Features | Accuracy | MCC | AUC under the ROC curve
RF | AAC | 0.6935 | 0.3879 | 0.7578
RF | DDE | 0.6935 | 0.3945 | 0.8034
RF | UNIREP | 0.7258 | 0.4603 | 0.8173
RF | AAC+DDE | 0.6774 | 0.3565 | 0.7563
RF | AAC+UNIREP | 0.7016 | 0.4075 | 0.8002
RF | DDE+UNIREP | 0.7419 | 0.4861 | 0.8055
RF | AAC+DDE+UNIREP | 0.7741 | 0.5509 | 0.8225
ANN | AAC | 0.6048 | 0.2104 | 0.7310
ANN | DDE | 0.6613 | 0.3241 | 0.7391
ANN | UNIREP | 0.7258 | 0.4518 | 0.7994
ANN | AAC+DDE | 0.6774 | 0.3595 | 0.7568
ANN | AAC+UNIREP | 0.7500 | 0.5001 | 0.8301
ANN | DDE+UNIREP | 0.7258 | 0.4526 | 0.8119
ANN | AAC+DDE+UNIREP | 0.7419 | 0.4880 | 0.8439
MLP | AAC | 0.6693 | 0.34 | 0.7393
MLP | DDE | 0.6612 | 0.33 | 0.6992
MLP | UNIREP | 0.7419 | 0.48 | 0.8251
MLP | AAC+DDE | 0.6854 | 0.38 | 0.7319
MLP | AAC+UNIREP | 0.7258 | 0.45 | 0.8344
MLP | DDE+UNIREP | 0.7500 | 0.51 | 0.8069
MLP | AAC+DDE+UNIREP | 0.7177 | 0.44 | 0.7950
DNN | AAC | 0.6855 | 0.3722 | 0.7690
DNN | DDE | 0.6371 | 0.2751 | 0.7106
DNN | UNIREP | 0.7500 | 0.5001 | 0.8163
DNN | AAC+DDE | 0.6452 | 0.2909 | 0.7447
DNN | AAC+UNIREP | 0.7258 | 0.4518 | 0.8371
DNN | DDE+UNIREP | 0.7177 | 0.4355 | 0.8040
DNN | AAC+DDE+UNIREP | 0.7661 | 0.5357 | 0.8439

Figure 3: Performance metrics of the random forest classifier between different featuresThe figure presents the performance metrics of the random forest (RF) classifier evaluated across different feature sets, including AAC, DDE, UniRep, and their merged combinations. The histogram illustrates key evaluation metrics such as accuracy, MCC, AUC-ROC, precision, recall, F1 score, sensitivity, and specificity. The comparison helps to assess how each feature type contributes to the model’s predictive performance.

Feature importance analysis (RF contributions)

Of the top 50 features identified from the RF classifier trained on 2,320 features, 48 were UniRep embeddings, while the remaining two were DDE descriptors. This finding indicates that UniRep embeddings captured a substantial proportion of the predictive information encoded in the latent sequence representations. The two DDE descriptors may also capture biologically relevant sequence patterns associated with dipeptide distributions linked to virulence.

Virulence-associated sequence pattern analysis

Analysis of AAC revealed an increased abundance of five amino acids (T, A, Q, G, and P) associated with virulence-related characteristics, including adhesion, biofilm formation, and host interactions (Figure 4A). DDE analysis identified recurring dipeptide motifs, including QQ, NN, SS, and TT, suggesting potential associations with virulence-associated protein characteristics (Figure 4B). Together, these analyses provide insights into sequence-level patterns that may contribute to the predictive performance of the selected model features.

Figure 4: Characteristic sequence patterns associated with fungal virulence(A) Amino acid composition (AAC). Comparison of the relative frequencies of amino acids between virulent and non-virulent proteins.(B) Dipeptide deviation from expected (DDE) mean. Comparison of the relative frequencies of dipeptides between virulent and non-virulent proteins.

Performance across VF families

Using Gene Ontology annotations, predicted virulent proteins from different fungal species (Table 3) were grouped into functional families. Model sensitivity was high for adhesins/biofilm-associated proteins and hydrolytic enzymes, with accuracies exceeding 0.83. In contrast, metabolic and signaling regulators exhibited higher false-positive misclassification rates, potentially reflecting greater sequence heterogeneity within these functional groups (Table 4; Figure 5). These findings indicate that FunVFPred can identify sequence patterns associated with biologically relevant virulence mechanisms in human fungal pathogens, including those represented by experimentally characterized proteins such as ALS3, HWP2, BUD4, and VPS11.

Fungal species | Number of virulent (positive) proteins | Number of virulent (positive) proteins after redundancy removal
C. albicans | 692 | 508
C. glabrata | 69 | 64
C. tropicalis | 04 | 04
C. parapsilosis | 18 | 15
C. dubliniensis | 23 | 21
C. orthopsilosis | 06 | 03
C. glycerinogenes/Pichia kudriavzevii | 01 | 01
Total | 813 | 616

VF family | Total proteins | Correct predictions | Incorrect predictions | Accuracy
Adhesin/Biofilm | 18 | 15 | 3 | 0.833
Non_VF/Other | 193 | 133 | 60 | 0.689
Signaling/transcription regulator | 27 | 15 | 12 | 0.556
Hydrolytic enzyme | 6 | 6 | 0 | 1.0
Hydrolytic enzyme; signaling/transcription regulator | 1 | 1 | 0 | 1.0
Metabolic enzyme | 1 | 0 | 1 | 0.0

Figure 5: Distribution of proteins and prediction accuracy across virulence factor (VF) familiesThe pie chart represents the proportion of proteins in each VF family, highlighting both virulent and non-virulent groups.

RF performance on an independent dataset: Assessment of generalizability

The RF classifier outperformed the other evaluated ML and DL models (ANN, MLP, and DNN) on the validation dataset. To assess the broader applicability of FunVFPred, an independent dataset was constructed using sequences from fungal species that were not included in the RF training dataset. The model achieved an overall accuracy of 68.7% using the UniRep and DDE+UniRep feature sets, with an MCC of 0.4803, demonstrating the ability of FunVFPred to generalize to sequences from previously unseen fungal organisms (Table 5).

Features | Accuracy | MCC | AUC-ROC | Precision | Recall | F1-score | Sensitivity | Specificity
AAC | 0.6875 | 0.4045 | 0.8437 | 0.6363 | 0.8750 | 0.7368 | 0.8750 | 0.5000
DDE | 0.6250 | 0.2581 | 0.7265 | 0.6000 | 0.7500 | 0.6666 | 0.7500 | 0.5000
UNIREP | 0.6875 | 0.4803 | 0.9140 | 0.6153 | 1.0000 | 0.7619 | 1.0000 | 0.3750
AAC+DDE | 0.6875 | 0.3779 | 0.8046 | 0.6666 | 0.7500 | 0.7058 | 0.7500 | 0.6250
AAC+UNIREP | 0.6250 | 0.2886 | 0.8437 | 0.5833 | 0.8750 | 0.7000 | 0.8750 | 0.3750
DDE+UNIREP | 0.6875 | 0.4803 | 0.8203 | 0.6153 | 1.0000 | 0.7619 | 1.0000 | 0.3750
AAC+DDE+UNIREP | 0.6250 | 0.3779 | 0.8125 | 0.5714 | 1.0000 | 0.7272 | 1.0000 | 0.2500

RF prediction performance on validation and independent datasets using 5-fold cross-validation

The RF model achieved high predictive performance using fused feature sets, with the AAC+DDE+UniRep combination yielding an accuracy of 77.4% and an MCC of 0.5509 for distinguishing virulent from non-virulent fungal proteins in the validation dataset. To further assess model robustness and generalizability, 5-fold cross-validation was performed on both the validation and independent datasets. The corresponding MCC values for the validation and independent datasets were 0.4681 and 0.7142, respectively. The accuracies for these datasets were 73.4% using DDE+UniRep and 85.7% using UniRep, respectively, as shown in Figure 6 and Tables 6 and 7. These results indicate that the RF classifier provides an effective approach for predicting fungal virulence-associated proteins. The incorporation of pre-trained protein embeddings such as UniRep contributed to improved predictive performance, with UniRep consistently enhancing results when used either independently or in combination with traditional sequence-derived features.

Figure 6: Performance of RF predictions for validation and independent datasets with 5-fold cross-validationThe figure illustrates the performance of the FunVFPred tool using the random forest (RF) classifier, evaluated through 5-fold cross-validation. The results are shown for both the validation and independent sets, demonstrating the robustness and generalizability of the model’s predictions.

Features | Accuracy | MCC | AUC-ROC | Precision | Recall | F1-score | Sensitivity | Specificity
AAC | 0.6531 | 0.3073 | 0.7475 | 0.6413 | 0.6860 | 0.6629 | 0.6860 | 0.6206
DDE | 0.6878 | 0.3778 | 0.7221 | 0.7105 | 0.6279 | 0.6666 | 0.6279 | 0.7471
UNIREP | 0.7109 | 0.4231 | 0.8108 | 0.6956 | 0.7441 | 0.7191 | 0.7441 | 0.6781
AAC+DDE | 0.6994 | 0.3990 | 0.7719 | 0.7073 | 0.6744 | 0.6904 | 0.6744 | 0.7241
AAC+UNIREP | 0.6936 | 0.3876 | 0.7959 | 0.6853 | 0.7093 | 0.6971 | 0.7093 | 0.6781
DDE+UNIREP | 0.7341 | 0.4681 | 0.8090 | 0.7325 | 0.7325 | 0.7325 | 0.7325 | 0.7356
AAC+DDE+UNIREP | 0.7225 | 0.4463 | 0.8074 | 0.7065 | 0.7558 | 0.7303 | 0.7558 | 0.6896

Features | Accuracy | MCC | AUC-ROC | Precision | Recall | F1-score | Sensitivity | Specificity
AAC | 0.5238 | 0.023 | 0.7772 | 0.5 | 0.2 | 0.2857 | 0.2 | 0.8181
DDE | 0.5714 | 0.1454 | 0.6636 | 0.5454 | 0.6 | 0.5714 | 0.6 | 0.5454
UNIREP | 0.8571 | 0.7142 | 0.8367 | 0.8571 | 0.8571 | 0.8571 | 0.8571 | 0.8571
AAC+DDE | 0.5238 | 0.055 | 0.5090 | 0.5 | 0.6 | 0.5454 | 0.6 | 0.4545
AAC+UNIREP | 0.7142 | 0.4272 | 0.7454 | 0.7 | 0.7 | 0.7 | 0.7 | 0.7272
DDE+UNIREP | 0.7619 | 0.5272 | 0.7545 | 0.7272 | 0.8 | 0.7619 | 0.8 | 0.7272
AAC+DDE+UNIREP | 0.7142 | 0.4272 | 0.7454 | 0.7 | 0.7 | 0.7 | 0.7 | 0.7272

Although virulence prediction tools such as VirulentPred, EffectorP, and FungalRV have demonstrated utility for predicting bacterial virulence proteins or fungal effectors, respectively, their specific training objectives and biological contexts differ from those of human fungal virulence prediction. Therefore, these tools were not included in direct performance comparisons. Instead, the statistically validated performance metrics of FunVFPred, obtained through 5-fold cross-validation and reported as mean ± standard deviation, provide an assessment of its predictive performance for virulence-associated proteins in human-pathogenic fungi.