Melanoma immune checkpoint inhibitor (ICI) response is related to complex biological pathways. A deep learning (DL) model has powerful fitting ability and transformer structure has interactive information extraction function. We designed a transformer neural network fed with multi-gene mRNA expression quantities to predict melanoma ICI response and explored gene interactions from the embedding layer.
We built our training/validation/testing datasets from four open databases, 208 pre-ICI and 58 post-ICI samples were included in training/validation dataset and testing dataset, respectively. Every sample has 160-dimension mRNA expression quantities sequenced from tumor tissues. The 160 genes were previously described as potentially associated with melanoma, inflammation, immunity, and the PD-L1/CTLA4 pathways. We designed a DL model named AMU with the transformer encoder followed by a convolutional neural network (CNN). We applied the method of upsampling for data enhancement and set the 20-dimension gene embeddings. SVM, Random Forest, AdaBoost, XGBoost and a simple CNN we designed were token as competing models. In model interpretation work, gene feature vectors were extracted from the embedding layer and gene interactions were calculated by t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm, then the gene interactions were compared with that inquired in the Functional Protein Association Network (STRING, https://cn.string-db.org/).
AMU showed the preferred performance with the area under the curve (AUC) of 0.941 and the mean average precision (mAP) of 0.960 in validation dataset and AUC of 0.672, mAP of 0.800 in testing dataset, respectively. In model interpretation, the TNF-TNFRSF1A pathway was indicated as a key pathway to influence melanoma ICI response. Further, gene features learned from AMU showed a local similarity with STRING.
The DL model built with transformer encoder structure has strong power to process mRNA expression data and precisely predicted the melanoma ICI response. Meanwhile, gene vector representation learned from trainable embedding layer is promising for DL application in the biomedical field.
Y. Yin.
Has not received any funding.
All authors have declared no conflicts of interest.