Fig. 2

Application to the human brain cortex 10X Visium dataset. A Layer annotations of the ST sample named “151,673” in the DLPFC dataset. B UMAP representation of the reference scRNA-seq dataset. C Box plots displaying the calculated AUC and ER values for the estimated cell type distribution in all 12 ST samples. Each box comprises 50 datapoints that represent scores for 10 layer-specific cell types in 5 repeated experiments, and ranges from the third and first quartiles with the median as the horizontal line, while whiskers represent 1.5 times the interquartile range from the lower and upper bounds of the box. D Estimated proportion heatmaps of 3 layer-specific excitatory neurons by each deconvolution method. E Box plots showing the calculated AUC and ER values for the estimated cell type distributions in the “151,673” ST sample. “original” includes all 28 cell types in the scRNA-seq dataset. “merge” indicates that the cell subtypes were merged before model training except for the 10 excitatory neurons. “del Inhib” indicates that inhibitory neurons were deleted from the scRNA-seq dataset. F UMAP representation of deconvolution results from different methods under the “original” and “del Inhib” conditions. G Scatter plots of cluster centroid distances in the UMAP computed for each method under different condition pairs. H Clustering results of the “151,673” ST sample given by GraphST. I Ablation study on the “151,673” ST sample. “cluster” represents the situation where LETSmix leverages clustering results given by GraphST as the layer annotation information. “wo_LETS” represents the situation where LETSmix ignores all spatial context information, and “wo_DA” represents the situation where LETSmix is trained without the implementation of the domain adaptation strategy. Error bars represent the mean ± standard deviation. An independent t-test was performed between LETSmix and the other ablated models. Statistical significance is indicated above the bars (ns: not significant, ****P-value < 0.0001)