Uncovering the transcriptional modules with context-specific cellular activities or functions is important for understanding biological network, deciphering regulatory mechanisms and identifying biomarkers. In this paper, we propose to use the penalized matrix decomposition (PMD) to discover the transcriptional modules from microarray data. With the sparsity constraint on the decomposition factors, metagenes can be extracted from the gene expression data and they can well capture the intrinsic patterns of genes with the similar functions. Meanwhile, the PMD factors of each gene are good indicators of the cluster it belongs to. Compared with traditional methods, our method can cluster genes of similar functions but without similar expression profiles. It can also assign a gene into different modules. Moreover, the clustering results by our method are stable and more biologically relevant transcriptional modules can be discovered. Experimental results on two public datasets show that the proposed PMD based method is promising to discover transcriptional modules.