High Efficiency Video Coding (HEVC) introduces a number of new coding tools such as 35 intra prediction modes, improved coding unit structure and SAO filter and so on to improve the compression capacity of encoding. Compared with H.264/AVC, the compression performance increased by nearly 50% at the cost of the algorithm complexity increasing significantly. One of the most time-consuming part is the CU partition process which taking over 50% of the encoding time. In this paper, in order to reduce the computational pressure on the encoder side, we design an algorithm based on contour detection and SVM classifier. First, we adopt an improved Canny algorithm which is special designed for CU partition task. Then, based on SVM classifier, we introduce a new CU partition decision strategy. Experimental results show that the proposed algorithm achieves about 50% time saving on average with negligible loss of coding efficiency compared with HEVC reference software HM16.0.