Hydronephrosis is a common renal disease in children which can lead to a series of complications, and ultrasonography is a basic examination usually performed on suspected hydronephrosis patients. If we can use deep learning approaches to judge and grade the disease in the ultrasonic examination stage, we can save a lot of manpower, medical resources, money, and help the suffered patients. For the semantic segmentation of kidney ultrasound image, we designed an Attention-based Pyramid Scene Parsing Network (A-PSPNet), the core of which is the basic feature extraction network combining Convolutional Block Attention Module (CBAM) and pyramid analysis module. Experiments were carried out on a hydronephrosis dataset containing 1850 annotated ultrasound images, including the arrangement of attention units, statistical computing power, and comparison of the effectiveness between the benchmark and our proposed method. Our constructed model achieved better segmentation performance than benchmarks with only little extra overhead, which validated the lightweight and effectiveness of the model.