Coronary artery analysis plays an important role in the diagnosis and treatment of coronary heart disease. Coronary artery segmentation, as an important part of quantitative research on coronary heart disease, has become the main topic in coronary artery analysis. In this paper, a deep convolutional neural network (CNN) based method called Coronary Artery U-net(CAU-net) is proposed for the automatic segmentation of coronary arteries in digital subtraction angiography (DSA) images. CAU-net is a variant of U-net. Based on the observation that coronary arteries are composed of many vessels with the same appearance but different thicknesses, a novel multi-scale feature fusion method is proposed in CAU-net. Besides, a new dataset is proposed to solve the problem of no available public dataset on coronary arteries segmentation, which is also one of our contributions. Our dataset contains 538 image samples, which is relatively large compared with the public datasets of other vessel segmentation tasks. In our dataset, a new labeling method is applied to ensure the purity of the labeling samples. From the experimental results, we prove that CAU-net can make significant improvements compared with the vanilla U-net, and achieve the state-of-the-art performance compared with other traditional segmentation methods and deep learning methods.