Investigating the link between mitochondrial function and its physical structure is a hot topic in neurobiology research. With the rapid development of Scanning Electron Microscope (SEM), we can look closely into the fine mitochondrial structure with high resolution. Consequently, many meaningful researches have focused on how to detect and segment the mitochondria from EM images. Due to the complex background, hand-crafted features designed by traditional algorithms cannot provide satisfying results. In this paper, we propose an effective deep neural network improved from Mask R-CNN to produce the detection and segmentation results. On this base, we use the morphological processing and mitochondrial context information to rectify the local misleading results. The valuation was performed on two widely used datasets (FIB-SEM and ATUMSEM), and the results demonstrate that the proposed method has comparable performance than state-of-the-art methods.