In brachytherapy, the segmentation accuracy of the target tumor and surrounding organs is very important. In recent years, deep learning models have improved the performance of organ segmentation and have been widely used. However, it is still a huge challenge for some organs with variable shapes. The basic idea of the work presented in this paper is to accurately divide the rectal computed tomography image dataset so that patients can obtain more accurate brachytherapy. In this work, we used 3D U-Net and Long ShortTerm Memory (LSTM) to improve the accuracy of rectal segmentation. This model was trained and tested on the rectal computed tomography image dataset, which contains 51 patients undergoing radiation therapy. The dice coefficient is used as the evaluation index in all results of organ segmentation. After experiments are done, it can be seen that the proposed method has good performance in rectal segmentation.