The rising of smart aquaculture offers the possibility to maintain or even increase the production of shrimps with less cost and environmental impact. The most critical factors that affect the growth rate of shrimp is the efficiency of feeding. Therefore, identify the location and number of shrimps are important for optimizing feeding approach. Recently, computer vision has been applied in many fields to extract useful information from images. To increase the production with less cost, computer vision is also used to monitor the products of aquaculture. Usually, to ensure the accuracy of computer vision, a relatively high-quality picture is required. Since shrimp is benthic animal and the water of shrimp pond is usually turbidity, the quality of underwater photos is usually poor. Compared to optical photos, the imaging mechanism of ultrasound is less affected by the above effects. Therefore, in this study, an innovative white shrimp monitoring system combining ultrasound imaging technology and YOLOv4 is proposed. The database of shrimps' ultrasonic images is built by imaging the different parts of the shrimps from various viewing angles with a linear ultrasonic probe. Through moving the ultrasound probe by a 2-axis motor, the images could be captured from two orthogonal viewing angles within a 12.69cm x 12.69cm region of interest. After 260 images are captured, the locations of shrimps in each image would be recognized and the centers of bounding boxes will be marked. After that, a custom grouping method based on spectral clustering algorithm would be applied to distinguish the number of shrimps. Finally, the length and location of shrimps would be calculated from the best corresponding fitting line. After examining the proposed system with 30 sets of different number, size, and location of shrimps, the accuracy of identifying the number of shrimps is 97.3% along with 8.99% averaged length error and 0.97 cm averaged position error. These results demonstrate the feasibility of monitoring shrimp farms using proposed system.