Fully homomorphic encryption (FHE) has attracted much attention because computations can be directly performed on ciphertexts. This work explores the hardware architecture of polynomial multiplication defined in BGV-FHE, targeting the applications of aggregate plaintext using cyclotomic polynomials. We show how to effectively combine the characteristics of cyclotomic polynomials and the prime-factor FFT algorithm to obtain a novel design derived by the concept of Chinese Remainder Theorem. Experimental results reveal that a significant speedup in terms of operation reduction can be achieved by adopting the proposed schemes as compared to existing works assuming a comparable security level. For example, about 2.44 and 6.34 times improvement in the total number of required modular addition and multiplication, respectively, can be obtained by using 32 one-bit aggregate slots as compared to Chen's work when the 21845-th cyclotomic polynomial is considered. The improvement could be huge if all of the available slots are involved in applications.