In a two-way relay network, two terminals exchange information over a shared wireless half-duplex channel with the help of a relay. Due to its fundamental and practical importance, there has been an increasing interest in this channel. However, surprisingly, there has been little work that characterizes the fundamental tradeoff between the communication reliability and transmission rate across all signal-to-noise ratio (SNR) ratios. In this paper, we consider amplify-and-forward (AF) two-way relaying due to its simplicity. We first derive the random coding error exponent for the link in each direction. From the exponent expression, the capacity and cutoff rate for each link are also deduced. We then put forth the notion of the bottleneck error exponent, which is the worst exponent decay between the two links, to give us insight into the fundamental tradeoff between the rate pair and information-exchange reliability of the two terminals.