We consider the problem of maximizing the time average throughput in energy harvesting networks with dynamic channel state acquisition. Previous works on energy harvesting systems do not account for the energy consumed to acquire the channel state information(CSI). However, when the nodes have a limited capacity batteries and the energy available in the battery is time varying, it becomes crucial to account for the energy spent in acquiring the CSI. In such a scenario, the available energy in the battery must be optimally divided between CSI acquisition and transmission. We model the energy harvesting battery as an energy queue and use the technique of Lyapunov optimization combined with the idea of weight perturbation to jointly optimize the channel probing and transmission decisions. Since the optimization problem corresponding to the optimal CSI acquisition decision in each time slot is a combinatorial problem, we provide a low-complexity scheme to solve this in the special case ON-OFF fading channels with binary power allocation scheme, and prove that this algorithm is optimal. Finally, we provide numerical results and show that when the mean rate of the harvested energy is low, it becomes crucial to account for the energy consumed in acquiring the CSI.