Electric vehicles (EVs) offer an attractive long-term solution to reduce the dependence on fossil fuel and greenhouse gas emission. However, a fleet of EVs with different EV battery charging rate constraints, that is distributed across a smart power grid network requires a coordinated charging schedule to minimize the power generation and EV charging costs. In this paper, we study a joint optimal power flow (OPF) and EV charging problem that augments the OPF problem with charging EVs over time. While the OPF problem is generally nonconvex and nonsmooth, it is shown recently that the OPF problem can be solved optimally for most practical power grid networks using its convex dual problem. Building on this strong duality result, we study a nested optimization approach to decompose the joint OPF and EV charging problem. We characterize the optimal offline EV charging schedule to be a valley-filling profile, which allows us to develop an optimal offline algorithm with computational complexity that is significantly lower than centralized interior point solvers. Furthermore, we propose a decentralized online algorithm that dynamically tracks the valley-filling profile. Our algorithms are evaluated on the IEEE 14 bus system, and the simulations show that the online algorithm performs almost near optimality (< 1% relative difference from the offline optimal solution) under different settings.