We describe a more efficient algorithm to compute p-adic Coleman integrals on odd degree hyperelliptic curves for large primes p. The improvements come from using fast linear recurrence techniques when reducing differentials in Monsky-Washnitzer cohomology, a technique introduced by Harvey arXiv:math/0610973 when computing zeta functions. The complexity of our algorithm is quasilinear in sqrt(p) and is polynomial in the genus and precision. We provide timings comparing our implementation with existing approaches.