There was the concepts of history to consider, there was the time distortion and ripple affects that Byron's presence in the 18th century, and there was also the matter of Reinette. The Doctor had mourned for her, privately, for some time, and seeing her again (on top of the ripples it would cause) would only reopen a scabbed wound.
He scowled a bit at himself, at his stupidly bendable will to Rose, and flipped a few switches on the TARDIS, setting courses and checking fluid levels. If all went well, he'd have Byron there and back in a few moments. If a fluid level was down or something malfunctioned, he might've been stuck with the man for longer.