But when Emily dumped him,
she said specifically that she had more confidence in Jef. That seems pretty clear to me

How else could he have interpreted it? I think he went to Charlotte to convince Emily she made a mistake, knowing that Jef was the one she chose. His leaving the journal just solidifies it IMO. He wanted to be the final one and it didn't matter that she'd picked Jef in his eyes.
Arie did say that Jef smoothed things over and gave him closure because Emily was too upset to do it. So that tells me that he knew she'd picked Jef but was hoping for a second chance (aka Jason, Molly and Melissa switcheroo). Just my opinion though.