You are all forgetting one possible scenario Boris impresses the Tory faithfull and they win the by election easily with an increased majority, unlikely but possible.
It's very unlikely, for three reasons (and none of them anything to do with Johnson). First of all, this has been a Lib Dem seat for most of the past 35 years (and before that, it swung between Labour and Tory). The Cons won it when the LD vote collapsed following the coalition but it's very far from being a solid Tory seat. And, with the LDs riding high in the polls, they'll be odds-on to take it back.
Second, the Brexit Party will hoover up votes from the Conservatives - any no-dealers will have a natural home. The vast majority will be Tories.
Finally, when an MP loses a seat in disgrace, there is always a slight backlash against that party in the subsequent by-election ... and there's often a strong vote against the government and for a party outside the big two in by-elections too.
I think this one is a shoo-in for the Lib Dems, unless Vince Cable is caught shagging a sheep in the Radnorshire hills or something