Why, whn shorting a stock do u owe th lender the dividend? Doen't that mean the company doesn't end up paying?

So I understand if say a stock on the date before the ex-date is at $10 with a $1 dividend. Then the next day the adjustment is made and the stock drops to $9. Now the short seller owes the lender the amount of the dividend (in this example $1) but shouldn't that be negated by the company owing 'someone' the dividend anyway. The company must still have to pay the dividend declared so why does the short seller loose the money?