If I upgrade in the middle of a month, do I pay for both plans?

No. The screen quotes the exact difference before you agree to it — the part of the new price covering the days left in your current period, and nothing for days you already paid for. You also choose whether the change happens now or at your next renewal, so you can simply wait for the boundary if you prefer a clean line.