When creating the account, I didn't check "Download all messages for offline use", so KMail just downloaded all message headers.

Now, I want it to go ahead and download all message bodies so I can use KMail offline, but there doesn't seem to be a way to do so. In Thunderbird, this can be done with "Work Offline". However, in KMail, "Work Offline" just disconnect everything.

Settings > Accounts > Modify > Advanced
and tick "Enable Disconnected Mode"

Did that, but nothing happened to existing mails (which only have headers downloaded). I think the setting only affects new mails.