There are a number of places to change font sizes, in general and for specific applications, but I could not find how to change the size of the font on the desktop.
I know I can change the icon size, but how to I change the font size on the desktop?

They should be using System Settings -> Fonts -> General. (iirc, in kde 4 there was a separate setting for that, but looks like that's been combined in plasma 5)

(it also could depend on the desktop theme you are using; I don't know if they can declare their own font (size) to use)

Okay.

Font > General is a little too general A lot of text in application windows changed. It's sort of a trade-off isn't it. But there's so much more real estate on the Desktop...

Do you know if there is a setting to remove the shadow on the Desktop font?

Regarding desktop and font shadow, would that be a function of "Look and Feel" or "Plasma Theme"?

I disabled font shadow for Windows, under Application Style > Window Decorations and clicking on the "Configure Plastik" (change Plastik to your theme) button.

"Look and Feel" is more of an aggregate of several theme components including Plasma Theme, Color scheme, Window Decorations, etc.

The font shadow that you'd disabled in Window Decorations would be specifically for the font in the title-bar. The shadows used by the rest of the applications would be provided by the Widget Style (which it looks like Breeze doesn't allow customizing the font shadows).

The shadows used by the desktop itself may be defined by the particular Desktop Theme (which would require changing the particular theme's qml files to customize). You might try downloading a few different desktop themes using the Get New Themes dialog to see how different themes handle the shadows.
There was a recent change to how the shadows are rendered in https://phabricator.kde.org/D16968. Checking the diff, you may be able to edit your own /usr/share/plasma/plasmoids/org.kde.desktopcontainment/contents/ui/FolderItemDelegate.qml and make your own tweaks in there.