When using kdialog file chooser certain directories will have an icon other than the default "folder" icon. For instance, "Music" shows musical notes, "Downloads" shows a down arrow, "Pictures" and "photos" shows an image icon, and so on. Is there any way add new directory names which will display icons of my choosing?

Doesn't have anything to do with how a folder is named, instead they'll have a hidden setting which indicates which icon to use for that folder (if you show hidden files/folders, it's saved in the .directory file).

You should be able to right-click on any given folder, select Properties, and then click on the icon to select a different one. (I've also seen tools which allow you to quickly set e.g. the colors of folders directly from right-click menus)