I've been searching for settings to limit a computer to access a specific website and serve no other function at all. Is this something I can do in Kubuntu or do I need to use some kind of external firewall?

You could deinstall all other useful software apart from Firefox (or edit/cripple the menu to such an extent that only firefox is shown) and have it work from a whitelist.

A proxy would of course be preferable...

In terms of limiting the functionality of KDE itself, I suggest you look into the Kiosk framework. Documentation on this can be found on KDE Techbase.
As to locking it down to accessing a single website only, I recommend a filtering proxy such as Squid with DansGuardian.