Never mind, I got it. All files are in the ~/.kde/share/apps/kabc/ directory. Here's the scoop, the mail application imports the files directly from the ~/.kde/share/apps/kmail/directory, but Contacts doesn't. Instead, it must be exported first to some silly file and then imported.
Shouldn't these two apps work the same? I don't understand why the developers have to make it that difficult. It's pretty annoying, next time I'll use Thunderbird.