You might be right, although it _is_ possible to get rid of it afterwards by killing the second kwin, kdesktop, kicker and khotkeys that get started when configuring a second screen.
And I did figure out that they get started by ksmserver but I haven't figured out yet if it's possible to tell ksmserver to leave the second screen alone. It's configuration file has an item called ScreenCount which is set to 2 in my case, but setting it to 1 doesn't do anything and it just gets reset to 2 after a restart of X.