Hi,
You can code the check in the PBO(Process Before Output) module of the maintenance view for the table to check if the DB is have one line already, and pop up the messages.
I think for the second requirement You will have to Put the logic to do so IN the PBO module again. The besy advice will be to design your screens and maintain rather that SM30 will give you better flexibility to design all the requirements.
You can get started with Screens and Module pool programming. Some links to help you.
http://help.sap.com/saphelp_40b/helpdata/en/9f/db9d0335c111d1829f0000e829fbfe/content.htm
http://help.sap.com/saphelp_nw04s/helpdata/en/47/a19fd982841503e10000000a42189c/content.htm
Cheers,
Arindam
