If you are replacing the current address book with the one you synced to ownCloud from Outlook, consider to delete the address book before syncing, otherwise you might end up with duplicated contact entries. Here are the necessary steps:
If you are replacing the calendar on the phone with a new calendar on the server, do not change the sync settings on the phone. Delete the address book on the server and make sure the sync settings on the computer are set to one-way (Outlook to Server).