Polycom phones have the option to include a company directory.  This adds all users from the PBX to the phone's internal directory.  If you plan on manually adding custom contacts to the Polycom phone, leave this box unchecked, or uncheck it before adding those contacts.  With the box checked, the phone will remove any custom contacts and only include users from the PBX if the phone is rebooted.  With the box unchecked, the custom contacts will remain if the phone is rebooted.