I think it's down to others not bothering to use headsets rather than an something you've not configured. I've played a few games of killzone and not one person seemed to be using a mic at all. I can only assume people don't want to use the mono ear mic that's boxed with the ps4. Give it some time and it'll take off. I note that on the mono headset that comes with it there's a switch to mute the mic.
I've tried using an Apple iPhone headset but I've found that it only worked for the headphones part, the mic didn't appear to work at all. I tried another headset (blackberry I think) and those worked for both headphones and microphone.