It's less about being good with one or the other, and more about the handicap that using a controller on PC brings. Even if you are a controller master, you are still likely to be at a disadvantage vs m/kb players, and not even super good ones. I guess this being a co-op game, you don't have to worry about encountering this, but there still runs the chance that you could be a detriment to your team.

If you're good with a controller than a mouse & keyboard then you're not handicapping your self at all regardless if other people are good with a mouse & keyboard.

If you suck with keyboard & mouse then you'll be at more of a handicap with that than a controller anyway.

Not to mention using the controller support in this game completely disables any keyboard commands for stuff like text chatting in game.