The Wireless-N 2230 supports up to 300Mb/s. The Wireless-AC 3165 supports up to 433Mb/s. Have you enabled and are you using the 5GHz band? If not, you should.
BTW, WPA mode has no bearing on this at all.
Modem? Nobody said anything about your modem.
The Wireless-AC 3165 card, like the Wireless-N 2230 before it, requires antennas to be connected to it.