Just a day after John Key's flippant remark over Phillip Smith, one of his ministers is in similar hot water.

Peter Dunne says he was "flippant" to liken New Zealand First to the KKK - but he's standing by his controversial tweet, amid an online backlash.

He questioned whether Winston Peters' party was relaxing its membership rules, when the KKK announced it'll accept black, Jewish and gay members.

Peter Dunne says he was drawing an analogy with New Zealand First's anti-immigration stance.

"When you've got MPs like Prosser and O'Rourke who tow that line, I think they deserve the criticism but look, this is hardy the end of the world.

"The sun will still came up this morning, people are still happy, and I think we should just get a life and move on."