New Zealand First leader Winston Peters has been kicked out of Parliament.

The Northland MP got his marching orders this afternoon after he had another testy exchange with Parliament's Speaker David Carter.

"Can I have the supplementary question please".

"Well, you're getting it," replied Mr Peters.

"Member will resume his seat, he will stand and apologise for that remark".

That exchange didn't get him booted but the next one did.

"Mr Speaker with the greatest of respect I've been here longer than you and I know how to answer a question", said Mr Peters.

Speaker Carter replied: "The member will leave the chamber".