The prime minister says she does not want to reconsider the decision not to deport the convicted Czech man Karel Sroubek to become a witch hunt.
For Jacinda Ardern Morning report she wants to get the case right, before asking questions about what went wrong.
Sroubek is at the center of the controversy after Immigration Minister Iain Lees-Gallovai prevented him from being deported.
Officials are now considering all new information about a case that occurs contrary to the information provided by Mr. Lees-Gallovai, and said the review would take three weeks.
Ms. Ardern said that evidence that was contrary to the information used to make this decision had now emerged.
"Here I am not guilty, because there seems to be a series of questions in this case and until we prove all the evidence before you, I do not want this to be a witch hunt, it's not what it should be."
Ms. Arder said ministers could make decisions only on the basis of the information they were in front of.
"I do not think anyone would have assumed that the minister would receive, for example, a significant file, and eventually ask questions – to ask questions, to check things, but always based on what is ahead of them."
She said that in this case it seems that there are contradictions that Mr. Lees-Gallovai could not exist to exist.
Ms. Arder said before considering the future procedures in dealing with deportation, it was first of all important that this case be made and that it be examined whether there is information that is contradictory, which are not available or missing.
"There is a question about whether or not the process for these decisions is whether or not it can be reversed, but I want to focus on this case before we go on that road."