Finance Minister Grant Robertson is tight-lipped over whether he thinks the Treasury Secretary should resign.

Gabriel Makhlouf has come under fire this week, after claiming the Budget was released early because of a hack after National released details from Budget early.

It was revealed this morning that the material was accessed through a simple search of the Treasury website. 

National leader Simon Bridges has called for Makhlouf to go, but Robertson told Heather du Plessis-Allan it's not up to him to fire him. 

"It is the State Services Commissioners responsibility, that's how it works with chief executives, and they are conducting an inquiry. That's what we will now follow."

Robertson says Makhlouf has only a few weeks left in the role anyway.

He added that he never accused National of hacking Treasury, that he was simply acting on the advice of Treasury. 

"I'm expressed my disappointment that the Treasury system was able to accessed like this, and I've expressed my disappointment that Treasury didn't do more to find out before going to the Police." 

The controversy over the hack has overshadowed Robertson's world-first Wellbeing Budget, which was delivered earlier today.

The budget includes $1.9 billion for mental health services and over $1 billion for rail. Robertson told Heather du Plessis-Allan that rail is a key example of the under investment of the previous government.

• Budget 2019: Key investment in mental health, education, beneficiaries

"We've got engines and carriages in the South Island that are 40, 50 years old. This is the investment that is going to turn this around." 

He says it is a really important investment, and this is just the beginning.

He rejects that there is not enough in the Budget for business, with a $300 million to assist companies that get past the start-up stage to help them get overseas. 

"There's huge investment in helping people do innovation, there's big investment in skills," he says, adding that is what businesses raise with him the most.