David Cameron has said he is "puzzled and disappointed" by the Work and Pensions Secretary's decision to quit the Cabinet.
In his scathing resignation letter, Iain Duncan Smith said proposed cuts to disability benefits were a "compromise too far".
He complained of pressure from the Treasury to "salami slice" welfare and an unwillingness to spread the burden of spending cuts.
Mr Duncan Smith concluded that the party leadership had abandoned the austerity principle of "all in this together" in its latest budget.
In his letter of reply, Mr Cameron hit back against the charges, saying the proposals in the latest budget had been "collectively agreed".
"I regret that you have chosen to step down from the Government at this moment," the Prime Minister wrote.
"Together we designed the Personal Independence Payment to support the most vulnerable and to give disabled people more independence.
"We all agreed that the increased resources being spent on disabled people should be properly managed and focused on those who need it most."
He added: "That is why we collectively agreed - you, No 10 and the Treasury - proposals which you and your department then announced a week ago. Today we agreed not to proceed with the policies in their current form and instead to work together to get these policies right over the coming months.
"In the light of this, I am puzzled and disappointed that you have chosen to resign."
Mr Duncan Smith's announcement came hours after the Treasury signalled it was planning a humiliating climbdown over cuts to the Personal Independence Payment (PIP).
In the face of a Tory rebellion in a Commons vote on the budget next week, Government sources said the Treasury wanted to kick the proposals "into the long grass".
Opponents claimed the cuts, which were expected to save around £1.3bn a year, could affect up to 640,000 people, with many losing as much as £100 a week.
In his letter, Mr Duncan Smith wrote: "I have for some time and rather reluctantly come to believe that the latest changes to benefits to the disabled and the context in which they've been made are, a compromise too far.
"While they are defensible in narrow terms, given the continuing deficit, they are not defensible in the way they were placed within a Budget that benefits higher earning taxpayers. They should have instead been part of a wider process to engage others in finding the best way to better focus resources on those most in need.
"I am unable to watch passively whilst certain policies are enacted in order to meet the fiscal self-imposed restraints that I believe are more and more perceived as distinctly political rather than in the national economic interest."
The Work and Pensions Secretary had been at loggerheads with Mr Cameron and Mr Osborne since announcing he would join the campaign for Britain to leave the European Union.
His letter indicates the row over PIP cuts was the final straw.
Sky's Chief Political Correspondent Jon Craig said: "There's clearly been an enormous bust-up between himself and the PM.
"The Government's whole welfare strategy, no doubt Labour will say, is in tatters. When a senior minister quits like this, there are enormous ramifications. It's a shattering blow for Mr Cameron."
