Zuma appointed him for the same portfolio after sacking two other finance ministers within a month.
They were encouraged by Mr Gordhan who called a news conference shortly before midday, urging ANC members to hark back to their routes as a party of activists and suggesting they "organise" against the presidency. Gordhan has been replaced by Malusi Gigaba, a former home affairs minister, a statement from the president's office said. Earlier a year ago, South Africa's highest court found that Zuma had violated his oath of office by refusing to abide by an order to pay back some of the millions of dollars in public money spent on upgrading his rural home. Jonas was also fired by Zuma.
His decision to replace the finance minister has drawn criticism from the ruling party, its allies, opposition parties and big business.
Zuma has repeatedly been accused of corruption, with intense scrutiny of his ties to a family of local tycoons.
Mr Zuma and Mr Gordhan's relationship has deteriorated rapidly over the past few months because the latter, as head of the treasury, had blocked numerous president's economic development plans, saying the country could not afford them. The Guptas and Zuma deny any wrongdoing.
"I don't come here gung-ho with an agenda to change the director-general (DG)", he said, referencing Lungisa Fuzile who was part of a tight team with Gordhan and Jonas. His main rival is Nkosazana Dlamini-Zuma, the former head of the African Union Commission and the president's ex-wife.
"I'm very uncomfortable because areas where ministers do not perform have not been touched".
"There are quite a number of other colleagues and comrades who are unhappy about this situation, particularly the removal of the minister of finance who was serving the country with absolute distinction".
"We were given a list that was complete, and in my own view as a secretary I felt like this list has been developed somewhere else and was given to us to legitimize it", Mantashe told Talk Radio 702.
Zuma, 74, has been South African president since 2009. The recall caused South Africa's rand to lose almost 5 percent. Share prices of South African's top four banks plunged more than 5% in early trade.
ANC administrations since Nelson Mandela's in 1994 have promised to spread prosperity and the government has made strides, including the construction of 4.5 million housing units for the poor and the erection of a social welfare system that puts food on the tables of the most vulnerable.
Zuma retains widespread support from ANC members in some rural areas and has been able to rely on the party's majority in parliament to survive several votes of no confidence. "The briefing by the president left a distinct impression that the ANC is no longer the center and thus depriving the leadership collective of its responsibly to advise politically on executive matters". This creates further suspicion that the president is aligning the treasury with his own business agendas.
The media in South Africa reported that the move pushed the rand more than 3% weaker while yields on South Africa's dollar-denominated sovereign debt rose as fears surfaced again of a sovereign credit rating downgrade on South African bonds.
Gordhan's dismissal has deepened the split in the country's ruling African National Congress (ANC) party.
The rand took its biggest weekly dive since 2015, sliding by more than 2.6% before recovering.