U.S. President Donald Trump fired Secretary of State Rex Tillerson on Tuesday and said he would nominate CIA Director Mike Pompeo to replace him, ending Tillerson's difficult tenure by tweeting an ouster that had been long expected and yet was shocking in its abruptness.

Speaking to reporters later on the White House lawn, Trump said he made the decision to replace Tillerson "by myself," and added that Tillerson will be "much happier now."

"We got along actually quite well but we disagreed on things," Trump said.

"When you look at the Iran deal: I think it's terrible, I guess he thinks it was OK. I wanted to break it or do something, and he felt a little bit differently," he continued, a diplomatic version of a difficult relationship with his top diplomat that included reports Tillerson had once called the president a "moron" in a meeting with other officials. 

Later at the State Department, a visibly emotional Tillerson said Trump called him around noon from Air Force One and that he had also spoken with White House chief of staff John Kelly.

During an impromptu news conference at the White House, Tillerson said he is delegating his responsibilities at the end of the workday Tuesday to deputy secretary of state John Sullivan. He added he will be formally stepping down from his post at the end of the month. 

'We disagreed on things': Donald Trump tells media why Rex Tillerson is out0:44

"What is most important is to ensure an orderly and smooth transition during a time that the country continues to face significant policy and national security challenges," Tillerson, whose voice quivered at times, told reporters in a packed briefing room.

The reshuffle comes at a time of intense turnover within Trump's administration that has alarmed those both in and out of the White House. Top economic adviser Gary Cohn announced his resignation last week, not long after communications director Hope Hicks and staff secretary Rob Porter both departed near the start of Trump's second year in office.

For Trump, the shift comes at a critical time for several key foreign policy priorities, including a decision on whether to withdraw the U.S. from the 2015 Iran deal that Pompeo has called "disastrous." Tillerson had long pushed Trump to remain in the agreement and had been pursuing a delicate strategy with European allies and others to try to improve or augment it to Trump's liking.

As part of Tillerson's ouster, Trump also nominated Gina Haspel to replace Pompeo at the CIA. Haspel, the agency's current deputy director, oversaw a black site in Thailand during the early years of the war in Afghanistan.

If confirmed, she would be the CIA's first female director. 

Donald Trump Rex Tillerson fired

Trump, speaking to reporters on the White House lawn on Tuesday, cited the Iran nuclear deal as an example of an issue that he and Tillerson disagreed on during Tillerson's time as secretary of state. (Leah Mills/Reuters)

North Korea diplomacy looms

Trump's moves also come amid a dramatic diplomatic opening with North Korea, with Trump set to hold a historic meeting with leader Kim Jong-un in May.

Pressuring North Korea with sanctions and other isolation measures had been a top Tillerson priority, and he had been one of the administration's more vocal advocates for holding talks in some form with the North. When Trump ultimately accepted Kim's invitation for a meeting, Tillerson was in Ethiopia, though he said he spoke with Trump about it shortly before it was announced.

Trump publicly undercut Tillerson's diplomatic initiatives numerous times, including on Monday when Tillerson's comments about Russia appeared to be at odds with those of the White House.

Trump and Tillerson, who had no diplomatic or political experience before becoming secretary of state, have diverged on policy numerous times, including over North Korea and Russia. Tillerson sharply criticized Russia over the poisonings in England of a former spy and his daughter, directly blaming Moscow after White House press secretary Sarah Huckabee Sanders stopped short of doing so.​

Tillerson learned of his firing by tweet

Trump's early-morning announcement took the State Department by surprise, barely four hours after Tillerson returned prematurely from a trip to Africa and just as officials had been preparing yet again to dispute rumours of his demise.

The circumstances behind the firing were unclear.

Two White House officials said Tillerson was told he was out on Friday. But Undersecretary of State Steve Goldstein said Tillerson "did not speak to the president and is unaware of the reason." He added that the former Exxon Mobil CEO "had every intention of staying" in the job because he felt he was making critical progress in national security.

Gina Haspel CIA director

Gina Haspel is a veteran CIA clandestine officer picked by U.S. President Donald Trump to head the agency. During the early years of the war in Afghanistan, Haspel oversaw a CIA blacksite in Thailand where agents used 'enhanced interrogation tactics' on detainees. (CIA handout via Reuters)

The State Department said Tillerson only learned of his termination when he read Trump's tweet on Tuesday morning.

Two senior department officials said Tillerson received a call from John Kelly, Trump's chief of staff, on Friday, but was only told that there might be a presidential tweet that would concern him.

Kelly didn't tell Tillerson what the tweet might say or when it might actually publish, according to the official, who wasn't authorized to speak publicly on the matter and demanded anonymity.

Goldstein was fired by the White House just hours after he provided his statement to the media. 

A confirmation hearing for Pompeo is expected to be held in April. 

Impact on Canada

Tillerson's ouster will also affect diplomatic relations with Canada. He and Foreign Affairs Minister Chrystia Freeland had developed a working relationship that included co-hosting the Vancouver Summit, a gathering of international partners organized to seek a diplomatic solution to the crisis over North Korea's nuclear weapons program.

Freeland congratulated Pompeo on Twitter and called Tillerson "a great friend and partner to Canada."

Prime Minister Justin Trudeau expressed optimism about U.S.-Canada relationships under Pompeo

"We're going to continue to build a strong productive relationship moving forward on highlighting just the closeness, the interconnected-ness, the unique relationship in all the world that is the friendship between Canada and the United States," Trudeau said. 

With files from CBC News