Only seven days after Donald Trump was sworn in as president, James Comey has told associates, the FBI director was summoned to the White House for a one-on-one dinner with the new commander in chief.

The conversation that night, Comey now believes, was a harbinger of his downfall this week, said two people who have heard his account.

The president and Comey made small talk about the election and the crowd sizes at Trump's rallies. The president then turned the conversation to whether Comey would pledge his loyalty to him.

Comey declined to make that pledge. Instead, he told Trump that he would always be honest with him. Top government officials like Comey swear an oath to the Constitution.

The White House said that this account is not correct. Trump, in an interview Thursday with NBC, described a far different dinner conversation in which loyalty never came up.

By Comey's account, his answer apparently did not satisfy the president. Trump again said to Comey that he needed his loyalty. Comey again replied that he would give him "honesty" and did not pledge loyalty. Trump pressed him on whether it would be "honest loyalty."

"You will have that," Comey replied, his associates said.