U.S. House Speaker Paul Ryan forced out the Jesuit priest who had served as chaplain of the U.S. House of Representatives since 2011 because he said in a prayer during deliberations on tax cuts that lawmakers should be “fair to all Americans,” a news report stated last week.

The Rev. Patrick Conroy’s letter of resignation said he was stepping down because of a request from the Wisconsin Republican, who is also a Catholic.

“As you have requested, I hereby offer my resignation as the 60th Chaplain of the United States House of Representatives,” stated Rev. Conroy’s April 15 letter to Speaker Ryan.

Four sources — two from each political party — said that Rev. Conroy was told he had to retire or he would be canned.

Some of the sources said Speaker Ryan thought Rev. Conroy favored Democrats in the House, and cited the prayer and the priest’s invitation to a Muslim to offer a prayer before the chambers as the reasons behind the chaplain’s forced resignation.

Democrats believe that Speaker Ryan pushed Rev. Conroy out “because Republicans thought he was aligned with Democrats,” according to a senior Democratic aide familiar with the discussion.

One Democratic lawmaker said Speaker Ryan objected to the prayer on the House floor that Republicans viewed as critical of the GOP tax cut bill.

A second Democratic aide said that Rev. Conroy’s ouster was “largely driven by a speech on the tax bill that the Speaker didn’t like,” and also offered the second explanation.

“Some of the more conservative evangelical Republicans didn’t like that the Father had invited a Muslim person to give the opening prayer,” the source said.

But a senior GOP aide said Rev. Conroy’s exit “was not because of any particular prayer.”

House Minority Leader Nancy Pelosi of California brought up the forced resignation during the Democrats’ whip meeting in the Capitol on April 26.

Ashlee Strong, a spokesman for Speaker Ryan, said it was the speaker’s decision, but offered no explanation for the move, adding that Rep. Pelosi and her office “were fully read in and did not object.”

Rep. Pelosi’s office disputed that she didn’t object.

“Leader Pelosi was given advance notice by Speaker Ryan,” said a Pelosi spokesman, but she “also made it clear to Speaker Ryan that she disagreed with this decision.”

Speaker Ryan has appointed Republican Reps. Mark Walker of North Carolina and Tim Walberg of Michigan, both former pastors, to lead the search for a new chaplain after Rev. Conroy leaves at the end of this month.