The African-American boy who grew up with an absent father, who started his work life as a community organizer on the payroll of a Catholic agency and who later became U.S. president had plenty to say about poverty in our “winner-take-all” economy.

President Obama spoke Tuesday of “ladders of opportunity” once denied to black people and now being dismantled for poor white people as their difficult lives get that much more difficult: “It’s hard being poor. It’s time-consuming. It’s stressful.”

President Obama joined two policy voices from the left and right in a rare moment of participating in a panel discussion, part of a three-day symposium at Georgetown University on combating poverty. The audience of 700 included 120 Catholic and evangelical leaders.

The president said he sees “the lucky and the successful” withdrawing from the shared spaces of American society, including schools and parks and other “common goods.” Meanwhile, the free market has “turbocharged” the divisions.

He shared the stage with Harvard public policy expert Robert Putnam, author of a searing look at the lives of poor youths in “Our Kids: The American Dream in Crisis,” and American Enterprise Institute President Arthur Brooks, author of “The Conservative Heart: How to Build a Fairer, Happier and More Prosperous America.”

The idea for the symposium and the panel was to address “purple problems” with solutions that could work in both red and blue states. It was co-sponsored by Georgetown’s Initiative on Catholic Social Thought and Public Life and the National Association of Evangelicals.

Although moderator E.J. Dionne pushed for specifics, asking the president what he would take to Congress, the discussion stayed chiefly on the underlying divisions of left and right politics.

President Obama said that often the contentious arguments over whether and how to help the poor are thrown off by a Fox News-fueled vision of the poor as “sponges, leeches … lazy and undeserving. They find people who make me mad!”

The president recently garnered private sector support to expand his “My Brother’s Keeper” program for mentoring and developing young black men. He addressed both the family and personal side of poverty, mentioning that he spoke about fatherhood, not “macroeconomics,” with men graduating from a historically black university.

“I make no apologies. I am a black man who grew up without a father and I know the costs I paid for that. And I knew I had the capacity to break that cycle. As a consequence, my daughters are better off.”

There’s no need to choose between promoting strong families and early childhood education and financial investment in public goods, the president said.

“I am all for values, I am all for character,” he said. “But when we get to actually spending money … it all breaks down. If we can’t ask from society’s lottery winners to make this investment, then this discussion is for show.”

President Obama presented it as an imperative of Christian faith that religious groups “speak out on this in a more forceful fashion.” There is, he said, “a transformative power to be had here.”

Mr. Putnam took the charge for change directly to the audience, charging the listeners to step up in mentoring and working with schools to eliminate fees for extracurricular activities that cut the poor out of mentoring and coaching opportunities.

Mr. Brooks, speaking from the “center right,” drew applause with a similar plea to “stop the unnecessary holy war over policy differences.”

“We have a commitment to the teaching of the Savior,” Mr. Brooks said. Poor people “are not liabilities to manage; they are assets to develop. They are human capital.”

But Mr. Brooks added qualifiers to his vision of a truly viable social safety net: “It should be limited to the truly indigent, and it should always come with the dignifying power of work. Then we can have an interesting moral consensus and make some progress.”