Mr Obama said the memory of 6 August 1945 must never fade, but did not apologise for the US attack - the world's first nuclear bombing.
Mr Obama spoke to a number of survivors and in an address called on nations to pursue a world without nuclear weapons.
At least 140,000 people died in Hiroshima and another 74,000 three days later in a second bombing in Nagasaki.
'Best of friends'
Mr Obama first visited the Hiroshima Peace Memorial Museum before walking to the Peace Memorial Park, accompanied by Prime Minister Shinzo Abe.
Both men stood in front of the eternal flame. Mr Obama laid a wreath first, followed by Mr Abe.
"Death fell from the sky and the world was changed," Mr Obama said in his address, noting that the bombing had shown that "mankind possessed the means to destroy itself".
Mr Obama said the memory of Hiroshima must never fade: "It allows us to fight complacencies, fuels our moral imagination and allows us to change."
Of nuclear weapons, he said: "We must have the courage to escape the logic of fear and pursue a world without them."
Mr Obama then spoke to survivors, hugging 79-year-old Shigeaki Mori.
Symbols and codes - BBC's John Sudworth in Hiroshima
The image of President Obama hugging a survivor will resonate deeply with the Japanese public. Opinion polls show that the majority of people welcome this visit and most, it seems, do not mind either about the absence of an apology.
The deep symbolism is enough; the leader of the only country ever to have used an atomic weapon laying a wreath in a city that has become a monument to the perils of our nuclear age.
But others will point out that, while his speech was long on lofty idealism, he remains the commander in chief of one of the world's largest nuclear arsenals, one that he has approved billions of dollars to modernise.
Standing just a few rows away from the president, as he always does, an officer could be seen holding the briefcase containing the nuclear launch codes.
The nuclear code briefcases loaded at the White House for the Asian trip AFP
Mr Obama told service personnel at the base: "This is an opportunity to honour the memory of all who were lost during World War Two."
Mr Obama praised the US-Japan alliance as "one of the strongest in the world", with his visit "a testament to how even the most painful divides can be bridged - how our two nations, former adversaries, cannot just become partners, but become the best of friends and the strongest of allies".
Mr Obama and Mr Abe both laid wreaths at the Peace Memorial Park AFP
Many in the US believe the use of the nuclear bomb, though devastating, was right, because it forced Japan to surrender, bringing an end to World War Two.
The daughter of one survivor, who was visiting the memorial on Friday, said the suffering had "carried on over the generations".
"That is what I want President Obama to know," Han Jeong-soon, 58, told the Associated Press news agency. "I want him to understand our sufferings."
Seiki Sato, whose father was orphaned by the bomb, told the New York Times: "We Japanese did terrible, terrible things all over Asia. That is true. And we Japanese should say we are sorry because we are so ashamed, and we have not apologised sincerely to all these Asian countries. But the dropping of the atomic bomb was completely evil."
Jimmy Carter visited Hiroshima in 1984, after the end of his presidency.
A US ambassador attended the annual commemoration for the first time in 2010.
'Just listen' - Japan's media on the visit
The Chugoku Shimbun urges Mr Obama to "hear the voices of Hiroshima". "The people of Hiroshima will be watching the president closely, eyeing to what extent he is truly resolved to advance the abolition of nuclear arms," it said.
The Asahi Shimbun carries an article saying Mr Obama's "gestures will shape the visit", with the "most powerful gesture" being to "just listen to the bomb victims' memories of suffering and activism".
The Japan Times says: "To truly pay homage to those whose lives were lost or irrevocably altered by the Hiroshima and Nagasaki bombings, Obama's visit must galvanise the international community to move without delay toward a world free of nuclear weapons. The fact that these weapons have not been used over the past 70 years does not guarantee a risk-free future for our children."
The bomb was nicknamed "Little Boy" and was thought to have the explosive force of 20,000 tonnes of TNT
Paul Tibbets, a 30-year-old colonel from Illinois, led the mission to drop the atomic bomb on Japan
The Enola Gay, the plane which dropped the bomb, was named in tribute to Col Tibbets' mother
The final target was decided less than an hour before the bomb was dropped. The good weather conditions over Hiroshima sealed the city's fate
On detonation, the temperature at the burst-point of the bomb was several million degrees. Thousands of people on the ground were killed or injured instantly