The incident gave supporters of Moammar Gadhafi's regime a new rallying point against the international intervention in Libya's civil war.
The foreign minister called for a "global jihad" on the West in response.
Early Sunday, journalists based in the Libyan capital were rushed by government officials to the damaged building, which appeared to have been partly under construction.
Reporters were escorted back to the site during the day, where children's toys, teacups and dust-covered mattresses could be seen amid the rubble.
In a statement issued late Sunday at Brussels headquarters, NATO said airstrikes were launched against a military missile site in Tripoli, but "it appears that one weapon did not strike the intended target and that there may have been a weapons system failure which may have caused a number of civilian casualties."
"NATO regrets the loss of innocent civilian lives and takes great care in conducting strikes against a regime determined to use violence against its own citizens," said Lt. Gen. Charles Bouchard, commander of the anti-Libya operation.
Foreign Minister Abdul-Ati al-Obeidi told reporters nine civilians, including two children, were killed in the explosion and said 18 people were wounded. He said the strike was a "deliberate attack on a civilian neighborhood," and follows other alleged targeting of nonmilitary targets such as a hotel, oxygen factory and civilian vehicles.
It has not always been possible to independently verify the government's reports of strikes on nonmilitary targets since NATO began its air operations in March.
"The deliberate bombing ... is a direct call for all free peoples of the world and for all Muslims to initiate a global jihad against the oppressive, criminal West and never to allow such criminal organizations as NATO to decide the future of other independent and sovereign nations," al-Obeidi said.
He did not take questions.
Journalists were shown the bodies of at least four people said to have been killed in the strike, including the two young children. Foreign reporters in Tripoli are not allowed to travel and report freely and are almost always shadowed by government minders.
Salem Ali Garadi, 51, who said his brother and sister were among the victims, said five people were killed. There was no explanation for the discrepancy in death counts.
Before Sunday's alleged strike, Libya's Health Ministry said 856 civilians had been killed in NATO air attacks since they began in March. The figure could not be independently confirmed. Previous government tolls from individual strikes have proven to be exaggerated.
Alliance warplanes struck Tripoli again Sunday afternoon. A number of explosions could be heard in the city, and smoke could be seen rising over the southern part of the capital.
A coalition including France, Britain and the U.S. launched the first strikes against Gadhafi's forces under a U.N. resolution to protect civilians on March 19. NATO, joined by some Arab allies, assumed control of the air campaign over Libya on March 31.