(AP Photo/The Citizens' Voice, Michael R. Sisak)
The massive shakeup Wednesday night came hours after Paterno announced that he planned to retire at the end of his 46th season.
But the outcry following the arrest of former assistant coach Jerry Sandusky on molestation charges proved too much for the board to ignore.
Speaking at his house to a couple of dozen students, Paterno said, "Right now, I'm not the football coach. And I've got to get used to that. After 61 years, I've got to get used to it. I appreciate it. Let me think it through."
He shook hands with many of the students, some of whom were crying.
Other students were upset. A large crowd descended on the administration building, shouting "We want Joe back!" then headed to Beaver Stadium.
One key question has been why Paterno and other top school officials didn't go to police in 2002 after being told a graduate assistant saw Sandusky assaulting a boy in a school shower.
Paterno says he should have done more. Spanier has said he was not told the details of the attack.
Sandusky has denied the charges.
Defensive coordinator Tom Bradley will serve as interim coach while Rodney Erickson will be the interim school president.
Earlier in the day, Paterno said in a statement he was "absolutely devastated" by the case, in which Sandusky, his onetime heir apparent was charged with molesting eight boys in 15 years, with some of the alleged abuse taking place at the Penn State football complex.
"This is a tragedy," Paterno said. "It is one of the great sorrows of my life. With the benefit of hindsight, I wish I had done more."
John Surma, the vice chair of the board of trustees said, "these decisions were made after careful deliberations and in the best interests of the university as a whole."
He said Paterno was told by telephone that he was out after spending most of his life at Penn State and guiding its football teams to two national championships in the 1980s.
"The past several days have been absolutely terrible for the entire Penn State community. But the outrage that we feel is nothing compared to the physical and psychological suffering that allegedly took place," Surma said.
The firings came three days before Penn State hosts Nebraska in its final home game of the season, a day usually set aside to honor seniors on the team.
The ouster of the man affectionately known as "JoePa" brings to an end one of the most storied coaching careers -- not just in college football but in all of sports. Paterno has 409 victories -- a record for major college football -- won two national titles and guided five teams to unbeaten, untied seasons. He reached 300 wins faster than any other coach.
Penn State is 8-1 this year, with its only loss to powerhouse Alabama. The Nittany Lions are No. 12 in The Associated Press poll.
After 19th-ranked Nebraska, Penn State plays at Ohio State and at No. 16 Wisconsin, both Big Ten rivals. It has a chance to play in the Big Ten championship game Dec. 3 in Indianapolis, with a Rose Bowl bid on the line.
After meeting Tuesday, Penn State's board of trustees said it would appoint a committee to investigate the "circumstances" that resulted in the indictment of Sandusky, and of athletic director Tim Curley and a vice president Gary Schultz, who are accused in an alleged cover-up.
Paterno notified Curley and Schultz about the 2002 abuse charge and is not a target of the criminal investigation. Curley and Schultz have been charged with failing to report the incident to the authorities.
Sandusky, who retired from Penn State in June 1999, maintained his innocence through his lawyer. Curley has taken a leave of absence and Schultz has decided to step down. They also say they are innocent.
The committee will be appointed Friday at the board's regular meeting, which Gov. Tom Corbett said he plans to attend, and will examine "what failures occurred and who is responsible and what measures are necessary to ensure" similar mistakes aren't made in the future.
Sandusky founded The Second Mile charity in 1977, working with at-risk youths. It now raises and spends several million dollars each year for its programs. Paterno is listed on The Second Mile's website as a member of its honorary board of directors, a group that includes business executives, golfing great Arnold Palmer and several NFL Hall of Famers and coaches, including retired Pittsburgh Steelers stars Jack Ham and Franco Harris.
On Wednesday morning, Paterno said he planned to retire at the end of the season, but the board had other ideas.
In a statement, Paterno said: "I grieve for the children and their families, and I pray for their comfort and relief."
He went on: "I have come to work every day for the last 61 years with one clear goal in mind: To serve the best interests of this university and the young men who have been entrusted to my care. I have the same goal today."