SAN FRANCISCO (KPIX 5) — It started out as a replacement for an old bus station. But it morphed into one of the biggest transit projects in America, complete with towering high-rises, an underground train station, breathtaking lobbies, even a rooftop park.

Now thanks to a deal gone bad between downtown high-rise owners and the city, what we could wind up with is something very different. “Empty towers, fancy in nature, overlooking a giant hole in the ground,” said Adam Alberti of the Transbay Authority.

READ MORE: Stimulus Check Update: When Might Your $1,400 Economic Relief Payment Arrive?

In return for being given the green light to build billions of dollars in high-rises, the developers agreed to help pay for the Transbay Terminal. But now they are claiming the $200 million the city wants is way too much.

“We need to make sure that these developers pay what they are supposed to pay,” said Supervisor Scott Wiener.

The builders are threatening lawsuits, something that could tie up the project’s funding for years.

Developers even brought in heavy hitter and former San Francisco Mayor Willie Brown and paid him $100,000 to help broker a new deal, only to have it fall apart.

READ MORE: San Jose Police Investigate 2nd Weekend Fatal Shooting, 9th Homicide Of The Year

“You’ve got too many people floating around in the clouds to make any rational deal,” Brown told KPIX 5.

The result is a giant game of chicken: the developers can tie up the funding for the center, but then the city can block them from opening their high-rises.

“We don’t believe it should come to that. We believe that everybody should honor their obligations and move this project forward for everybody’s benefit,” Alberti said.

When asked if he thought the developers would sue, Brown said, “I don’t think they have a choice.”

Alberti said, “The end result could be a stop to the Transbay prorgram, temporarily.”

MORE NEWS: Storm Front Packing Potential Lightning Strikes Headed Toward San Francisco Bay Area

Construction on the project began in 2011. Work is supposed to be completed in 2017, but at this point, that’s not looking likely.