SANTA ROSA (CBS SF) — The Sonoma County District Attorney’s Office and California Attorney General Kamala Harris filed a 164-count complaint Thursday against a Petaluma man suspected of stealing $20 million from dozens of investors in an alleged securities fraud Ponzi scheme.

Aldo Joseph Baccala, 71, delayed entering a plea at his arraignment Thursday afternoon in Sonoma County Superior Court. His next court date is June 15.

READ MORE: Video: Motorist Rescued From Fiery Freeway Crash In San Jose

Baccala is being held under $2 million bail in the Sonoma County jail. Judge Robert LaForge denied a request by Deputy District Attorney Robin Hammond to raise Baccala’s bail to $3,980,000.

Baccala is charged with 131 counts of making untrue statements or omissions to buy and sell securities, 24 counts of grand theft, elder financial abuse and one count of securities fraud.

The criminal complaint alleges the crimes resulted in a $3.2 million loss.

Court records show 85 plaintiffs were awarded a total of $21 million in a civil suit settlement with the Aldo and Karen Baccala Trust in March 2009.

The criminal complaint alleges Baccala used his Petaluma-based company, Baccala Realty, to raise millions of dollars from more than 50 investors for ventures, including assisted living facilities, a car wash and other businesses in California, South Carolina and other states.

Many of the victims were elderly and knew Baccala and his family for many years, Sonoma County District Attorney Jill Ravitch and Harris said in a news release.

They were promised annual returns of 12 percent or more to invest in a specific project that was supposed to be secured by a first or second deed of trust on the property, prosecutors said.

None of the promised deeds of trust were ever recorded and the funds raised were not used as promised, according to the complaint.

Baccala allegedly used the victims’ money to invest in the stock market and to cover margin calls and stock trading losses, prosecutors said.

Baccala lost $8 million in the stock market between 2003 and 2008, and as his debts grew, he promised new investors annual returns of up to 27.5 percent to cover payments to earlier investors, according to the complaint.

READ MORE: Mothers Tearfully Remember Children Slain In Bay Area Homicides

Baccala issued letters to investors in November 2008 informing them he would no longer make promised monthly payments. After receiving complaints, the Sonoma County District Attorney’s Office began investigating Baccala realty in January 2009.

Baccala was arrested Tuesday.

Maribel Chin of Petaluma, who attended the arraignment Thursday afternoon, said she and her husband made investments with Baccala because they knew him for a long time.

“We want to believe he went overboard and got caught in a web. He was taking money from one to pay for another,” she said.

She said some people who invested in a South Carolina nursing home and other investments “were okay.”

“My husband doesn’t think he did it maliciously,” Chin said.

Chin and her husband were not plaintiffs in the civil suit.

“My husband didn’t think the civil suit was a good idea. They (the plaintiffs) didn’t get much,” Chin said.

“He (Baccala) had a lot of money,” Chin said, adding Baccala reportedly was living on his daughter’s property when he was arrested.

Baccala’s attorney Stephen Gallenson said some of the plaintiffs who settled the civil suit stipulated there was no elder abuse or misconduct.

He said he could not comment on the criminal complaint because he had just received it.

MORE NEWS: With Playoff Dreams Dancing In Their Heads; San Francisco 49ers Fans Return To Levi's Stadium

(Copyright 2012 by CBS San Francisco and Bay City News Service. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.)