In some cases workers claimed they were unable to work even while they played golf, shovelled snow or rode bikes, the complaint says.
The move allowed them to get disability pay on top of their retirement pension, prosecutors said.
In filing the claims, the railway workers allegedly paid between $800 and $1,200 to hire one of several disability doctors.
Those doctors would then conduct unnecessary tests and concoct a medical issue that would allow the workers to go on disability, prosecutors said.
Two have been charged and a third doctor has died.
The U.S. attorney's office in Manhattan said the scheme cost the Railroad Retirement Board more than $1 billion.
The investigation developed after a series of reports by The New York Times starting in 2008.
The Times said that almost every longtime LIRR employee was receiving disability payments, resulting in a disability rate sharply higher than other railroads.
In many cases, workers were far healthier than those claims would indicate, according to prosecutors.
One defendant, a former engineering manager, receives about $105,000 a year in pension and disability pay, based on "severe pain when gripping and using simple hand tools and pain in his knees, shoulder and back from bending or crouching," the complaint says.
An investigation found, however, that he plays tennis several times a week and golfs regularly in his retirement.
Another defendant was seen shovelling heavy snow and walking with a baby stroller for 40 minutes, despite a disability claim in which she claimed to be unable to stand for more than five minutes without leg pain.
She is paid at least $108,000 a year by the railroad authority, prosecutors said.
U.S. Attorney for the Southern District of New York Preet Bharara was expected to announce the mail fraud and conspiracy to commit health care fraud charges later on Thursday.