Britain's Chris Froome, 12 seconds behind the German in second, will move into the overall lead.
Etixx-Quick Step's Martin crashed in the final kilometre with defending champion Vincenzo Nibali also involved.
Martin's team-mate Zdenek Stybar came through to win the 191.5km stage from Abbeville to Le Havre.
"Tony Martin will fly to BG Hospital in Hamburg immediately for surgery, and must withdraw from Le Tour de France," a team spokesman said.
Team doctor Helge Riepenhof added: "The collarbone is in lots of pieces, so it was a major impact. One of the pieces came through the skin, which means it's an open fracture. Therefore, even if it was Tony's wish to start tomorrow, I have to say he is not allowed to. Riders always want to race. Tony especially."
Martin is the second race leader to pull out after Swiss rider Fabian Cancellara fractured two vertebrae in his lower back on Monday.
Team Sky's Froome tweeted shortly after Thursday's stage that he had managed to get away with a "minor graze to his knee" after finding himself in the middle of the chaos, but Martin was not so fortunate.
"I was unlucky. I don't even remember how I went down. I touched the rear wheel of the rider in front of me - Bryan Coquard - but it's the Tour, luck and bad luck are always close," the German said.
Martin, who was in yellow for the second day, went down heavily within the last kilometre and, after eventually picking himself up, had to be nursed to the finish line by three of his team-mates.
Team doctor Helge Riepenhof added: "The collarbone is in lots of pieces, so it was a major impact. One of the pieces came through the skin, which means it's an open fracture. Therefore, even if it was Tony's wish to start tomorrow, I have to say he is not allowed to. Riders always want to race. Tony especially."
Martin is the second race leader to pull out after Swiss rider Fabian Cancellara fractured two vertebrae in his lower back on Monday.
Team Sky's Froome tweeted shortly after Thursday's stage that he had managed to get away with a "minor graze to his knee" after finding himself in the middle of the chaos, but Martin was not so fortunate.
"I was unlucky. I don't even remember how I went down. I touched the rear wheel of the rider in front of me - Bryan Coquard - but it's the Tour, luck and bad luck are always close," the German said.
Martin, who was in yellow for the second day, went down heavily within the last kilometre and, after eventually picking himself up, had to be nursed to the finish line by three of his team-mates.
