South Korea was also eliminated despite the victory.
South Korea's Kim Young-gwon celebrates scoring his sides first goal during the win over Germany.
Joachim Loew is safe in his role as Germany coach even if the defending champions exit the World Cup early, the boss of the German FA said ahead of Wednesday's crunch clash against South Korea.
Despite being flagged offside initially, the goal was eventually given after VAR got involved, with the final touch before the goal actually coming off a German player.
Low's side briefly rallied for an equaliser, but their legs were gone and when substitute Ju Se-jong hit a long ball forward to Son, the Tottenham man was able to run in behind Neuer and rub salt in Germany's wounds with the easiest of finishes.
Germany may be the fourth of the last five defending champions - France in 2002, Italy in 2010 and Spain in 2016 - to exit the group phase, but their capitulation will send alarm bells ringing across the country and the Bundesliga, which has always been a one-horse race led by Bayern Munich.
Egypt goalkeeper El-Hadary becomes World Cup's oldest-ever player
Saudi Arabia face Egypt on Monday afternoon with both sides already out of the World Cup and now purely playing for pride . Both sides returned to the pitch for the remainder of the game after observing the 15 minutes compulsory rest.
"It was only after the match that we found out the other score, of course we were disappointed and that's why some of us cried".
With Germany desperately looking for a goal to get back in the match, Neuer joined the attack.
The BBC and ITV are splitting World Cup TV coverage here in the United Kingdom, and the Germany vs South Korea clash has gone to the Beeb.
Germany's task was made more hard when key defender Jerome Boateng was sent off in the 82nd minute for a second yellow card. Germany defeated South Korea 1-0 in the next encounter in the 2002 World Cup while the latter was victorious in a friendly match in 2004. More than £430,000 was matched on the Germans at [1.17]. Germany, one of the pre-tournament favorites, was out.
Germany vs South Korea: When is today's World Cup match?