Alexis Sanchez put the home side in front after 14 minutes with a superbly-struck free-kick from 25 yards out.
However, Dusan Tadic equalised from the penalty spot six minutes later after Tomas Rosicky brought down debut boy Sadio Mane.
Nathaniel Clyne then stunned Arsenal with what would prove a fine 40th-minute winner, with the full-back blasting past David Ospina from fully 30 yards.
Arsenal tasted defeat in the third round of this competition for the first time in 12 years as Southampton secured a first victory at this venue, but Wenger said of his much-changed side: "I cannot fault the attitude.
"The reason we lost is because of our opponents' performance. It is not the players' fault. You have to give credit to Southampton. They are a good side and they played very well.
"It's disappointing because we had a good start, but we made big errors when we had the ball. We gave a lot but it's frustrating."
One plus point for Arsenal was the appearance of injury-plagued midfielder Abou Diaby, with Wenger adding: "It was difficult for him but he got 60 minutes and was not injured. He was acceptable but tired."