Khan had offered £600m for the national stadium, with the FA retaining the Club Wembley hospitality rights, which it valued at £250m to £300m.
"We fully respect his decision," said FA chief executive Martin Glenn.
Khan has not ruled out making another bid in future if there is more support from FA Council members and said he would respect the "national treasure".
The FA had said it would invest the proceeds of the sale into improving grassroots football facilities.
FA executives made a presentation to the FA Council last week about why they were backing the sale to Khan, and the 127 council members were scheduled to vote on the proposed sale on 24 October.
A senior FA source told BBC Sport that the board believed the odds were slightly against the purchase being backed, given the strong objections of some councillors to the home of English football being sold off.
FA chief executive Glenn said Khan had believed his offer "would be well received by all football stakeholders".
And Glenn added: "At a recent meeting with Mr Khan he expressed to us that, without stronger support from within the game, his offer is being seen as more divisive than it was anticipated to be and has decided to withdraw his proposal.
"Wembley Stadium is an iconic venue that is revered around the world and it will continue to thrive under the ownership and direction of the FA."
Khan said he wanted "to do right by everyone in a manner that strengthens the English game and brings people together, not divides them".
He added: "Unfortunately, given where we are today, I've concluded that the outcome of a vote next week would be far from sufficient in expressing the broad support favoured by the FA chairman to sell Wembley Stadium.
"Until a time when it is evident there is an unmistakable directive from the FA to explore and close a sale, I am respectfully withdrawing my offer to purchase Wembley Stadium."
A chance missed here to make a real difference to grass roots football in this country, well done FA you morons.
Does anyone agree with me or think we were mad to even consider it?