Total stamp tax received by the government has dropped by 5% to £15.6m between 2017 and 2019, HMRC stamp duty statistics have showed.
If first-time buyer relief and the devolution to Wales were discounted, the fall would be around 3%.
Andrew Southern, chairman of property developer Southern Grove, said: “First-time buyer relief is a bit of a distraction from the real issue, which is that this is an overly punitive tax that forces many to sit on their hands.
“Even after first-time buyer relief and Welsh devolution have been taken into account, the Treasury has still seen its tax take from property transactions fall by £369m in a year.
“This is against a backdrop of continued property price increases nationwide.
“Stamp duty is the weak link that will continue to bind the market’s progress and only last month was blamed for a monumental retreat in the number of cash buyers.”
In addition, the amount of stamp duty government collected from higher rates on additional dwellings fell by 6% year-on-year.
Southern added: “A healthy 9% rise in receipts in England a year ago has been replaced by a 6% decline and, while you can be sure no one will shed a tear for the Chancellor, it is what this means for ordinary people that matters.
“They want to move, feel they have the capital to move but can’t get their head around the absolutely vicious sting in the tail that means someone purchasing a £500,000 home pays £15,000 just for the privilege of doing so.”