According to an official at the Ministry of Foreign Affairs who spoke told Channels TV, the demolition took place late Friday.
The man was said to have visited the commission earlier last week with pieces of evidence to back his claim.
The Nigerian High Commission reportedly petitioned Ghanaian authorities over it but there was no response.
The businessman then returned on Friday with a bulldozer and began tearing down the building.
The source, according to the broadcast station, said the police were contacted, but they got to the scene late.
The man was said to have left the premises without questioning.
The reported incident comes five months after the foreign affairs ministry denied reports that the Nigerian Commission in Ghana had been evicted.
AFRICA TODAY NEWS, NEW YORK