Can he charge me with assault? ?

The police have to charge you with assault. However, he can make a criminal complaint. You would lose as you didn't have to punch him. The anyone would have done it defense doesn't work, sorry to say.