After the Yiling Patriarch returns, all those who died at his hand, or blame him for their deaths, decide to haunt him. It doesn't matter that he can't see them — they'll just follow him around and curse him for all eternity.
…Of course they aren't expecting to unwittingly become witnesses to his epic love story with the Second Jade of Lan.
(Or, in which a bunch of stubborn, vengeful spirits somehow become Wangxian shippers).