Dormammu was always coming to earth, just waiting for someone to open the door for him. Kaecilius and his followers stole the pages from the spellbook that would call Dormy, they just had to destroy enough sanctums to open the shield protecting earth.
