I've got a bit of the opposite problem. Sometimes I forget my phone or forget that he is going to call at a certain time. I have a young baby at home and things can get a bit hectic. I always apologise and explain but he then takes it out on me, claiming that I am breaking the telephone section of our orders. Yet he doesn't remember that he has done the same or similar things in the past, with forgetting things or making mistakes, but I never had a go at him for it. Such as failing to pick my daughter up from school twice because his alarm didn't go off. And like forgetting to pack certain things in her bag when she comes to my place (which I sometimes do too).
It is probably different in your case but that's my two cents worth
maybe he is just a scatter-brain? Not defending him or anything! But that's the case in my situation, I'm a bit of a scatter-brain sometimes.
Having a set time when they aren't busy is good and if he continues to prevent your children from speaking with you then you could give your children their own mobile maybe. Not sure if you have any court orders in place? That's a good thing to put in the orders.
Good luck with your situation