That's a very human and very understandable reaction. I procrastinated for years because of the massive blowback on the children from their mother whenever things looked like they were coming to a head. That changed when the older ones were old enough to make their own decisions and just refuse to live with her.But I did procrastinate on sending the draft proceedings back signed because I was scared of his (ex) reaction and consequences
The main thing you and they need is stability and you won't get that until you have a legally binding agreement in place. Remember that someone who coercively controls you will do the same to children and they are more vulnerable and more likely to be damaged by it.