For what it's worth ... I have sole custody of our son and when, in the past, there was a potential problem with getting his mother to agree to him accompanying me on an EU trip years ago, my solicitor advised me not to just go and, if necessary, to go to the family court to get an order officially allowing me to take him abroad. (As it happened that didn't turn out to be necessary).
This could be a very tricky question to answer authoritatively here and taking any action unilaterally could potentially result in a serious situation (e.g. child trafficking!) so I would strongly urge you to seek professional legal advice.