First things first. You need to determine if your children are in any immediate danger (hopefully not), and if you beleive this to be the case, file a court order to have the children taken out of the house and for the short term, placed in your home. Your ex will have to agree to this. If your ex does not agree, and he will not have a place for the children to reside ad be taken care of properly, go immediately to the police and report the situation while contacting the dpeartment of children resources. It is a good bet if it gets to this point, the children will be wards of the state, and placed with the nearest relatives, most likely you. Because you were not granted custody of your children at the time of your divorce, it might be they will be asked to go the next nerest family member. Another resource for you depending on the age of your children, is go the school, and ask to see the school worker and report this issue to them. The social worker can then use his / her resources to guide you. Best of luck to all involved, and remember our children come first.

How (why) did he get custody in the first place? What would your children prefer? Depending on the answers to those questions, the least selfish choice may be to leave them with their father.

In my state, it is called a "modification of parenting plan." But all states are different. Obviously if you don't even know the name of the legal action you need to bring, you should seek legal counsel in your own home state who DOES know the name of the legal action you want to take. When there are "change of circumstances," parents can ask for their custody status to be changed. I would never, ever, having lost (or given up) custody historically, go into court and try to prove I should have custody back WITHOUT A LAWYER (if I could AT ALL afford legal help). It is often an uphill battle to prove that whatever was going on in YOUR life before has resolved itself so that you are fit to have custody. And you don't want to do that without a lawyer unless you ABSOLUTELY CANNOT AFFORD ONE OR FIND ONE IN ANY WAY. One reason is that...one thing a family law court can do is, instead of place the child in your custody, they can ask for child welfare services to investigate you and determine if there are still (or are now) any problems in your home, like drugs, alcohol, domestic violence, inordinate anger, criminal history, or mental health problems. Again, for all of the reasons stated above, I would try to hire a lawyer immediately or find out if there is some kind of legal clinic near you town where law students or volunteer lawyers could assist you. I will give you two places to call to find this question out: 1) the local court clerk's office might know if there is some kind of clinic that helps people with family law cases. They will also know what this type of case is CALLED in your state. I AM NOT TALKING ABOUT A FACILITATOR HERE, OR A PERSON WHO HELPS YOU DO YOUR OWN WORK. YOU NEED A PRO, AND SHOULD ONLY USE A "FACILITATOR" IF THAT IS YOUR EXACTLY LAST OPTION. 2) Call the state bar association, and find out either the name of a good family law lawyer in your town, or see if there are legal clinics in your town dealing in family law. Unless you think that the ex-wife is HARMING your children, or your child is IN DANGER...which is unlikely...you should not be focused on vindictively removing the children, but indeed on getting custody because it is the right thing to do: for one of the parents to have custoday. This is something that is better done WELL than fast. So, take a few months to save up every penny in order to hire the best lawyer you can get. Tap all your relatives and sign actual loans with them so that they know you are serious about paying them back. If you had shit in your living room, you would find a way to pay a plumber, because a plumber would know what he was doing and you do not. Well you have legal shit in your living room and everyone thinks they can do a lawyer's job. But the truth is that this might be the most important fight of your life and what if you don't know a bolt from a lug nut/ a petition from an object. If your response is that: if I had shit in my living room, I would call the landlord. I'm a renter and could never aford a plumber: I've lived in my own shit for days, then...: 1) either find a legal clinic in your area that does this work for cheap to free or 2) do it yourself. Most states have on-line, human volunteers, or lecturers who teach the public how to work on their own family law stuff. You need to find out who that is in your area. If you tell me the state and county that you live in, I might be able to help you find out too.