Without actually being provided a photo of your problem, I'll need to guess. If it's just a slight scuff with white paint only, apply a small amount of auto wax/polish to a soft cloth and lightly rub it over the deposited white paint until it "erases". However, if the scuff has breached the clearcoat, you will have only a temporary fix with the similar treatment and eventually you may wish to have that portion of the clearcoat professionally restored. This is especially true if the scuff breached the pigmented portion of the car finish, then a professional touchup is called for. Try my first suggestion carefully first though before you panic.

Your question suggests that there is white from the wall now imbedded (won’t wash out) in your paint but the depth of the scratches is not known. You can buy polishing compound from your local auto PAINT supply store. These compounds come in varying grits like sandpaper. The staff at the store may be willing to look at your problem and recommend a course of action. This may start with an aggressive compound followed by finer compound. They may also recommend using special equipment such as random orbital buffer and foam pads. It takes skill earned from practice to use these compounds and tools without damaging your paint, but it sounds like you already have damage. There is also colored wax to match your auto paint which fills scratches a bit better than clear wax. On the subject of clear coats, clear coats were initially used only on metal flake paints, but most cars today have clear coats on all colors. So if you have a solid color paint, which is older, it may not have clear coat, which makes the polishing procedure more likely to succeed. Clear coats are too thin to do much polishing before you break through and begin seeing color on your buffer. Ask a follow-up question in the comments if you need more help.

This is only to remove the white paint and will obviously have little to no affect on any scratch marks that penetrated the paint. I have had good luck with paint thinner on a rag but it takes a LOT of rubbing. My experience is that this method works better than rubbing compound with which I have not had good results.