Was your colored cocoa butter properly tempered when you added it to the mold? That could cause it to stay in the mold and not adhere to the chcolate even if the chcolate was properly tempered.
As for cleaning them, I'd run hot water from the tap and fill the molds allowing them to sit. You can probably get most of the cocoa butter out this way or by using the sprayer. You will probably need to clean them with cotton balls really well to remove any stubborn bits.