I've found that eucalyptus or tea tree oil is good for removing glue residue from just about anything, including paintwork. Just rub it on, allow it to work for a while, then wipe it off. Keep repeating until all the glue is gone.

KJ.