Campaigners say banning residents aged 65 and over from taking trips outside homes is unlawful.