The Christians believed that Jerusalem was controlled by infidels because, at that time, Asia Minor was controlled by the Muslims which are the ones who follow or practice Islam (a monotheistic Abrahamic religion.) thus, in the Christians eyes they were unbelievers and infidels because they didn't believe in Christianity. Infidel is a term used in certain religions for those accused of unbelief when comes to their own religion.
