The best explanation is the fact that photons of radiation with higher
frequencies (shorter wavelengths) carry more energy.
Photons of visible light don't carry enough energy to damage cells,
no matter how many of them hit you. But above the frequencies of
visible light, each photon carries enough energy to damage a cell.
UV photons can just barely do it, X-ray photons can do it more often,
and gamma ray photons are deadly for sure.