Je ne parlais jamais avec elle car je ne la connaissais pas.
Ne... jamais is used together to mean never
De temps en temps, souvent and toujours are mostly not used in conjunction with ne and they also express a duration of time (from time to time, often, always) that does not make sense in this context where the speaker says that he didn't know the woman, thus 'jamais' is the only option left.
