If I understand what you are asking, you can use the median as an average since it is not affected by the mean ( median is a measure of center). Since a mean is affected by outliers, it might lower the score if there are very low outliers. But note that if there are outliers way greater it can increase the average. You can use the Inter Quartile Range if shown on a box plot ( measure of variability).
