Outliers, which are data values that are far away from other data values, can strongly affect your results. On a scatterplot, isolated points identify outliers.
Try to identify the cause of any outliers. Correct any data-entry errors or measurement errors. Consider removing data values that are associated with abnormal, one-time events (special causes). Then, repeat the analysis.