The Breakdown
Important
- Scales of Measurement – These determine the appropriate statistical methods for analysis and data display, and identifying the scale of the dependent variable is a crucial initial step in research.
- Population vs. Sample Differentiation – It is a foundational concept because different statistical procedures and formulas are used depending on whether data is from an entire population or a selected sample.
- Parameters vs. Statistics – Parameters are statistical results from an entire population, possessing 100% accuracy with no associated error. Statistics, conversely, are results from a sample and inherently contain some error because they do not include every member of the population.
- Random Selection for Representation – Achieving a good representation of the population in a sample requires random selection to minimise bias.
- Sampling Error – This discrepancy between sample statistics and true population parameters is inherent in all samples and cannot be eliminated, but it can be minimised through methods like random sampling and using larger sample sizes.
- Continuous Scale Considerations – When measuring data on a continuous scale, it is crucial to consider that data points exist within intervals and to understand the concept of “real limits” (boundaries related to rounding).
- Scale Type Dictates Analysis – The type of measurement scale used for the dependent variable directly determines which statistical analyses and inferential statistics can be performed.
- Ratio Scale as Primary Objective – Collecting data on a ratio scale is often the primary objective because of its flexibility; it can be transformed into lower-level scales (ordinal or nominal) to address various research questions.
- Scale Transformation Direction – It is possible to transform data from higher-level scales (ratio, interval) to lower-level scales (ordinal, nominal), but the reverse transformation (e.g., nominal to ordinal) is not possible.
Core concepts
- Definition of Statistics: Statistics refers to the collection, organisation, and interpretation of information. It involves a set of procedures and rules for reducing large amounts of data into manageable proportions and drawing conclusions, as well as being the results of mathematical manipulation applied to data (e.g., the mean).
- Samples and Populations: A population is the entire collection of events a researcher is interested in, defined by the researcher’s interest (e.g., all Carleton students). A sample is a set of individuals selected from a population, intended to represent that population in a research study, as it’s often impractical to study an entire population.
- Sampling Error: This is the expected discrepancy between a statistic calculated from a sample and the true parameter of the entire population, due to not including every member of the population. It is inherent but can be minimised by employing random sampling, using reliable and valid methodologies, and opting for larger sample sizes.
- Scales of Measurement: These are crucial for determining appropriate statistical analyses. There are four types: nominal, ordinal, interval, and ratio.
- Descriptive Statistics: This branch of statistics organises, summarises, and communicates a group of numerical observations. Examples include averages, standard deviations, and ranges, used to describe data within both samples and populations.
- Inferential Statistics: This branch uses sample data to draw conclusions, make statements, and predict about the larger population. It helps determine if observed differences are due to chance or represent a real effect.
- Variables: Observations of physical, attitudinal, or behavioural characteristics that can take on different values.
- Independent Variable: The variable manipulated or observed by the researcher, always having two or more levels or groups.
- Dependent Variable: The variable observed or measured, which is expected to change in response to the independent variable. It is always a score or measurement.
- Confounding Variable: A variable that systematically varies with the independent variable, making it difficult to isolate the true effect on the dependent variable.
- Control Variables: Variables kept constant in an experiment to prevent them from influencing the dependent variable, thereby isolating the effect of the independent variable.
- Discrete Variables: Consist of separate, indivisible categories with no values in between them, often representing data that can be counted as frequencies (e.g., number of people on a bus). Nominal and ordinal variables are typically discrete, and sometimes ratio variables can also be discrete.
- Continuous Variables: Can assume an infinite number of values between any two points on a scale, typically involving decimal points (e.g., time, speed, length). Interval and ratio variables are generally continuous.
- Nominal Scale: Labels data for classification purposes only, where the labels or numbers have no intrinsic meaning or order other than to distinguish categories (e.g., types of cars, religious affiliation).
- Ordinal Scale: Orders data along a continuum (e.g., from lowest to highest) but does not provide information about the magnitude of differences between data points (e.g., rankings, star ratings).
- Interval Scale: Features legitimate differences between scale points, meaning the intervals between data points are of equal size. However, it has an arbitrary zero point, which does not signify the absolute absence of the measured attribute (e.g., temperature in Celsius or Fahrenheit, IQ scores).
- Ratio Scale: Possesses all characteristics of an interval scale but includes a true zero point, indicating the complete absence of the measured quantity. This allows for meaningful ratio comparisons (e.g., height, time, number of items).
- Reliability and Validity: A measure is considered useful if it is both reliable (consistent over time) and valid (accurately assesses what it intends to assess).
- Hypothesis Testing: A specific, statistics-based process for drawing conclusions about whether a particular relationship between variables is supported by evidence, typically by examining sample data to make inferences about a population.
- Random Assignment: A critical component of experimental designs where participants are assigned to different groups (e.g., treatment or control) purely by chance, ensuring each has an equal likelihood of being in any group. This helps to control for confounding variables and establish causality.
- Replication: The duplication of scientific results, ideally in a different context or with a sample having different characteristics, which builds confidence in the truth of an observation and is considered an important ethical data practice.
- Standardization and Z-Scores: Standardization is a method to convert individual scores from different normal distributions into a shared normal distribution, allowing for meaningful comparisons. A z-score quantifies the distance a raw score is from its distribution’s mean in terms of standard deviations.
- Central Limit Theorem: States that the distribution of sample means, when based on samples of 30 or more scores, will approximate a normal distribution, even if the original population distribution is not normal.
- Confidence Intervals: A type of interval estimate that provides a range of plausible values for a population parameter, giving insight into the uncertainty of a statistic. Researchers are increasingly encouraged to report these alongside or instead of traditional hypothesis testing.
- Effect Size: A measure of the magnitude of an observed effect, independent of sample size. It quantifies the practical significance of a finding (e.g., Cohen’s d).
- Statistical Power: The probability of correctly rejecting the null hypothesis when it should indeed be rejected. It is primarily influenced by sample size, and researchers often aim for at least 80% power before conducting a study.
- t-Tests: A family of hypothesis tests used when population standard deviation is unknown or when comparing two sample means.
- Single-Sample t-Test: Compares a sample mean to a known population mean.
- Paired-Samples t-Test: Compares two means from a within-groups design where the same participants are in both samples (e.g., before-and-after measurements).
- Independent-Samples t-Test: Compares the means of two independent groups where different participants are in each sample.
Theories and Frameworks
- VIPERR Approach to Learning Statistics: A pedagogical framework based on cognitive psychology research, using vivid examples, integration of new with existing knowledge, practice and participation, examination of misconceptions, real-time feedback, and repetition to enhance learning.
- Data Ethics/Open Science Movement: A set of guidelines and practices promoting transparent and ethical research across all stages of data work, including design, collection, analysis, interpretation, and reporting, often involving the sharing of research methodology and data.
- Severe Testing: A concept in statistics and philosophy of science that advocates for subjecting hypotheses to rigorous scrutiny to uncover flaws, embodying an ethical research approach focused on transparency and aggressive weakness detection.
- Yerkes-Dodson Law: A psychological principle illustrating a non-linear (curvilinear) relationship between arousal levels and performance, suggesting an optimal level of arousal for peak performance.
- Cohen’s Conventions: Guidelines for interpreting the practical significance of effect sizes (e.g., Cohen’s d), categorising them as small, medium, or large based on the degree of overlap between distributions.
- HARKing (Hypothesizing After the Results are Known): An unethical research practice where a researcher alters their initial hypotheses to align with unexpected study results, making it seem as if the results were predicted.
- p-Hacking: An unethical practice in research involving manipulating data analysis or experimental conditions until a statistically significant p-value (typically p < 0.05) is obtained.
- New Statistics: An evolving approach in the behavioral sciences that increasingly emphasizes reporting effect sizes, confidence intervals, meta-analyses, and statistical power, providing a more comprehensive understanding beyond just statistical significance.