hypothesis testing statistics penn state

Geography Department Penn State

• Instructor Information
• Using the Penn State Library
• Class Communication Guidelines
• Getting Help!

Hypothesis Testing: Randomization Distributions

Like bootstrap distributions, randomization distributions tell us about the spread of possible sample statistics. However, while bootstrap distributions originate from the raw sample data, randomization distributions simulate what sort of sample statistics values we should see if the null hypothesis is true . This is key to using randomization distributions for hypothesis testing, where we can then compare our actual sample statistic (from the raw data) to the range of what we'd expect if the null hypothesis were to be true (i.e., the randomization distribution).

Randomization Procedure 

The procedure for generating a randomization distribution, and subsequently comparing to the actual sample statistic for hypothesis testing, is depicted in the figure below.

The pseudo-code (general coding steps, not written in a specific coding language) for generating a randomization distribution is:

Obtain a sample of size n

For i in 1, 2, ..., N

Manipulate and randomize sample so that the null hypothesis condition is met. I t is important that this new sample has the same size as the original sample (n).

Calculate the statistic of interest for the ith randomized sample

Store this value as the ith randomized statistic

Combine all N randomized statistics into the randomization distribution

Here, we've set the number of randomization samples to N = 1000, which is safe to use and which you can use as the default for this course. The validity of the randomization distribution depends on having a large enough number of samples, so it is not recommended to go below N = 1000. In the end, we have a vector or array of sample statistic values; that is our randomization distribution.

As we'll see in the subsequent pages, we'll use different strategies for simulating conditions under the assumption of the null hypothesis being true (Step 2.1 above). The choice of strategy will depend on the type of testing we're doing (e.g., single mean vs. single proportion vs. ...). Generally speaking, the goal of each strategy is to have the collection of sample statistics agree with the null hypothesis value on average, while maintaining the level of variability contained in the original sample data. This is really important, because our goal with hypothesis testing is to see what could occur just by random chance alone, given the null conditions are true , and then compare our data (representing what is actually happening in reality) to that range of possibilities. 

• The randomization distribution is centered on the value in the null hypothesis (the null value).
• The spread of the randomization distribution describes what sample statistic values would occur with random chance and given the null hypothesis is true.

The figure below exemplifies these key features, where the histogram represents the randomization distribution and the vertical red dashed line is on the null value.

The p-value

Our goal with the randomization distribution is to see how likely our actual sample statistic is to occur, given that the null hypothesis is true. This measure of likelihood is quantified by the p-value:

Let's elaborate on some important aspects of this definition and provide guidance on how to determine the p-value:

• The p-value is a probability, so it has a value between 0 and 1.
• This probability is measured as the proportion of samples in the randomization distribution that are at least as extreme as the observed sample (from the original data)
• If the alternative is < (a.k.a. "left-tailed test"), the p-value = the proportion of samples ≤ the sample statistic.
• If the alternative is > (a.k.a. "right-tailed test"), the p-value = the proportion of samples ≥ the sample statistic.
• If the alternative is ≠ (a.k.a. "two-tailed test"), the p-value = twice the smaller of: the proportion of samples ≤ the sample statistic or the proportion of samples ≥ the sample statistic

The default threshold for rejecting or not rejecting the null hypothesis is 0.05, refer to as the "significance level" (more on this later). Thus,

• If the p-value < 0.05, we can reject the null hypothesis (in favor of the alternative hypothesis)
• If the p-value > 0.05, we fail to reject the null hypothesis

Although it should be noted that some researchers are moving away from the classical paradigm and starting to think of the p-value on more of a continuous scale, where smaller values are more indicative of rejecting the null. 

 Assess It: Check Your Knowledge

• All bulletins
• Undergraduate
• Penn State Law
• Dickinson Law
• College of Medicine
• Departments

Statistics (STAT)

Statistics is the art and science of decision making in the presence of uncertainty. The purpose of Statistics 100 is to help students improve their ability to assess statistical information in both everyday life and other University courses. Topics covered include methods for collecting and summarizing data, analyzing the relationship between variables, and using basic probability concepts to draw conclusions about populations based on data. The course is less technical and more conceptual than Statistics 200. Statistical concepts and interpretations will dominate over techniques and calculations ¿ but students should be comfortable working with fractions and square roots.

Bachelor of Arts: Quantification

General Education: Quantification (GQ)

GenEd Learning Objective: Crit and Analytical Think

GenEd Learning Objective: Key Literacies

R is a powerful, open-source programming language used widely for applications in statistics and data science. It is easily extendible, and thousands of user-created packages are publicly available to extend its capabilities. This course will introduce students to data computing fundamentals and a reproducible workflow using the R programming language and related tools. Students will be expected to access, join, wrangle, clean, and visualize real data from various sources (e.g. CSV, HTML scraping, web URL, R packages). The course will emphasize use of "tidyverse" R packages (e.g. dplyr, ggplot2), although students will also be exposed to Base R and other packages. In addition, students will be exposed to one or more integrated development environments (e.g. RStudio) and will be expected to write well-documented code using a reproducible workflow (e.g. RMarkdown, Git/GitHub). The course focuses on descriptive and graphical summary techniques rather than inferential statistical techniques

Enforced Prerequisite at Enrollment: Placement into MATH 21 or higher

Courses offered in foreign countries by individual or group instruction.

International Cultures (IL)

Descriptive statistics, frequency distributions, probability, binomial and normal distributions, statistical inference, linear regression, and correlation. STAT 200 is a standard first course in statistics. Student who have successfully completed this course will understand basic concepts of probability and statistical inference, including common graphical and numerical data summaries; notions of sampling from a population of interest, including the sampling distribution of a statistic; construction and interpretation of confidence intervals, test statistics, and p-values; and connections between probabilistic concepts like the normal distribution and statistical inference. They will recognize various types of data, appropriate statistical methods to analyze them, and assumptions that underlie these methods, They will also gain extensive experience in the use of statistical software to analyze data and the interpretation the output of this software.

Statistical analysis, sampling, and experimentation in the agricultural sciences; data collection, descriptive statistics, statistical inference, regression, one factor AOV, probability. Students may take only one course from STAT 200 , 220, 240 , 250 for credit. STAT 240 Introduction to Biometry (3) (GQ)(BA) This course meets the Bachelor of Arts degree requirements. This is a course concerned with statistical analysis pertaining to the natural and agricultural sciences. The objective of the course is to provide students with a good basis for understanding uncertainty and its effects on understanding observational studies and experiments. Course content includes data collection, descriptive statistics, statistical inference, regression, and ANOVA. Students will learn through lectures, individual and group problem solving, computer-based activities, and case study discussions. Since real-life use of statistics relies upon computers, this course will provide a strong hands-on analysis element necessitating regular access to computer labs. The statistical background gained by students will provide them with a base for future use of statistics in both their course work and careers.

Statistical analysis and interpretation of data in the biological sciences; probability; distributions; statistical inference for one- and two-sample problems. STAT 250 is a standard first course in statistics, with an emphasis on applications and statistical techniques of particular relevance to the biological sciences. Students who have successfully completed this course will understand basic concepts of probability and statistical inference, including common graphical and numerical data summaries; notions of sampling from a population of interest, including the sampling distribution of a statistic; construction and interpretation of confidence intervals, test statistics, and p-values; and connections between probabilistic concepts such as normal distributions and statistical inference. They will recognize various types of data, appropriate statistical methods to analyze them, and assumptions that underlie these methods. Students will use statistical software to analyze real data.

Creative projects, including research and design, that are supervised on an individual basis and that fall outside the scope of formal courses.

Formal courses given infrequently to explore, in depth, a comparatively narrow subject that may be topical or of special interest.

This course is designed to serve as a bridge between introductory statistics (including AP statistics) and more advanced applied statistics courses. The course will emphasize applied statistical modeling for real data using computer software (e.g. R, Minitab). Broad statistical topics include simple linear regression, multiple linear regression, analysis of variance (ANOVA) and factorial designs, logistic regression, multiple linear regression.

Enforced Prerequisite at Enrollment: STAT 100 or STAT 200 or STAT 240 or STAT 250 or SCM 200 or PSYCH 200 or DS 200 or IE 323

Combinatorial analysis, axioms of probability, conditional probability and independence, discrete and continuous random variables, expectation, limit theorems, additional topics. Students who have passed either MATH(STAT) 414 or 418 may not schedule this course for credit.

Enforced Prerequisite at Enrollment: MATH 141

Cross-listed with: MATH 318

Statistical inference: principles and methods, estimation and testing hypotheses, regression and correlation analysis, analysis of variance, computer analysis. Students who have passed STAT (MATH) 415 may not schedule this course for credit.

Enforced Prerequisite at Enrollment: MATH 318 or STAT 318 or MATH 414 or STAT 414 or STAT 418 or MATH 418

Cross-listed with: MATH 319

A case study-based course in the use of computing and statistcal reasoning to answer data-intensive questions. STAT 380 Data Science Through Statistical Reasoning and Computation (3) This course addresses the fact that real data are often messy by taking a holistic view of statistical analysis to answer questions of interest. Various case studies will lead students from the computationally intensive process of obtaining and cleaning data, through exploratory techniques, and finally to rudimentary inferential statistics. This process will exploit students' exposure to introductory statistics as well as the R programming language -hence the required prerequisites- yet novel computing and analytical techniques will also be introduced throughout the course. For the collection of data, students will learn scripting and database querying skills; for their exploration, they will employ R capabilities for graphical and summary statistics; and for their analysis, they will build upon the basic concepts obtained in their introductory statistics course. The varied case studies will elucidate additional statistical topics such as identifying sources of bias and searching for high-dimensional outliers. A possible textbook for this course is Data Science in R: A Case Studies Approach to Computational Reasoning and Problem Solving (2015) by Deborah Nolan and Duncan Temple Lang.

Enforced Prerequisite at Enrollment: ( STAT 200 or DS 200 or PLSC 309 ) and STAT 184

This course is intended to build directly upon STAT 300 (Applied Statistical Modeling I) for students pursuing a major in statistics or a closely related program. Topics include likelihood-based inference, generalized linear models, random and mixed effects modeling, multilevel modeling. In particular, the applied nature of the course seeks to examine the advantages and disadvantages of various modeling tools presented, identify when they may be useful, use R software to implement them for analysis of real data, evaluate assumptions, interpret results, etc.

Enforced Prerequisite at Enrollment: STAT 184 and MATH 220 and ( STAT 300 or STAT 462 )

Random variables; probability density functions; estimation; statistical tests, t-tests; correlation; simple linear regression; one-way analysis of variance; randomized blocks.

Enforced Prerequisite at Enrollment: MATH 111 or MATH 141

STAT(MATH) 414 is an introduction to the theory of probability for students in statistics, mathematics, engineering, computer science, and related fields. The course presents students with calculus-based probability concepts and those concepts can be used to describe the uncertainties present in real applications. Topics include probability spaces, discrete and continuous random variables, transformations, expectations, generating functions, conditional distributions, law of large numbers, central limit theorems. Students may take only one course from STAT(MATH) 414 and 418.

Enforced Prerequisite at Enrollment: MATH 230 or Concurrent: MATH 232 or ( MATH 231 and RM 214 )

Cross-listed with: MATH 414

A theoretical treatment of statistical inference, including sufficiency, estimation, testing, regression, analysis of variance, and chi-square tests.

Enforced Prerequisite at Enrollment: MATH 414 or STAT 414

Cross-listed with: MATH 415

Review of distribution models, probability generating functions, transforms, convolutions, Markov chains, equilibrium distributions, Poisson process, birth and death processes, estimation.

Enforced Prerequisite at Enrollment: ( STAT 318 or MATH 318 or STAT 414 or MATH 414 ) and ( MATH 230 or MATH 232 )

Cross-listed with: MATH 416

Introduction to probability axioms, combinatorics, random variables, limit laws, and stochastic processes. Students may take only one course from MATH414 / STAT 414 and MATH 418 / STAT 418 for credit. STAT 418 / MATH 418 Introduction to Probability and Stochastic Processing for Engineering (3) This course gives an introduction to probability and random processes. The topics are not covered as deeply as in a semester-long course in probability only or in a semester-long course in stochastic processes only. It is intended as a service course primarily for engineering students, though no engineering background is required or assumed.The topics covered include probability axioms, conditional probability, and combinatorics; discrete random variables; random variables with continuous distributions; jointly distributed random variables and random vectors; sums of random variables and moment generating functions; and stochastic processes, including Poisson, Brownian motion, and Gaussian processes.

Enforced Prerequisite at Enrollment: MATH 230 or MATH 231

Cross-listed with: MATH 418

Topics related to computing in statistics, including numerical linear algebra, optimization, simulation, numerical integration, and bootstrapping. STAT 440 Computational Statistics (3)This course introduces many important ideas in statistical computing. Students are expected to possess knowledge of mathematical statistics at the level of STAT 415 and matrices at the level of MATH 220 . Students will learn the statistical computing environment called R and use R to implement many of the theoretical computing topics, which include numerical linear algebra, optimization, numerical and Monte Carlo integration, random number generation and simulation, and bootstrapping. Other statistical and mathematical software may be treated briefly, including symbolic mathematics environments like Mathematics and Maple.

Enforced Prerequisite at Enrollment: STAT 200 and MATH 220 and ( STAT 415 or MATH 415 )

Review of hypothesis testing, goodness-of-fit tests, regression, correlation analysis, completely randomized designs, randomized complete block designs, latin squares.

Enforced Prerequisite at Enrollment: STAT 200 or STAT 240 or STAT 250 or STAT 401

Analysis of variance for single and multifactor designs; response surface methodology.

Introduction to linear and multiple regression; correlation; choice of models, stepwise regression, nonlinear regression.

Identification of models for empirical data collected over time; use of models in forecasting. STAT 463 Applied Time Series Analysis (3)This course covers many major topics in time series analysis. Students will learn some theory behind various time series models and apply this theory to multiple examples. An introduction to time series and exploratory data analysis will be followed by a lengthy study of several important models, including autoregressive, moving average, autoregressive moving average (ARMA), autoregression integrated moving average (ARIMA), and seasonal models. For each model methods for parameter estimation, forecasting, and model diagnostics will be covered. Additional topics will include spectral techniques for periodic time series, including power spectra and the Fourier transform, and one or more miscellaneous topics chosen by the instructor, such as forecasting methods, transfer function models, multivariate time series methods, Kalman filtering, and signal extraction and forecasting. The use of statistical software will be a central component of this course, as will the proper interpretation of computer output. Students enrolling for this course are assumed to have taken a semester-long course on regression.

Enforced Prerequisite at Enrollment: ( STAT 300 or STAT 462 ) and ( STAT 318 or MATH 318 or STAT 414 or MATH 414 or STAT 418 or MATH 418 )

Tests based on nominal and ordinal data for both related and independent samples. Chi-square tests, correlation.

Introduction to design and analysis of sample surveys, including questionnaire design, data collection, sampling methods, and ratio and regression estimation. STAT 466 Survey Sampling (3)This course covers classical sampling design and analysis methods useful for research and management in many fields. Topics include design of questionnaires; methods of data collection, sample-survey designs including simple random sampling, stratified sampling, cluster sampling, and systematic sampling ratio, regression, and difference estimation; two-stage cluster sampling; population size estimation; methods for dealing with nonresponse; and possibly other topics at the discretion of the instructor. Statistical software will be used to apply many of the techniques covered by this course.

This is a capstone course intended primarily for undergraduate statistics majors in their last semester prior to graduation. The course is designed to reinforce problem solving and communication skills through development of writing ability, interaction with peers and the SCC, statistical consulting center (SCC), and oral presentations. Course objectives are tailored to the needs of each cohort and may include the application of statistical reasoning to real-world problems and case studies, recognition or recommendation of appropriate experimental designs, proficient use of ANOVA & GLMs with understanding of associated modeling assumptions, ability to identify concerns about the use or interpretation of statistical models in context, and both written and verbal communication of statistical findings.

Enforced Prerequisite at Enrollment: STAT 461 and STAT 462 and 7th Semester standing

Writing Across the Curriculum

Introduction to SAS with emphasis on reading, manipulating and summarizing data. STAT 480 Introduction to SAS (1) STAT 480 addresses the fundamentals of the SAS programming language. It addresses the programming environment and major aspects of the Base SAS software, including reading in, manipulating, and transforming data. It also addresses techniques for reshaping and restructuring data files, merging and concatenating data sets, creating summaries and subsets of data sets, formatting and printing data, as well as using some of the basic statistical procedures.

Enforced Prerequisite at Enrollment: 3 credits in Statistics

Intermediate SAS for data management. STAT 481 Intermediate SAS for Data Management (1) STAT 481 builds on the skills and tools learned in STAT 480 to provide intermediate level ability to use the Statistical Analysis System (SAS). It covers additional capability and major uses of the program, such as error checking, report generation, date and time processing, random number generation, and production of presentation quality output for graphs and tables. Other possible topics include advanced merging, PROC SQL, importing and exporting data sets, SAS GRAPH, and the Output Delivery System.

Enforced Concurrent at Enrollment: STAT 480

Advanced statistical procedures in SAS, including ANOVA, GLM, CORR, REG, MANOVA, FACTOR, DISCRIM, LOGISTIC, MIXED, GRAPH, EXPORT, and SQL. STAT 482 Advanced Topics in SAS (1) STAT 482 builds on the skills and tools learned in STAT 480 and STAT 481 to provide advanced programming ability to use the Statistical Analysis System (SAS). It provides a survey of the major statistical analysis procedures, such as the TTEST, GLM, REG, MANOVA, FACTOR, DISCRIM, LOGISTIC, and MIXED procedures. Other topics include using the TABULATE procedure to create reports, generating random numbers, exporting data from SAS data sets, using the SAS/Graph module to produce presentation quality graphs, using the SQL procedure to query and combine data tables, and using macros to write more efficient SAS programs. Credit can not be received for both STAT 482 and STAT 480 /481/483.

Enforced Concurrent at Enrollment: STAT 480 and STAT 481

Introduction, intermediate, and advanced topics in SAS. Credit can not be received for both STAT 483 and STAT 480 /481/482. STAT 483 Statistical Analysis System Programming (3) The three-credit STAT 483 course is a combination of the three one-credit courses STAT 480 , STAT 481 , and STAT 482 . In STAT 480 , students are introduced to the SAS windowing system, basic SAS programming statements, and descriptive reporting procedures, such as the FORMAT, PRINT, REPORT, MEANS, and FREQ procedures. In STAT 481 , the focus is primarily on extending the programming skills of the students, as they learn how to read messy data into SAS data sets, how to combine SAS data sets in various ways, how to use SAS character functions, how to read and process date and time variables, how to use arrays and do loops to write more efficient programs, and how to use the Output Delivery System to create SAS output in a variety of formats. STAT 482 provides a survey of the major statistical analysis procedures, such as the TTEST, GLM, REG, MANOVA, FACTOR, DISCRIM, LOGISTIC, and MIXED procedures. Other STAT 482 topics include using the TABULATE procedure to create reports, generating random numbers, exporting data from SAS data sets, using the SAS/Graph module to produce presentation quality graphs, using the SQL procedure to query and combine data tables, and using macros to write more efficient SAS programs. Credit can not be received for both STAT 483 and STAT 480 /481/482.

Builds an understanding of the basic syntax and structure of the R language for statistical analysis and graphics. R is a popular tool for statistical analysis and research used by a growing number of data analysts inside corporations and academia. The flexibility and extensibility of R are key attributes that have driven its adoption in a wide variety of fields. This course begins with an overview of the R language and the basics of R programming. Building upon these basic understandings and procedures, this course then provides students with hands on experience in implementing statistical analysis of data in univariate, bivariate and multivariate contexts using the R software. In addition, the course works through accessing, importing and manipulating data. Documentation of work and report writing are also important aspects of the course content, and R Markdown is utilized to illustrate best practices.

Builds an understanding of the basic syntax and structure of the R language for statistical analysis and graphics. R is a popular tool for statistical analysis and research used by a growing number of data analysts inside corporations and academia. The flexibility and extensibility of R are keys attributes that have driven its adoption in a wide variety of fields. This course begins extends the application of statistical analyses by providing students with hands on experience implementing R in various regression and ANOVA contexts. In addition, data visualization options are considered for producing customized graphics and simple programming is learned. Documentation of work and report writing is also an important aspect of the course content.

Enforced Concurrent at Enrollment: STAT 484

Due to the pervasiveness of Python as a statistical analysis tool, there is a demand for statisticians to learn Python to perform descriptive and inferential data analysis. The course will take a case study approach to introduce students to Python. Students will learn to work with complex data using Python and will get hands-on experience on how to use Python to conduct statistical analyses.

Enforced Prerequisite at Enrollment: STAT 300 or STAT 460 or STAT 461 or STAT 462 or STAT 500

Supervised student activities on research projects identified on an individual or small group basis.

Enforced Prerequisite at Enrollment: 6 credits in Statistics

Supervised off-campus, nongroup instruction including field experiences, practica, or internships.

Creative projects, including research and design, which are supervised on an individual basis and which fall outside the scope of formal courses.

Formal courses given infrequently to explore, in depth, a comparatively narrow subject which may be topical or of special interest.

Print Options

Print this page.

The PDF will include all information unique to this page.

Download Complete Bulletin PDFs

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Hypothesis Testing | A Step-by-Step Guide with Easy Examples

Published on November 8, 2019 by Rebecca Bevans . Revised on June 22, 2023.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics . It is most often used by scientists to test specific predictions, called hypotheses, that arise from theories.

There are 5 main steps in hypothesis testing:

• State your research hypothesis as a null hypothesis and alternate hypothesis (H o ) and (H a  or H 1 ).
• Collect data in a way designed to test the hypothesis.
• Perform an appropriate statistical test .
• Decide whether to reject or fail to reject your null hypothesis.
• Present the findings in your results and discussion section.

Though the specific details might vary, the procedure you will use when testing a hypothesis will always follow some version of these steps.

Table of contents

Step 1: state your null and alternate hypothesis, step 2: collect data, step 3: perform a statistical test, step 4: decide whether to reject or fail to reject your null hypothesis, step 5: present your findings, other interesting articles, frequently asked questions about hypothesis testing.

After developing your initial research hypothesis (the prediction that you want to investigate), it is important to restate it as a null (H o ) and alternate (H a ) hypothesis so that you can test it mathematically.

The alternate hypothesis is usually your initial hypothesis that predicts a relationship between variables. The null hypothesis is a prediction of no relationship between the variables you are interested in.

• H 0 : Men are, on average, not taller than women. H a : Men are, on average, taller than women.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

For a statistical test to be valid , it is important to perform sampling and collect data in a way that is designed to test your hypothesis. If your data are not representative, then you cannot make statistical inferences about the population you are interested in.

There are a variety of statistical tests available, but they are all based on the comparison of within-group variance (how spread out the data is within a category) versus between-group variance (how different the categories are from one another).

If the between-group variance is large enough that there is little or no overlap between groups, then your statistical test will reflect that by showing a low p -value . This means it is unlikely that the differences between these groups came about by chance.

Alternatively, if there is high within-group variance and low between-group variance, then your statistical test will reflect that with a high p -value. This means it is likely that any difference you measure between groups is due to chance.

Your choice of statistical test will be based on the type of variables and the level of measurement of your collected data .

• an estimate of the difference in average height between the two groups.
• a p -value showing how likely you are to see this difference if the null hypothesis of no difference is true.

Based on the outcome of your statistical test, you will have to decide whether to reject or fail to reject your null hypothesis.

In most cases you will use the p -value generated by your statistical test to guide your decision. And in most cases, your predetermined level of significance for rejecting the null hypothesis will be 0.05 – that is, when there is a less than 5% chance that you would see these results if the null hypothesis were true.

In some cases, researchers choose a more conservative level of significance, such as 0.01 (1%). This minimizes the risk of incorrectly rejecting the null hypothesis ( Type I error ).

The results of hypothesis testing will be presented in the results and discussion sections of your research paper , dissertation or thesis .

In the results section you should give a brief summary of the data and a summary of the results of your statistical test (for example, the estimated difference between group means and associated p -value). In the discussion , you can discuss whether your initial hypothesis was supported by your results or not.

In the formal language of hypothesis testing, we talk about rejecting or failing to reject the null hypothesis. You will probably be asked to do this in your statistics assignments.

However, when presenting research results in academic papers we rarely talk this way. Instead, we go back to our alternate hypothesis (in this case, the hypothesis that men are on average taller than women) and state whether the result of our test did or did not support the alternate hypothesis.

If your null hypothesis was rejected, this result is interpreted as “supported the alternate hypothesis.”

These are superficial differences; you can see that they mean the same thing.

You might notice that we don’t say that we reject or fail to reject the alternate hypothesis . This is because hypothesis testing is not designed to prove or disprove anything. It is only designed to test whether a pattern we measure could have arisen spuriously, or by chance.

If we reject the null hypothesis based on our research (i.e., we find that it is unlikely that the pattern arose by chance), then we can say our test lends support to our hypothesis . But if the pattern does not pass our decision rule, meaning that it could have arisen by chance, then we say the test is inconsistent with our hypothesis .

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Descriptive statistics
• Measures of central tendency
• Correlation coefficient

Methodology

• Cluster sampling
• Stratified sampling
• Types of interviews
• Cohort study
• Thematic analysis

Research bias

• Implicit bias
• Cognitive bias
• Survivorship bias
• Availability heuristic
• Nonresponse bias
• Regression to the mean

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

A hypothesis states your predictions about what your research will find. It is a tentative answer to your research question that has not yet been tested. For some research projects, you might have to write several hypotheses that address different aspects of your research question.

A hypothesis is not just a guess — it should be based on existing theories and knowledge. It also has to be testable, which means you can support or refute it through scientific research methods (such as experiments, observations and statistical analysis of data).

Null and alternative hypotheses are used in statistical hypothesis testing . The null hypothesis of a test always predicts no effect or no relationship between variables, while the alternative hypothesis states your research prediction of an effect or relationship.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 22). Hypothesis Testing | A Step-by-Step Guide with Easy Examples. Scribbr. Retrieved August 12, 2024, from https://www.scribbr.com/statistics/hypothesis-testing/

Is this article helpful?

Rebecca Bevans

Other students also liked, choosing the right statistical test | types & examples, understanding p values | definition and examples, what is your plagiarism score.

• Diversity and Inclusion
• Faculty Positions
• Location and Maps
• Our Department
• Graduate Programs
• How to Apply
• Online Programs
• Student Resources
• Grad Student Assoc.
• Graduate Student Highlights
• Frequently Asked Questions
• Online Courses
• Prospective Students
• Undergraduate Programs
• Integrated Degree
• Research Opportunities
• Statistics Club
• Syllabus Archive
• Career Development
• Administrative Faculty
• Graduate Students
• M.A.S. Students
• Affiliated Faculty
• Lindsay Assistant Professors
• Online Instructors
• Post-docs / Visitors
• Astrostatistics
• Bayesian Statistics
• Biostatistics & Bioinformatics
• Business Analytics
• Computational Statistics
• Functional Data Analysis
• High Dimensional Data
• Imaging Science
• Social Science
• Spatial and Spatiotemporal Data

• Statistical Network Science
• Statistical and Machine Learning
• Statistics Education
• Giving to Statistics

Dept. of Department of Statistics

Runze Li is Eberly Family Chair Professor of Statistics at Penn State.

Li received his Ph.D. in Statistics from University of North Carolina at Chapel Hill in 2000.

Li's research interest includes variable selection and feature screening for high dimensional data, nonparametric modeling and semiparametric modeling and their application to social behavior science research. He is also interested in longitudinal data analysis and survival data analysis and their application to biomedical data analysis.

Li joined Penn State as an assistant professor of statistics in 2000, and became associate professor, full professor, distinguished professor and Verne M. Willaman Professor of Statistics in 2005, 2008, 2012 and 2014, respectively. Since 2018, he is the Eberly Family Chair Professor of Statistics. He received his NSF Career Award in 2004. He is a fellow of IMS, ASA and AAAS. He was co-editor of Annals of Statistics, and served as associate editor of Annals of Statistics and Statistica Sinica. He currently serves as associate editor of JASA and Journal of Multivariate Analysis.

Honors and Awards

• The United Nations' World Meteorological Organization Gerbier-Mumm International Award for 2012
• Editor of The Annals of Statistics (2013 - 2015)
• Highly Cited Researcher in Mathematics (2014 - )
• ICSA Distinguished Achievement Award, 2017
• Faculty Research Recognition Awards for Outstanding Collaborative Research. College of Medicine, Penn State University, 2018
• IMS Medallion Lecturer at Joint Statistical Meetings, August 5-10, 2023 in Toronto
• Distinguished Mentoring Award, Eberly College of Science, Penn State University, 2023
• Fellow, IMS, ASA and American Association for the Advancement of Science

Publications

• Zhong, W., Qian, C., Liu, W., Zhu, L. and Li, R. (2023). Feature screening for interval-valued response with application to study association between posted salary and required skills. Journal of American Statistical Association. 118, 805 - 817.
• Sheng, B., Li, C., Bao, L. and Li, R. (2023). Probabilistic HIV recency classification - a logistic regression without labelled individual level training data. Annals of Applied Statistics. 17, 108-129.
• Guo, X, Li, R, Liu, J. and Zeng, M. (2023). Statistical inference for linear mediation models with high-dimensional mediators and application to studying stock reaction to COVID-19 pandemic. Journal of Econometrics. 235, 166-179.
• Bao, L, Li, C., Li, R. and Yang, S. (2022). Causal structural learning on MPHIA individual dataset. Journal of American Statistical Association. 117, 1642-1655.
• Li, C. and Li, R. (2022). Linear hypothesis testing in linear models with high dimensional responses. Journal of American Statistical Association. 117, 1738-1750.
• Guo, X., Li, R., Liu, J. and Zeng, M. (2022). High-dimensional mediation analysis for selecting DNA methylation Loci mediating childhood trauma and cortisol stress reactivity. Journal of American Statistical Association. 117, 1110-1121.
• Nandy, D., Chiaromonte, F. and Li, R. (2022). Covariate information number for feature screening in ultrahigh-dimensional supervised problems. Journal of American Statistical Association. 117, 1516 - 1529.
• Ren, H., Zou, C., Chen, N. and Li, R. (2022). Large-scale data streams surveillance via pattern-oriented-sampling. Journal of American Statistical Association. 117, 794-808.
• Liu, W., Ke, Y., Liu, J. and Li, R. (2022). Model-free feature screening and FDR control with knockoff features. Journal of American Statistical Association. 117(537), 428-443.
• Zou, T, Lan, W, Li, R. and Tsai, C.-L. (2022). Inference on Covariance-Mean Regression. Journal of Econometrics. 230, 318 - 338.
• Liu, W., Yu, X. and Li, R. (2022). Multiple-splitting project test for high dimensional mean vectors. Journal of Machine Learning and Research. 23(71), 1-27.
• Cai, Z., Li, R. and Zhang, Y. (2022). A distribution free conditional independence test with applications to causal discovery. Journal of Machine Learning and Research. 23(85), 1-41.
• Shi, C., Song, R., Lu, W. and Li, R. (2021). Statistical inference for high-dimensional models via recursive online-score estimation. Journal of American Statistical Association. 116, 1307 - 1318.
• Li, Z., Wang, Q. and Li, R. (2021). Central limit theorem for linear spectral statistics of large dimensional Kendall's rank correlation matrices and its applications. Annals of Statistics. 49, 1569 -1593.
• Xiao, D., Ke, Y. and Li, R. (2021). Homogeneity structure learning in large-scale panel data with heavy-tailed errors. Journal of Machine Learning Research. 22 22(13):1-42, 2021.
• Wang, L., Peng, B., Bradic, J., Li, R. and Wu, Y. (2020). A tuning-free robust and efficient approach to high-dimensional regression (with discussions and rejoinder). Journal of American Statistical Association. 115, 1700 - 1729.
• Fang, X. E., Ning, Y. and Li, R. (2020). Test of signi cance for high-dimensional longitudinal data. Annals of Statistics. 48, 2622 - 2645.
• Zhou, T., Zhu, L., Xu, C. and Li, R. (2020). Model-free forward regression via cumulative divergence. Journal of American Statistical Association. 115, 1393 - 1405.
• Zou, C., Wang, G. and Li, R. (2020). Consistent selection of the number of change-points via sample-splitting. Annals of Statistics. 48, 413-439.
• Cui, X., Li, R., Yang, G. and Zhou, W. (2020). Empirical likelihood test for large dimensional mean vector. Biometrika. 107, 591 - 607.
• Wang, L., Chen, Z., Wang, C. D. and Li, R. (2020). Ultrahigh dimensional precision matrix estimation via refitted cross validation. Journal of Econometrics. 215, 118-130.
• Chu, W., Li, R., Liu, J. and Reimherr, M. (2020). Feature screening for generalized varying coefficient mixed effect models with application to obesity GWAS. Annals of Applied Statistics. 14, 276 - 298.
• Cai, Z, Li, R. and Zhu, L. (2020). Online sufficient dimension reduction through sliced inverse regression. Journal of Machine Learning and Research. 21(10), 1-25.
• Li, X., Li, R., Xia, Z. and Xu, C. (2020). Distributed feature screening via componentwise debiasing. Journal of Machine Learning and Research. 21 (24), 1-32
• Zhong, P.-S., Li, R. and Santo, S. (2019). Homogeneity test of covariance matrices and change-points identification with high-Dimensional longitudinal data. Biometrika. 106, 619 - 634.
• Zheng, S., Chen, Z., Cui, H. and Li, R. (2019). Hypothesis testing on linear structures of high dimensional covariance matrix. Annals of Statistics. 47, 3300 - 3334.
• Shi, C., Song, R., Chen, Z. and Li, R. (2019). Linear hypothesis testing for high dimensional generalized linear models. Annals of Statistics. 47, 2671 - 2703.
• Liu, H., Wang, X., Yao, T., Li, R. and Ye, Y. (2019). Sample average approximation with sparsity-inducing penalty for high-dimensional stochastic programming. Mathematical Programming, 78, 69-108.
• Chu, W., Li, R. and Reimherr, M. (2016). Feature screening for time-varying coefficient models with ultrahigh dimensional longitudinal data. Annals of Applied Statistics, 10, 596 - 617.
• Li, R., Zhong, W. and Zhu, L. (2012). Feature screening via distance correlation learning. Journal of American Statistical Association. 107, 1129 - 1139.
• Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models (with discussion). Annals of Statistics, 36, 1509-1566.
• Li, R. and Liang, H. (2008). Variable selection in semiparametric regression modeling. Annals of Statistics. 36, 261-286
• Fan, J. and Li, R. (2006). Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery. Proceedings of the International Congress of Mathematicians (M. Sanz-Sole, J. Soria, J.L. Varona, J. Verdera, eds.), Vol. III, European Mathematical Society, Zurich, 595-622.
• Li, R. and Sudjianto, A. (2005). Analysis of computer experiments using penalized likelihood in Gaussian kriging Models. Technometrics. 47, 111-120.
• Fan, J. and Li, R. (2004). New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. Journal of American Statistical Association, 99, 710-723.
• Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and it oracle properties, Journal of American Statistical Association. 96, 1348-1360.
• Cai, Z., Fan, J. and Li, R. (2000). Efficient estimation and inferences for varying coefficient models. Journal of the American Statistical Association. 5, 888-902.

Stat 565 - Multivariate Analysis

Stat 597 - Statistical Foundations of Data Science

Stat 597 - Statistical Inference on High-dimensional Data

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

• Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

5.1 - hypothesis testing overview.

  Jin asked Carlos if he had taken statistics, Carlos said he had but it was a long time ago and he did not remember a lot of it. Jin told Carlos understanding hypothesis testing would help him understand what the judge just said. In most research, a researcher has a “research hypothesis”, that is, what the research THINKS is going to occur because of some kind of intervention or treatment. In the courtroom the prosecutor is the researcher, thinking the person on trial is guilty. This would be the research hypothesis; guilty. However, as most of us know, the U.S. legal system operates that a person is innocent until PROVEN guilty. In other words, we have to believe innocence until there is enough evidence to change our mind that the person on trial is actually not innocent. In hypothesis testing, we refer to the presumption of innocence as the NULL HYPOTHESIS. So while the prosecutor has a research hypothesis, it must be shown that the presumption of innocence can be rejected.

Like the judge in the TV show, if we have enough evidence to conclude that the null is not true, we can reject the null. Jin explained that if the judge had enough evidence to conclude the person on trial was not innocent she would have. The judge specifically stated that she did not have enough evidence to reject innocence (the null hypothesis).

When the judge acquits a defendant, as on the T.V. show, this does not mean that the judge accepts the defendant’s claim of innocence. It only says that innocence is plausible because guilt has not been established beyond a reasonable doubt.

On the other hand, if the judge returns a guilty verdict she has concluded innocence (null) is not plausible given the evidence presented, therefore she rejects the statute of the null, innocence and concludes the alternative hypothesis- guilty .

Let’s take a closer look at how this works.

Making a Decision Section  

Taking a sample of 500 Penn State students, we asked them if they like cold weather, we observe a sample proportion of 0.556, since these students go to school in Pennsylvania it might generally be thought the true proportion of students who like cold weather is 0.5, in other words the NULL hypothesis is that the true population proportion equal to 0.5 ,

In order to “test” what is generally thought about these students (half of them like cold weather) we have to ask about the relationship of the data we have (from our sample) relative to the hypothesized null value. In other words, is our observed sample proportion far enough away from the 0.5 to suggest that there is evidence against the null? Translating this to statistical terms, we can think about the “how far” questions in terms of standard deviations. How many standard deviations apart would we consider to be “meaningfully different”?

What if instead of a cutoff standard deviation, we found a probability? With a null hypothesis of equal to 0.5, the alternative hypothesis is not equal to 0.50. To test this, we convert the distance between the observed value and the null value into a standardized statistic. We have worked with standardized scores when working with z scores. We also learned about the empirical rule. Combining these two concepts, we can begin to make decisions about “how far” the observed value and null hypothesis need to be to be “meaningfully different”.

To do this we calculate a Z statistic, which is a standardized score of the difference.

\(z^{*}=\dfrac{\hat{p}-p_{0}}{\sqrt{\frac{p_{0}\left(1-p_{0}\right)}{n}}}\)

We can look at the results of calculating a z test (which we will do using software). Large test statistics indicate a large difference between the observed value and the null, contributing to greater evidence of a significant difference, thus casting doubt that the true population proportion is the null value.

Accompanying the magnitude of the test statistic, our software also yields a “probability”. Returning to the values of the empirical rule we know the percentiles under a standard normal curve. We can apply these to determine the probability (which is really a percentile) of getting an observed score IF the null hypothesis is indeed true (or the mean of the distribution). In this class, we will not be calculating these by hand, but we do need to understand what the “p-values'' in the output mean. In our example, after calculating a z statistic, we determine that if the true proportion is 0.5, the probability we would get a sample proportion of 0.556 is 0.0061. This is a very small probability as measure against the standard defining “small” as a probability less than .05. In this case, we would reject the null hypothesis as a probable value for the population based on the evidence from our sample.

While p values are a standard in most statistics courses and textbook there have been recent conversations about the use of p values. 

  American Statistical Association Releases Statement on Statistical Significance and P-Values

The use of p-values is a common practice in statistical inference but also not without its controversy. In March of 2016, the American Statistical Association released a statement regarding p-values and their use in statistical studies and decision making.

You can review the full article: ASA Statement on p-Values: Context, Process and Purpose

P-Values Section  

Before we proceed any further we need to step away from the jargon and understand exactly what the heck a p value is. Simply a p value is the probability of getting the observed sample statistic,  given the null hypothesis is true . In our example, IF the true proportion of Penn State students who like the cold IS really .5 (as we state in the null hypothesis), what is the probability that we would get an observed sample statistic of .556?

When the probability is small we have one of two options. We can either conclude there is something wrong with our sample (however, if we followed good sampling techniques as discussed early in the notes then this is not likely) OR we can conclude that the null is probably not the true population value. 

To summarize the application of the p value:

• If our p-value is less than or equal to \(\alpha \), then there is enough evidence to reject the null hypothesis (in most cases the alpha is going to be 0.05).
• If our p-value is greater than \(\alpha \), there is not enough evidence to reject the null hypothesis.

One should be aware that \(\alpha \) is also called level of significance. This makes for a confusion in terminology. \(\alpha \) is the preset level of significance whereas the p-value is the observed level of significance. The p-value, in fact, is a summary statistic which translates the observed test statistic's value to a probability which is easy to interpret.

We can summarize the data by reporting the p-value and let the users decide to reject \(H_0 \) or not to reject \(H_0 \) for their subjectively chosen \(\alpha\) values.

• Help & FAQ

Linear hypothesis testing for high dimensional generalized linear models

• Department of Public Health Sciences
• Penn State Cancer Institute
• Cancer Institute, Cancer Control

Research output : Contribution to journal › Article › peer-review

This paper is concerned with testing linear hypotheses in high dimensional generalized linear models. To deal with linear hypotheses, we first propose the constrained partial regularization method and study its statistical properties. We further introduce an algorithm for solving regularization problems with folded-concave penalty functions and linear constraints. To test linear hypotheses, we propose a partial penalized likelihood ratio test, a partial penalized score test and a partial penalized Wald test. We show that the limiting null distributions of these three test statistics are χ 2 distribution with the same degrees of freedom, and under local alternatives, they asymptotically follow noncentral χ 2 distributions with the same degrees of freedom and noncentral parameter, provided the number of parameters involved in the test hypothesis grows to ∞ at a certain rate. Simulation studies are conducted to examine the finite sample performance of the proposed tests. Empirical analysis of a real data example is used to illustrate the proposed testing procedures.

All Science Journal Classification (ASJC) codes

• Statistics and Probability
• Statistics, Probability and Uncertainty

Access to Document

• 10.1214/18-AOS1761

Other files and links

• Link to publication in Scopus
• Link to the citations in Scopus

Fingerprint

• Linear Hypothesis Mathematics 100%
• Generalized Linear Model Business & Economics 97%
• Hypothesis Testing Business & Economics 76%
• High-dimensional Mathematics 60%
• Partial Mathematics 49%
• Regularization Business & Economics 48%
• Degree of freedom Mathematics 44%
• Empirical Analysis Mathematics 34%

T1 - Linear hypothesis testing for high dimensional generalized linear models

AU - Shi, Chengchun

AU - Song, Rui

AU - Chen, Zhao

AU - Li, Runze

N1 - Publisher Copyright: © Institute of Mathematical Statistics, 2019.

N2 - This paper is concerned with testing linear hypotheses in high dimensional generalized linear models. To deal with linear hypotheses, we first propose the constrained partial regularization method and study its statistical properties. We further introduce an algorithm for solving regularization problems with folded-concave penalty functions and linear constraints. To test linear hypotheses, we propose a partial penalized likelihood ratio test, a partial penalized score test and a partial penalized Wald test. We show that the limiting null distributions of these three test statistics are χ2 distribution with the same degrees of freedom, and under local alternatives, they asymptotically follow noncentral χ2 distributions with the same degrees of freedom and noncentral parameter, provided the number of parameters involved in the test hypothesis grows to ∞ at a certain rate. Simulation studies are conducted to examine the finite sample performance of the proposed tests. Empirical analysis of a real data example is used to illustrate the proposed testing procedures.

AB - This paper is concerned with testing linear hypotheses in high dimensional generalized linear models. To deal with linear hypotheses, we first propose the constrained partial regularization method and study its statistical properties. We further introduce an algorithm for solving regularization problems with folded-concave penalty functions and linear constraints. To test linear hypotheses, we propose a partial penalized likelihood ratio test, a partial penalized score test and a partial penalized Wald test. We show that the limiting null distributions of these three test statistics are χ2 distribution with the same degrees of freedom, and under local alternatives, they asymptotically follow noncentral χ2 distributions with the same degrees of freedom and noncentral parameter, provided the number of parameters involved in the test hypothesis grows to ∞ at a certain rate. Simulation studies are conducted to examine the finite sample performance of the proposed tests. Empirical analysis of a real data example is used to illustrate the proposed testing procedures.

UR - http://www.scopus.com/inward/record.url?scp=85072180069&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072180069&partnerID=8YFLogxK

U2 - 10.1214/18-AOS1761

DO - 10.1214/18-AOS1761

M3 - Article

C2 - 31534282

AN - SCOPUS:85072180069

SN - 0090-5364

JO - Annals of Statistics

JF - Annals of Statistics