Forecasting Labor Market Data: Initial Claims and Non-Farm Payrolls

[WITH CODE] Creating and testing statistical models for prediction markets

Reviewed and updated 20 July 2026

Hello!

Welcome back to another post on forecasting labor market data. This will be the final post in this series.

Today, we will be expanding and improving our forecasts for initial jobless claims, as well as creating models to forecast monthly non-farm payrolls data.

The government shutdown has definitely put a damper on this project, as I have been unable to test the model in live conditions (and take views with the information from these models).

The research companion is available to paid members below. As always, this is for educational purposes only and is not financial advice.

Let’s get into it.

---

Recap of Last Post

In last week’s research, I highlighted how Choi & Varian (2009) showed Google searches for unemployment-related terms lead official claims and improve simple AR forecasts, how Aaronson, Brave, Butters et al. used hurricanes as a natural experiment to separate news-driven search spikes from true job loss and calibrate Google Trends, and how Larson & Sinclair (2022) found that during COVID the best nowcasts included data from emergency declarations, with standard AR models catching up only later. In stable regimes, high-frequency signals like search can add value, but in unstable regimes, localized or event-driven data perform better.

I also reported that naïve (last week equals this week) is a strong benchmark on weekly claims, as most linear models underperformed. Additionally, continuing claims added little, and AR(1) offered only a small improvement in some windows. To map forecasts to Kalshi’s “at least X” structure, I used ARIMA’s one-step mean and standard error to estimate threshold probabilities and compared them with unconditional hit rates since January 2025.

---

Improving Initial Jobless Claims Forecasting

Once the plain AR work was in place, I decided to try out a more complex model. Specifically, I tried using XGBoost to forecast weekly jobless claims. However, I do not wish to overcomplicate the model (and potentially overfit the data). Therefore, I wanted to keep the features clean, use lagged versions of the series we care about, add a light seasonal rhythm, and see if a flexible model could edge past the benchmark. The benchmark, of course, was the model that forecasted next week’s claims as the prior week’s claim. However, this crude model is annoyingly difficult to beat.

I began with seasonally adjusted initial claims as the target. I built lags out to four weeks and added a sine and cosine to reflect the weekly pattern through the year. I also added holiday and calendar flags because filings and holidays have a messy relationship.

While the initial claims data is already seasonally adjusted, I wanted to see if there was more seasonality to remove from the data. In the later 2025-YTD comparison described below, the model using a single prior seasonally adjusted claims print produced the lowest error. That is a short revised-data result, not a real-time validation.

Two earlier XGBoost comparisons are excluded because their features were not available at the stated forecast origin. In the first, an unshifted four-week average included the target value in the same row, and the scored test set also controlled early stopping. In the second, a broad column filter retained raw same-week initial and continuing claims even for models labelled as lag-only. The XGB_SA_l1_4 chart used that same feature set. I removed both tables and the chart because their error rankings are not one-week-ahead forecasting evidence.

The separate 2025-YTD comparison uses an explicit one-week lag and training-tail validation. Across 38 revised-data weeks, XGB_SA_l1 recorded MAE of 8,117 versus 8,605 for Naive and RMSE of 10,632 versus 11,108. I treat that modest result as exploratory because the window is short and the specification was selected from several alternatives.

I ran a bunch more tests that included seasonality variables, but they just hurt the performance of all the models, and made them underperform the Naive model. I wasn’t happy with this performance yet (and was in a flow state for a few hours), so I added more data to see if it would improve our models.

Therefore, I gathered two series of job posting data from Indeed: Job Postings and New Job Postings. I have attached a chart of these series below. As expected, new job postings are more volatile than job postings.