01 / live dashboard / 43,193 contacts / in your browser
Who actually says yes.
A Portuguese bank phoned people to sell a fixed-term savings account and logged 45,211 calls. Of the 43,193 with complete records, only 11.6% said yes. Filter by job, marital status, education, month or balance, and watch that number move.
Python · scikit-learn · SQLite · originally Dash and Plotly
Loading the dashboard…
Can a model tell who’ll say yes?
I trained three models on most of the calls and tested them on the rest, which they had never seen. The best was gradient boosting: hand it one person who said yes and one who said no, and it picks the right one about 92 times in 100.
Two things I checked before trusting that. 88% of people said no, so a model that always guesses no is right 88% of the time, which is why I didn’t score it on plain accuracy. And its strongest clue is how long the call lasted, which you only know once the call is over. Take that away and it still picks the right one about 76 times in 100.
The scores for all three models
| Model | Accuracy | Precision | Recall | F1 | ROC AUC |
|---|---|---|---|---|---|
| Logistic regressionScaled features, 1000 iterations. | 0.9027 | 0.6468 | 0.3576 | 0.4606 | 0.9099 |
| Decision treeUnpruned, on unscaled features. | 0.8713 | 0.4501 | 0.4851 | 0.4669 | 0.7036 |
| Gradient boostingScikit-learn defaults, scaled features. | 0.9064 | 0.6507 | 0.4193 | 0.5100 | 0.9164 |
Trained on 80% of the cleaned rows and scored on the other 20%, split so both halves carry the same yes rate. Without call length, gradient boosting scores 0.7645 ROC AUC.
What gradient boosting leaned on
- duration50.3%
- poutcome_success19.7%
- pdays6.6%
- housing_yes4.9%
- age4.8%
- month_mar4.5%
- month_oct1.6%
- month_sep1.3%
- balance1.0%
- month_may0.8%
Are those differences real, or luck?
Real. Job, education, having a home loan, and how the last campaign went each change how often people say yes, by far more than chance would explain. So the gaps in the charts above aren’t noise.
The 5 tests behind that
| Question | Test | Statistic | p | n |
|---|---|---|---|---|
| Balance by previous outcomeMean balance of contacts whose last campaign succeeded against those where it failed. | Welch t-test | 4.29 | 1.91e-5 | 6,133 |
| Job against subscriptionWhether subscription rate is independent of job. | Chi-squared | 772.49 | 1.69e-159 | 43,193 |
| Education against subscriptionWhether subscription rate is independent of education. | Chi-squared | 233.40 | 2.08e-51 | 43,193 |
| Housing loan against subscriptionWhether subscription rate is independent of housing loan. | Chi-squared | 825.28 | 1.72e-181 | 43,193 |
| Previous outcome against subscriptionWhether subscription rate is independent of previous outcome. | Chi-squared | 4027.72 | < 1e-300 | 43,193 |
How the data was cleaned.
The charts don’t read the raw file. 2,018 of the 45,211 calls were dropped because the person’s job or education wasn’t recorded, which leaves 43,193.
Every step, with what it cost
- 01loaded bank-full.csv45,211 rows
- 02dropped campaign and day45,211 rows
- 03removed unknown job44,923 rows · −288
- 04removed unknown education43,193 rows · −1,730
- 05reassigned unknown contact in proportion43,193 rows · 12,286 moved
- 06poutcome unknown folded into other43,193 rows
Source: UCI Bank Marketing (bank-full.csv). UCI Machine Learning Repository. Moro, S., Cortez, P. and Rita, P. (2014). A data-driven approach to predict the success of bank telemarketing. Decision Support Systems.