hyper-shotgun

Pigeons, Birthdays, and Collisions

Ever stuffed too many socks in a drawer? That’s the Pigeonhole Principle in action, hiding everywhere from birthday parties to hash tables. This article explores collision mathematics through pigeons, birthdays, and feature hashing. We’ll show why collisions are inevitable (spoiler: sooner than you think) with interactive visualizations proving even 512-bit hashes eventually collade. Buckle up for a fun ride through the math governing how our digital world handles too much stuff in too little space.

2026-01-20

Joost de Theije + LLM

6 min

Mastering PySpark ⚡️ : Best Practices for Efficient Coding

PySpark is a powerful tool for processing massive datasets, but it presents unique challenges, especially for experienced Python developers. This guide provides best practices for efficient PySpark coding, helping you avoid common pitfalls and write clean, performant code. By following these tips, you’ll enhance your PySpark skills and improve the reliability and performance of your data processing tasks. As the PySpark syntax draws on the JVM heritage of Spark and therefore implements code patterns that may be unfamiliar. Adapted from here and here

2025-01-25

Joost de Theije + LLM

17 min

Multi-Git-Account-Mastery

Streamline Your Git Workflow. Manage multiple GitHub accounts and SSH keys using custom .gitconfig files. Learn how to configure global Git settings, include specific configuration files, and remap hostnames in your SSH config file. Automate your git workflow and say farewell to fiddeling around with SSH and GPG keys.

2024-04-23

Joost de Theije + LLM

3 min

Some points are more or less random than others

This article delves into various techniques for hyperparameter optimization, comparing the efficiency and effectiveness of Grid Search, Random Search, and Latin Hypercube Sampling in the context of gradient boosting models.

2024-03-14

Joost de Theije + LLM

8 min

Introduction to Partial Dependence Plots

This article provides an overview of Partial Dependence Plots (PDPs) and Individual Conditional Expectation (ICE) plots, powerful tools for visualizing the relationships between features and predictions in machine learning models. We’ll explore how these plots can help to interpret model behavior, revealing both average effects and individual data point responses.

2024-02-11

Joost de Theije + LLM

11 min

Boring linear forecast

Discover how the simple addition of dummy variables can transform a linear forecast from mundane to insightful. This article explores the significant impact of encoding seasonality into your models and provides a step-by-step guide on improving forecast accuracy.

2023-03-5

Joost de Theije + LLM

7 min