Subsampling

Subsampling means taking a subset of the data instead of using all of it — for speed, for a quick experiment on a hundred million rows, or as a step inside another procedure. In machine learning the subset is drawn at random; in signal processing the same word means keeping every k-th sample, which is decimation and a different operation entirely.

Within the machine learning sense there is a second ambiguity, and it is the one that bites.

With replacement or without

import numpy as np
from sklearn.datasets import make_classification
from sklearn.utils import resample

X, y = make_classification(n_samples=20000, n_features=8, n_informative=5,
                           weights=[0.98], flip_y=0.01, random_state=0)
rows = np.arange(len(y))

for label, kwargs in (("resample(...) as written", {}),
                      ("replace=False", {"replace": False}),
                      ("replace=False, stratify=y", {"replace": False, "stratify": y})):
    picked, labels = resample(rows, y, n_samples=2000, random_state=42, **kwargs)
    print(f"{label:<26} distinct rows {len(set(picked.tolist())):4d} of 2000"
          f"   event rate {labels.mean():.4f}")
print(f"{'the full dataset':<26} distinct rows {len(rows)} of {len(rows)}   event rate {y.mean():.4f}")
# resample(...) as written   distinct rows 1890 of 2000   event rate 0.0245
# replace=False              distinct rows 2000 of 2000   event rate 0.0220
# replace=False, stratify=y  distinct rows 2000 of 2000   event rate 0.0245
# the full dataset           distinct rows 20000 of 20000   event rate 0.0246

The first call is the one that appears in most tutorials, written as though it takes a subsample. It returned 1,890 distinct rows out of 2,000 — 110 draws repeated previously selected rows; 106 distinct original rows were selected more than once in this run. scikit-learn documents replace=True as the default and describes it as “one step of the bootstrapping procedure,” which is exactly what it is.

That is a legitimate operation and sometimes the one you want — bootstrapping is how a sampling distribution gets estimated. But it is not a subsample, and using it as one has consequences: duplicated rows appear on both sides of a subsequent split, and a model fitted on the result has seen some observations several times and others not at all. If the intent is “a smaller version of this dataset,” pass replace=False.

A random subset drifts

The second line is honest about a smaller effect. With replace=False every row is distinct, but the event rate came out at 0.0220 against a true 0.0246 — a tenth of the positives lost to chance, because at this base rate a 2,000-row sample holds only about fifty of them.

stratify=y fixes it: 0.0245, within rounding of the full dataset. On rare-event data this is not a refinement. Rate estimates, class priors and anything calibrated against the base rate all inherit the drift, and the drift is invisible unless you check for it.

What subsampling does not do

It is worth separating this from the undersampling family, because the two get listed side by side as if they were variants of one idea.

• Proportionate subsampling preserves the intended population prior in expectation. Stratification can keep class proportions close to the full dataset. This avoids deliberately fitting to a different class mix, but it does not guarantee calibrated probabilities: model specification, regularization and sampling error still matter.
• Undersampling changes it deliberately. Only the majority class shrinks, so the model’s probabilities describe a population that does not exist and need a prior correction to be usable.

Same verb, same function call, opposite effect on calibration. What separates them is whether the sampling depends on the label.

The cost subsampling does have is statistical power. Fewer rows means wider intervals on every estimate, and on a rare outcome the binding constraint is the count of minority rows, not the total — 2,000 rows holding fifty positives supports far less than the number 2,000 suggests. When a model is only being sanity-checked for runtime, that is fine; when a result is going to be reported, it is not.

In signals

Here subsampling means keeping every k-th sample, and a low-pass filter has to come first or frequencies above the new Nyquist limit fold down into the retained band. scipy.signal.decimate applies that filter and then thins; scipy.signal.resample works in the frequency domain instead, which handles the filtering implicitly but assumes the signal is periodic. Neither is a slice, and the article on downsampling shows what a slice does to a pure tone.

Reporting one

Three facts make a subsample reproducible and interpretable: the size, whether it was drawn with replacement, and whether it was stratified and on what. A line saying “trained on a 1% subsample” answers none of them, and the reader cannot tell whether the reported event rate is the data’s or the sampler’s.

References

• scikit-learn API reference, sklearn.utils.resample
• SciPy API reference, scipy.signal.decimate
• SciPy API reference, scipy.signal.resample