Time-Frequency Analysis for Signal Processing

Real-world signals often have frequency content that changes over time. Therefore, there is a need to describe signals jointly in time and frequency. Signal processing techniques for time-frequency analysis have been developed in response to this need and constitute a powerful tool for practitioners.

There is no unique or universally optimal time-frequency analysis technique. However, the proliferation of time-frequency analysis techniques should be regarded as an advantage. The signal processing engineer or data scientist is free to choose the method best suited to their type of data or application. In this talk we discuss several time-frequency analysis techniques and illustrate their application to common signal processing workflows. The theoretical underpinnings of these techniques and differences between them are highlighted to elucidate their strengths or weaknesses with respect to specific types of signals and applications. Finally, we discuss the important role that time-frequency analysis plays in AI applications with signals.

What this presentation is about and why it matters

This talk surveys practical time-frequency analysis techniques and explains why we need them when real signals change their spectral content over time. Traditional Fourier methods treat time and frequency separately and work best for stationary signals. Many real-world signals — speech, music, machinery vibrations, seismic traces, biomedical recordings, and spacecraft measurements — are nonstationary: their frequency content evolves. Time-frequency methods give you a joint view that answers questions like "which frequencies are present, and when do they occur?"

Wayne King walks through several widely used methods (short-time Fourier, continuous wavelet, reassignment/synchrosqueezing, data-adaptive multi-resolution methods like EMD) and shows where each shines or fails. For practitioners this matters because choosing the wrong representation can hide important events (transients, chirps, closely spaced tones) or introduce artifacts. The talk also highlights how time-frequency tools plug into modern machine learning workflows (2D CNNs on spectrograms, recurrent networks fed by wavelet channels, and differentiable transforms for end-to-end learning).

Who will benefit the most from this presentation

• Signal processing engineers who need to analyze nonstationary signals (audio, sonar, radar, vibration, seismic, biomedical).
• Data scientists and ML practitioners working with time-series who want to use time-frequency representations as features for classification, segmentation, or anomaly detection.
• Students learning applied signal processing who want intuition about trade-offs between time and frequency resolution.
• Researchers exploring phase retrieval, invertible transforms, or differentiable signal-processing layers for neural networks.

What you need to know

Reviewing a few core ideas before you watch will make the talk easier to follow.

• Time vs. frequency view: A raw time signal x(t) shows when events occur; its Fourier transform X(f) shows which frequencies exist but loses timing. Time-frequency representations bridge that gap.
• Stationarity: A stationary signal has statistical properties that do not change with time. Many practical signals are nonstationary, which motivates joint time-frequency analysis.
• Short-time Fourier transform (STFT): STFT computes local spectra by windowing the signal and taking a Fourier transform of each window. A common form is

  $X(t,f)=\int x(\tau)\,w(\tau-t)\,e^{-j2\pi f\tau}\,\mathrm{d}\tau$

  The magnitude-squared of the STFT is the spectrogram.
• Continuous wavelet transform (CWT): The CWT uses scaled and shifted versions of a mother wavelet to obtain a multi-scale view. One form is

  $W_x(a,b)=\frac{1}{\sqrt{a}}\int x(t)\,\psi^*\big(\frac{t-b}{a}\big)\,\mathrm{d}t$

  The squared magnitude is often called a scalogram.
• Resolution trade-off / uncertainty principle: You cannot simultaneously localize arbitrarily well in time and frequency. STFT has fixed time-frequency tiling (window size sets a constant resolution). Wavelets provide variable tiling (good frequency resolution at low frequencies, good time resolution at high frequencies).
• Reassignment and synchrosqueezing: These are post-processing methods that "sharpen" smeared energy in a time-frequency map by relocating energy to more precise frequency (or time) coordinates. Synchrosqueezing is notable because it can be invertible, enabling component extraction and reconstruction.
• Data-adaptive, multi-resolution decompositions: Techniques like empirical mode decomposition (EMD) or variational mode decomposition (VMD) adapt to the signal content and produce time–aligned components that can reconstruct the signal exactly. These are useful when templates (windows/wavelets) are not appropriate.
• Practical issues: Closely spaced tones, chirps with rapidly changing instantaneous frequency, and impulsive events favor different transforms. Watch for artifacts when a chosen method's assumptions are violated.

Glossary

• Time-frequency representation: Any 2D depiction of signal energy as a function of time and frequency (e.g., spectrogram, scalogram).
• Short-time Fourier transform (STFT): Local Fourier analysis using a sliding window; basis of the spectrogram.
• Spectrogram: Magnitude-squared of the STFT; commonly used 2D display for time-frequency energy.
• Continuous wavelet transform (CWT): A transform based on dilated and shifted wavelets that gives a multi-scale time-frequency view.
• Scalogram: Magnitude-squared of the CWT; shows energy across scales (frequencies) and time.
• Window function: The short-time analysis kernel (e.g., Gaussian, Hamming) that controls local time and frequency resolution.
• Wavelet: A localized oscillatory kernel used for multi-scale analysis; its dilation controls frequency localization.
• Uncertainty principle: A fundamental limit on simultaneous time and frequency localization (trade-off enforced by the analysis kernel).
• Synchrosqueezing (reassignment): A method that reassigns smeared time-frequency energy to sharper locations, often yielding ridge-like structures.
• Empirical mode decomposition (EMD): A data-adaptive decomposition that extracts intrinsic mode functions (AM-FM components) for Hilbert-based instantaneous-frequency analysis.

Final notes — why watch this talk

Wayne King provides a thoughtful, application-oriented tour of time-frequency tools. The talk balances core theory (tiling, uncertainty, transform definitions) with clear, visual examples and practical guidance about when to prefer STFT, wavelets, reassignment, or adaptive methods. I especially recommend this talk if you want to understand not just how each method works, but when it will help — and when it might mislead. The inclusion of modern ML uses and differentiable transforms makes the material immediately relevant to both classical signal-processing workflows and current data-driven pipelines.

This overview is AI-generated from the session transcript. Spot an issue? Let us know.