Modern CPUs burn most cycles idling because memory bandwidth can't keep up with compute throughput. The von Neumann bottleneck is real: arithmetic units sit on one side of a narrow bus, DRAM on the other, so every operation pays round-trip latency tax.
Processing-in-memory (PIM) drops compute cores directly onto DRAM dies. UPMEM's commercial chips prove the point: when you process millions of database rows or image pixels in-place across the memory array, data never crosses the bus. Cache-resident workloads see no gain, but once your dataset spills to main memory, PIM destroys conventional CPUs and GPUs on memory-bound kernels.
Dataflow architectures flip the script entirely. Instead of fetch-decode-execute loops, data streams through a mesh of simple processing nodes. Load once, compute multiple ops in-flight, store once. Perfect for dense matrix/tensor ops in ML training because intermediate results stay on-chip. Memory traffic collapses.
Neuromorphic chips like Intel Loihi merge storage and compute at the circuit level, mimicking biological synapses. Each asynchronous spiking core holds its own weights and talks directly to neighbors via spike events. Hala Point scales to 1B+ neurons, hits petabyte/sec internal bandwidth, sips single-digit kilowatts, and shows ~100x energy efficiency vs GPUs on sparse optimization and certain inference tasks.
Tradeoffs: PIM cores clock slower than CPUs and need embarrassingly parallel workloads. Dataflow fabrics choke on irregular control flow. Neuromorphic toolchains are immature and narrow. But all three share one insight: the bus is the enemy. Co-locate compute and memory on the same die, and the idle cycles vanish.
Processing-in-memory (PIM) drops compute cores directly onto DRAM dies. UPMEM's commercial chips prove the point: when you process millions of database rows or image pixels in-place across the memory array, data never crosses the bus. Cache-resident workloads see no gain, but once your dataset spills to main memory, PIM destroys conventional CPUs and GPUs on memory-bound kernels.
Dataflow architectures flip the script entirely. Instead of fetch-decode-execute loops, data streams through a mesh of simple processing nodes. Load once, compute multiple ops in-flight, store once. Perfect for dense matrix/tensor ops in ML training because intermediate results stay on-chip. Memory traffic collapses.
Neuromorphic chips like Intel Loihi merge storage and compute at the circuit level, mimicking biological synapses. Each asynchronous spiking core holds its own weights and talks directly to neighbors via spike events. Hala Point scales to 1B+ neurons, hits petabyte/sec internal bandwidth, sips single-digit kilowatts, and shows ~100x energy efficiency vs GPUs on sparse optimization and certain inference tasks.
Tradeoffs: PIM cores clock slower than CPUs and need embarrassingly parallel workloads. Dataflow fabrics choke on irregular control flow. Neuromorphic toolchains are immature and narrow. But all three share one insight: the bus is the enemy. Co-locate compute and memory on the same die, and the idle cycles vanish.