The Rise of Computational Science

When the first electronic computers came online in the 1940s, they did not merely speed up existing science. They invented an entirely new way of doing science. Where theory and experiment had been the two pillars of the scientific method since Galileo, a third pillar appeared: simulation.

This page tells the story of how that pillar was raised, and introduces vocabulary you will see throughout the rest of the course — time stepping, discretization, Monte Carlo, stiff systems — all of which were invented in the first decade of computational science.

ENIAC and the Manhattan Project

The Electronic Numerical Integrator and Computer (ENIAC), unveiled at the University of Pennsylvania in 1946, could perform about 5,000 additions per second — roughly as many in one second as a human computer would do in a month. It was built to compute ballistics tables for the U.S. Army, but its first major problem was different: a thermonuclear weapons calculation for Los Alamos.

That calculation, performed in late 1945 by Stan Frankel, Nick Metropolis, and Anthony Turkevich, ran for six weeks and produced a million numerical results. The methods invented to run it — punch-card input, batched arithmetic, deck-shuffling for parallelism — are still recognizable in modern HPC pipelines.

The Monte Carlo revolution

In 1946, while recovering from an illness, Stanislaw Ulam was playing solitaire and wondering what fraction of randomly dealt hands could be won. The combinatorics were intractable. Instead, he realized, he could just deal a lot of hands and count.

He mentioned the idea to John von Neumann, who immediately saw it could be applied to neutron diffusion — a problem Los Alamos was struggling with. Within weeks they had a name (Monte Carlo, after the casino), an algorithm, and ENIAC running the first production code.

Monte Carlo is shockingly easy to demonstrate. Here is a five-line program that estimates $\pi$ by throwing random darts at a square and counting how many land inside the inscribed circle.

The remarkable property is that this works for any integral, in any number of dimensions, with an error that shrinks as $1/\sqrt{N}$ regardless of dimension. We will return to Monte Carlo in its own chapter near the end of the course.

Numerical weather forecasting

In 1950, a small team led by John von Neumann and Jule Charney used ENIAC to produce the first computer weather forecast. They discretized the atmosphere over North America into a grid of about 270 cells, used a simplified set of equations called the barotropic vorticity equation, and ran a 24-hour forecast.

It took 24 hours to compute — a real-time forecast that finished just in time to be irrelevant. But it was qualitatively correct, and it kicked off a research program that, six decades later, underlies every weather app on your phone.

Discretizing a continuous problem into a grid and stepping through time is still the dominant pattern in computational science. Here is the simplest possible version: the 1-D heat equation showing heat diffusing along a metal rod.

Notice the comment on the time-step dt. Pick a dt that is too large and the simulation explodes — the numbers grow without bound after a few steps even though the physics says the heat should just smoothly diffuse. Try editing 0.4 to 0.6 and re-running. This is your first encounter with a stability condition, a concept invented in the late 1940s and now central to every PDE solver in existence.

A new scientific method

By the late 1950s the three-pillar picture was widely accepted. A theorist could write down equations; a computational scientist could simulate them at scales experiments could not reach; and an experimentalist could test the predictions. The interplay between the three is now so tight that fields like computational chemistry and computational fluid dynamics are research disciplines in their own right, with Nobel Prizes attached.

The hardware ladder

Computational science is uniquely sensitive to hardware. Every time processors got faster or memory got cheaper, a new class of problem became tractable. The ladder, roughly:

Each rung is about a thousand times faster than the one before it, and each one unlocked a research program that ran for a decade or two.

What the early years left us

The first decade of computational science gave us a remarkably durable toolkit:

• Time stepping — march a system forward through small increments of time.
• Discretization — replace continuous fields with grids.
• Iteration — solve nonlinear problems by repeatedly applying cheaper linear ones.
• Monte Carlo — use random sampling to estimate integrals and rare events.
• Stability analysis — understand when a method amplifies error and when it doesn't.

Every one of these will appear in this course. None of them requires a supercomputer to learn — they all run beautifully in the Pyodide kernel powering this page.

Check your understanding

QuestionSelect one

The first 24-hour numerical weather forecast in 1950 took 24 hours of compute time. What does the page identify as the deepest lesson from that experiment?

That computers were too slow to be useful for weather

That discretizing a continuous physical problem onto a grid and stepping it forward in time was a viable scientific method

That the barotropic vorticity equation was not a useful approximation

That FORTRAN was necessary for weather modeling

QuestionSelect one

In the 1-D heat equation example, why does the simulation explode when dt is set too large?

The grid spacing becomes too coarse

The numerical scheme violates the CFL stability condition: each step amplifies, rather than damps, the high-frequency error modes

Python runs out of memory

Matplotlib cannot plot the values