Professor Oops

Sound and shaking

Sound and shaking

A speaker shakes the paper, and salt draws the figure by itself

Hold the speaker on the paper and the whole sheet starts to shake, in a pattern set by the note and by where you press. Salt hops off the parts that move and settles on the lines that stay still, so a figure appears that nobody drew. Flour does the opposite. And every sound effect you hear is made by code at that moment, from what just happened.

Salt on a shaking sheet gathered into a symmetrical figure around the speaker in the middle, at 519 Hz

The speaker only shakes the sheet, and its code never mentions salt

Sieve salt over the page, pick up the speaker and hold it in the middle. You hear a steady note. Within seconds the salt runs together into lines and curls, a symmetrical figure like the ones Ernst Chladni made famous with sand on a vibrating plate.

The surprising part is that the speaker's code knows nothing about salt. It does one thing: it sets the sheet shaking, hard in some places and not at all along certain lines. Everything lying on the paper reads that shaking in its own way. A salt grain gets bounced off the parts that move and comes to rest where the sheet is still. No line of code describes the figure. It's what's left once every grain has hopped away from the shaking.

A test holds the code to this. It reads the speaker's source file and fails if it ever mentions grains, foam, soap bubbles, water, ripples or particles. It also reads ten simulation files, among them the grains, the foam, the puddles and the shake map itself, and fails if any of them mentions the speaker. A second test sets the sheet ringing with no speaker on the page at all, and glitter still drifts to the still lines.

The rule each grain follows is written by hand, and it's short: when the sheet under you shakes hard enough, take a step toward where it shakes less. Salt doesn't fly through the air in the game, and it doesn't find the lines by chance. The figure is the only part that nobody wrote.

A speaker shakes the paper, and salt draws the figure by itself

For engineers

A speaker shakes the paper, and salt draws the figure by itself

The speaker writes one thing: a 64 × 48 shake map, the sum of up to 224 cosine vibration shapes, each weighted by a narrow resonance in log frequency and by how much it moves under the press point. Loose grains are a height field on the paper grid (512 × 384 at full size) that steps down the slope of that map if coarse, up it if fine. An automated test reads the speaker's code and fails if it names the grains, glitter, foam or water that lie on the sheet. No sound waves are computed, and the game contains no recorded sound effects.

Under the hood
The speaker only shakes the sheet, and its code never mentions salt

Under the hood

The speaker only shakes the sheet, and its code never mentions salt

One shake map per page: a 64 × 48 grid of how hard each coarse patch of paper shakes, in the game's own "g". Two sources add up in every patch: a standing figure, which only the speaker writes and which holds until it's replaced, and short local shakes that fade. The speaker's figure also sets how long the ripples on puddles are. When nothing shakes, the map goes to sleep and nothing reads it.

The grain rule, in words, for each cell of loose grains:

  1. Wet or frozen grains stay put, and so do grains on a sheet that barely moves.

  2. Shaken very hard, some grains are thrown a few cells in a random direction.

  3. Shaken moderately, a heap steeper than its angle of repose slides down.

  4. Past a bounce threshold, a share of the grains steps one cell along the shake map's slope, downhill for coarse grains, uphill for fine ones, never up a step steeper than the grain can stand.

The share that moves grows with the shaking, up to a cap, and fine powder moves more slowly than coarse grains. Each pass visits part of the cells, in turn, at a lower rate than the frames. Moves are gathered and applied after the pass, so nothing moves twice.

Try it: hold a speaker on salt The game's own rule

The sheet starts covered in salt, with the speaker already playing in the middle. Drag on the paper to move the speaker, or press Move the speaker; slide the note, or switch to sand or flour. What to look for: salt leaves the shaking parts and draws the still lines; flour does the reverse. Turn the volume right down and nothing moves. Turn it right up and grains get thrown about. This is the game's plate formula and grain rule, ported line for line, on a 160 × 120 sheet instead of 512 × 384. Only the checks for wet, frozen and holes are left out, since this sheet has none.

Salt, sand or flour on a sheet shaken by a speaker; drag to move the speaker
0.768
note
—
0.66
On the sheet
clarity
—
strongest shaking
—
grain on still lines
—
Salt on the paper gathered into a symmetrical lacy figure around the speaker held in the centre
Salt, speaker in the middle, default note. The released game after eight seconds. The salt lies where the sheet stands still.
Cocoa powder covering the paper except for thin bare lines that follow the same figure, around the speaker held in the centre
Cocoa, same note, same spot. The picture turns inside out: fine powder gathers where the sheet shakes and leaves the still lines bare. Flour does the same, but white on white paper it hardly shows.

Only some notes make a clean figure, and where you press changes it

Slide the note up and the figure doesn't morph smoothly. It snaps in, goes weak and muddled, and then another one snaps in. Real plates do this too. A plate has a set of natural ways to vibrate, each with its own note, and it only answers strongly when you play close to one of them.

The game's sheet has 224 of these shapes. Each is a pattern of stripes across and down the sheet. For a speaker pressed in the middle of the page, 17 places along the slider give a clean figure. Between them the sheet barely answers, and the shaking is a small fraction of a clean figure's.

Where you press matters as much as the note. A shape that is still under the speaker can't be set going from there. Press in the exact middle and only shapes that are symmetric about the middle can answer, which is why centred figures look so tidy. Move the speaker off-centre and other shapes join in, so the figure changes and can lose its symmetry. An automated test checks both.

The figure the slider starts on is a mix. At 519 Hz two shapes, one with six still lines across and two down, the other with four each way, have notes only 3 % apart. Both answer at once, and their sum gives the lacy figure instead of a plain grid. Higher notes broadly give more lines, though not at every step.

Under the hood
Only some notes make a clean figure, and where you press changes it

Under the hood

Only some notes make a clean figure, and where you press changes it

The plate is a closed formula: no solver, no state that changes over time. Each of the 224 shapes is a stripe pattern across times a stripe pattern down, with up to fourteen still lines each way. Its natural note follows a plate's frequency law, so shapes with more lines ring higher, and the sheet's side ratio sets which notes lie close together.

The slider picks a note on a log scale. Each shape answers with a narrow resonance peak around its own note (a Lorentzian in log frequency), and its weight is that answer times how much the shape moves under the press point. The figure is the weighted sum, normalised. Only the size of the shaking is kept, so the still lines are where the sum crosses zero.

Clarity, the strongest single answer, is the only thing that scales the strength, so a weak answer keeps its shape but shakes less.

Because every shape splits into a row part and a column part, a recompute is cheap, and it only runs when the note changes or the press point moves noticeably. An automated test checks that two fingers on the page hardly ever trigger one.

The note you hear is a separate scale. The model's range is wider than the ear's; they rise together, but the note is not the frequency the model sheet rings at.

Try it: find the sheet's clean notes The game's own rule

This is the shake map itself, as the game works it out. White is where the sheet stands still, where salt would gather; red is where it shakes hardest. Drag to move the press point, or press Move the press point, and step from one clean note to the next with the buttons. What to look for: press on a white line and the figure weakens or changes. Switch the view to "up and down" to see that neighbouring patches swing in opposite directions, with the still lines running between them. The readout names the strongest shape in the mix by its stripes across and down.

The shake map of the sheet: white still lines, red where it shakes most; drag to move the press point
0.768
note
—
View
model F
—
clarity
—
modes in the sum
—
strongest: stripes across
—
stripes down
—
its weight
—
Salt figure on the paper with the speaker held off-centre, near the top left, giving a lopsided pattern
The same note, pressed off-centre. Other shapes join in, so the released game draws a different figure, and it is no longer symmetric about the middle.
Clean figures sit on narrow peaks along the note sliderBetween peaks the sheet answers weakly, and the figure shakes less.

For a speaker in the middle, 17 narrow peaks between 90 and 880 Hz reach a clarity of 0.6 or more; between them the clarity falls far lower. The slider starts on the peak at 519 Hz.

Clean figures sit on narrow peaks along the note slider0.00.20.40.60.81.0100200300400500600700800the note you hear (Hz)claritystart, 519 HzClean figures sit on narrow peaks along the note slider0.00.20.40.60.81.0200400600800the note you hear (Hz)claritystart, 519 Hz
Data table
Clean figures sit on narrow peaks along the note slider
the note you hear (Hz)speaker in the middle of a 4:3 sheet
900.003
90.50.003
910.003
91.60.003
92.10.004
92.60.004
93.10.004
93.70.004
94.20.004
94.70.004
95.30.004
95.80.004
96.40.005
96.90.005
97.50.005
980.005
98.60.005
99.20.006
99.70.006
100.30.006
100.90.006
101.40.007
1020.007
102.60.007
103.20.008
103.80.008
104.40.009
1050.009
105.60.01
106.20.01
106.80.011
107.40.011
1080.012
108.60.013
109.20.014
109.90.015
110.50.016
111.10.017
111.80.018
112.40.02
1130.021
113.70.023
114.30.025
1150.028
115.70.03
116.30.033
1170.037
117.70.041
118.30.046
1190.052
119.70.059
120.40.068
121.10.078
121.70.092
122.40.108
123.10.13
123.80.158
124.60.195
125.30.245
1260.315
126.70.412
127.40.544
128.20.712
128.90.887
129.60.994
129.81
130.40.961
131.10.811
131.90.633
132.60.48
133.40.364
134.10.281
134.90.221
135.70.177
136.40.144
137.20.119
1380.1
138.80.085
139.60.073
140.40.064
141.20.056
1420.049
142.80.044
143.60.039
144.40.035
145.30.032
146.10.029
146.90.026
147.80.024
148.60.022
149.50.021
150.30.019
151.20.018
152.10.016
152.90.015
153.80.015
154.70.016
155.60.017
156.50.018
157.30.02
158.20.022
159.10.023
160.10.026
1610.028
161.90.031
162.80.034
163.70.038
164.70.042
165.60.047
166.60.053
167.50.06
168.50.069
169.40.08
170.40.094
171.40.111
172.40.134
173.40.163
174.30.202
175.30.256
176.30.33
177.40.432
178.40.571
179.40.743
180.40.913
181.40.999
181.70.999
182.50.941
183.50.781
184.60.604
185.60.458
186.70.348
187.80.269
188.80.212
189.90.17
1910.139
192.10.116
193.20.097
194.30.083
195.40.072
196.50.062
197.60.055
198.80.048
199.90.043
2010.038
202.20.035
203.40.031
204.50.028
205.70.026
206.90.024
2080.026
209.20.029
210.40.032
211.60.035
212.80.039
214.10.044
215.30.049
216.50.056
217.80.064
2190.073
220.20.085
221.50.1
222.80.119
2240.144
225.30.176
226.60.22
227.90.28
229.20.363
230.50.478
231.80.631
233.20.809
234.50.959
235.61
235.80.995
237.20.889
238.50.714
239.90.546
241.30.413
242.70.316
2440.246
245.40.195
246.80.158
248.30.13
249.70.108
251.10.092
252.50.078
2540.068
255.40.059
256.90.052
258.40.046
259.80.041
261.30.037
262.80.038
264.30.042
265.80.048
267.40.054
268.90.061
270.40.07
2720.081
273.50.095
275.10.113
276.70.136
278.20.166
279.80.206
281.40.261
2830.337
284.70.442
286.30.584
287.90.758
289.60.925
291.21
292.90.93
294.60.766
296.20.591
297.90.447
299.60.341
301.30.264
303.10.208
304.80.167
306.50.137
308.30.114
310.10.096
311.80.082
313.60.071
315.40.062
317.20.054
3190.048
320.80.043
322.70.038
324.50.036
326.40.04
328.20.044
330.10.05
3320.057
333.90.065
335.80.074
337.70.087
339.70.102
341.60.122
343.60.147
345.50.181
347.50.226
349.50.288
351.50.375
353.50.494
355.50.651
357.60.83
359.60.971
360.81
361.70.989
363.70.871
365.80.694
367.90.529
3700.4
372.10.307
374.20.239
376.40.19
378.50.154
380.70.127
382.90.106
385.10.117
387.30.14
389.50.172
391.70.214
393.90.272
396.20.352
398.50.463
400.70.612
4030.789
405.30.946
407.21
407.60.998
4100.907
412.30.735
414.70.564
4170.427
419.40.326
421.80.253
424.20.244
426.70.313
429.10.409
431.60.54
4340.707
436.50.883
4390.993
4400.999
441.50.964
4440.817
446.60.638
449.10.484
451.70.368
454.30.344
456.90.452
459.50.597
462.10.773
464.70.936
467.41
470.10.919
472.80.751
475.50.577
478.20.437
480.90.333
483.70.258
486.40.204
489.20.165
4920.135
494.80.155
497.60.191
500.50.24
503.40.308
506.20.401
509.10.53
5120.695
5150.872
517.90.989
519.11
520.90.97
523.80.853
526.80.982
5280.999
529.90.98
532.90.848
535.90.669
5390.509
542.10.386
545.20.296
548.30.232
551.40.185
554.60.15
557.70.124
560.90.104
564.10.088
567.40.076
570.60.066
573.90.057
577.10.051
580.40.045
583.80.04
587.10.036
590.50.039
593.80.044
597.20.049
600.60.055
604.10.063
607.50.073
6110.085
614.50.099
6180.118
621.50.143
625.10.175
628.70.218
632.30.278
635.90.36
639.50.474
643.20.626
646.80.804
650.50.956
653.51
654.30.996
655.81
6580.979
661.80.847
665.50.668
669.40.768
673.20.932
6771
680.90.923
684.80.756
688.70.843
692.60.978
6951
696.60.984
700.60.858
704.60.679
708.60.517
712.70.392
716.70.301
720.80.291
7250.379
729.10.499
733.30.657
737.50.836
741.70.974
744.21
745.90.987
750.20.865
754.50.687
758.80.523
763.10.396
767.50.304
771.90.237
776.30.191
780.70.24
785.20.308
789.70.402
794.20.53
798.70.695
803.30.872
807.90.99
809.71
812.50.97
817.10.828
821.80.649
826.50.493
831.20.525
8360.69
840.80.867
845.60.988
847.51
850.40.973
855.30.834
860.20.655
865.10.522
8700.685
8750.863
8800.986

Source: Computed on this page with the game's own plate formula, 401 slider positions plus the 17 clean-figure peaks.

Sand and fine powder go opposite ways, as they do on a real plate

Sieve cocoa instead of salt and the picture turns inside out. The cocoa gathers where the sheet shakes hardest and leaves the still lines bare. Flour does the same, but white on white paper it hardly shows. Real powders do this. Félix Savart noticed it, and Michael Faraday explained it in 1831: very fine powder is swept along by the air the vibrating plate stirs up, while heavier grains bounce until they land where the plate is still.

The game doesn't compute that air. It sorts each material by grain size. Coarse grains (sand, salt, yeast, chia seeds) step down the shake map. Fine powders (flour, cocoa, icing sugar, baking soda, cornflour) step up it, more slowly. That's a written rule, and it's in the table at the bottom of this page. What isn't written anywhere is the figure each one draws.

The volume knob sets how thin the lines are. A grain only bounces where the sheet shakes harder than a set threshold, so the quiet strip along each still line is where grains come to rest. Turn it up and that strip narrows. But shake too hard and the grains on the most shaken parts are thrown a few cells in any direction, and the figure turns messy. Too quiet, and nothing lifts at all.

Sand piles its lines only as steep as sand can stand. Bouncing heaps grains up, the slope rule slides them down, and an automated test checks that after ten seconds almost no cell is steeper than sand can hold. Wet the sand first and it stays where it is.

Under the hood
Sand and fine powder go opposite ways, as they do on a real plate

Under the hood

Sand and fine powder go opposite ways, as they do on a real plate

Every material carries a grain size and its own angle of repose. Ten of them are loose grains: five coarse (sand, salt, yeast, chia and flax seeds) and five fine (flour, cocoa, icing sugar, baking soda, cornflour).

Why lines form: the size of the shaking has a V-shaped valley along each still line, so the shake map slopes into the line from both sides. Coarse grains go downhill and stop once the shaking under them drops below the bounce threshold. Louder playing pushes that point closer to the line, so the lines get thinner. With no slope at all, a coarse grain hops in a random direction and a fine one stays.

The direction comes from the slope, and each grain steps across and down at random in proportion to it, so a diagonal slope gives diagonal steps on a square grid. Everything a grain carries, such as its colour, travels with it.

Salt goes to the still lines, flour leaves them"Still line": shaking below 25 % of the peak. A direction, not an accuracy: there is no real-world reference value.

Before the speaker 42 % of the grains lie on still lines; after 8 s salt has 90 % there and sand 68 %, while baking soda has 22 % and flour 20 %.

Salt goes to the still lines, flour leaves them020406080100at firstsaltsandbaking sodaflourshare on still lines, %Salt goes to the still lines, flour leaves them020406080100at firstsaltsandbaking sodaflourshare on still lines, %
Data table
Salt goes to the still lines, flour leaves them
after 8 s
at first42
salt90
sand68
baking soda22
flour20

Source: The game itself, speaker held 8 s, recorded in the project's fact sheet.

Louder narrows the lines, until grains flyGrains rest where the line dips below the bounce threshold.

Along the middle row at the starting note, the shaking dips to zero at each still line; at the default volume it peaks at 1.32 "g", at full volume at 2, past the 1.5 at which grains fly wild.

Louder narrows the lines, until grains fly0.00.51.01.52.00.00.20.40.60.81.0across the middle of the sheet (0 left, 1 right)shaking, game "g"grains bounce above 0.12grains fly wild above 1.5Louder narrows the lines, until grains fly0.00.51.01.52.00.00.20.40.60.81.0across the middle of the sheet (0 left, 1 …shaking, game "g"grains bounce above 0.12grains fly wild above 1.5
  • volume 0.66 (default)
  • full volume
Data table
Louder narrows the lines, until grains fly
across the middle of the sheet (0 left, 1 right)volume 0.66 (default)full volume
00.3630.55
0.00630.3590.543
0.01250.3460.524
0.01870.3250.492
0.0250.2960.449
0.03130.260.394
0.03750.2170.329
0.04370.1680.255
0.050.1140.172
0.05630.0550.083
0.06250.0070.011
0.06880.0720.109
0.0750.1380.208
0.08130.2040.308
0.08750.2690.407
0.09380.3310.502
0.10.3910.592
0.10620.4460.676
0.11250.4960.752
0.11870.540.818
0.1250.5770.874
0.13130.6060.918
0.13750.6260.948
0.14370.6370.965
0.150.6380.967
0.15630.630.955
0.16250.6120.928
0.16880.5840.885
0.1750.5460.828
0.18120.4990.756
0.18750.4430.671
0.19380.3780.572
0.20.3050.462
0.20620.2260.342
0.21250.140.212
0.21880.0490.075
0.2250.0450.068
0.23130.1420.215
0.23750.2410.365
0.24370.3390.514
0.250.4370.662
0.25620.5320.806
0.26250.6230.943
0.26870.7091.074
0.2750.7881.194
0.28130.861.303
0.28750.9231.399
0.29380.9771.48
0.31.021.545
0.30631.0511.592
0.31251.0711.622
0.31871.0781.633
0.3251.0721.624
0.33121.0531.595
0.33751.0211.547
0.34380.9761.479
0.350.9191.392
0.35630.8491.287
0.36250.7691.165
0.36880.6771.026
0.3750.5760.873
0.38120.4670.707
0.38750.350.53
0.39370.2270.345
0.40.10.152
0.40630.030.046
0.41250.1620.245
0.41880.2940.445
0.4250.4230.642
0.43130.550.833
0.43750.6721.018
0.44370.7871.193
0.450.8951.356
0.45620.9931.505
0.46251.0811.638
0.46881.1571.753
0.4751.2211.85
0.48131.2721.927
0.48751.3081.982
0.49381.332.016
0.51.3382.027
0.50621.332.016
0.51251.3081.982
0.51881.2721.927
0.5251.2211.85
0.53131.1571.753
0.53751.0811.638
0.54370.9931.505
0.550.8951.356
0.55630.7871.193
0.56250.6721.018
0.56870.550.833
0.5750.4230.642
0.58130.2940.445
0.58750.1620.245
0.59380.030.046
0.60.10.152
0.60620.2270.345
0.61250.350.53
0.61880.4670.707
0.6250.5760.873
0.63120.6771.026
0.63750.7691.165
0.64380.8491.287
0.650.9191.392
0.65630.9761.479
0.66251.0211.547
0.66871.0531.595
0.6751.0721.624
0.68131.0781.633
0.68751.0711.622
0.69371.0511.592
0.71.021.545
0.70630.9771.48
0.71250.9231.399
0.71880.861.303
0.7250.7881.194
0.73120.7091.074
0.73750.6230.943
0.74380.5320.806
0.750.4370.662
0.75620.3390.514
0.76250.2410.365
0.76880.1420.215
0.7750.0450.068
0.78130.0490.075
0.78750.140.212
0.79370.2260.342
0.80.3050.462
0.80630.3780.572
0.81250.4430.671
0.81870.4990.756
0.8250.5460.828
0.83130.5840.885
0.83750.6120.928
0.84380.630.955
0.850.6380.967
0.85620.6370.965
0.86250.6260.948
0.86880.6060.918
0.8750.5770.874
0.88120.540.818
0.88750.4960.752
0.89380.4460.676
0.90.3910.592
0.90630.3310.502
0.91250.2690.407
0.91870.2040.308
0.9250.1380.208
0.93130.0720.109
0.93750.0070.011
0.94370.0550.083
0.950.1140.172
0.95630.1680.255
0.96250.2170.329
0.96880.260.394
0.9750.2960.449
0.98120.3250.492
0.98750.3460.524
0.99380.3590.543
10.3630.55

Source: The game's own shake map at the starting note (519 Hz), speaker in the middle, sampled along the middle row with the game's plate formula; thresholds from the game's grain rule.

A hammer blow or a passing toy car shakes the paper too, and sand, bubbles and ants feel it

The speaker isn't the only thing that shakes the paper. A hammer blow, a water drop landing, a toy car driving past, the table being bumped and the monster truck rumbling by all write to the same shake map. They make a local shake that fades with distance and has no figure in it. Only the speaker can set the whole sheet ringing in a pattern.

Everything lying on the sheet reads that one map, each thing at its own threshold. Marbles are shaken loose by almost anything. Glitter and confetti need a little more, the same as it takes to bounce a grain of salt. Puddles ripple, the bubbles on soapy suds pop, sand turns runny. Shaving foam is a stiff bed with no bubbles, and the shaking leaves it as it is. The shake ladder below puts them all on one scale, with what each thing that shakes the sheet sets off.

Creatures feel it too. Ants and snails can't hear anything in the game, but they feel the sheet. Hold the speaker near a line of ants and they scatter. Bump the table and a pigeon takes off, a snail tucks into its shell, ants scatter and sand jumps, all from one shake. Automated tests check each of these. Animals react to change, so a fan that keeps blowing startles a pigeon at most once.

Under the hood
A hammer blow or a passing toy car shakes the paper too, and sand, bubbles and ants feel it

Under the hood

A hammer blow or a passing toy car shakes the paper too, and sand, bubbles and ants feel it

A local shake has a place, a reach and a strength. It is strongest at the centre and fades to nothing at the edge of its reach. Repeated shakes in one place are peak-held, so they don't add up, and each one dies away within a fraction of a second. Any blow shakes the sheet in proportion to the root of its energy.

Things with friction or glue share one loosening rule: nothing until the shaking beats their grip, then a rattle plus a push toward the still lines.

Creatures read the sheet through a coarser layer of what they feel, which also fades. Each species has its own sensitivity: ants and snails feel the most, dogs the least.

Cost: the shake map is 3,072 values against 196,608 paper cells, it sleeps when still, and passes only visit the rows a shake can reach. A test pins it under 4 ms a tick for a full-size sheet covered in sand. This is plain JavaScript; no WebAssembly kernel reads the shake map.

The game's own rule The game's own constants. The scale is the game's own "g", tuned for play, not real acceleration. The speaker at its default volume reaches everything up to runny sand without throwing grains about; turn it up past 0.75 and it reaches the wild rung too.

The shake ladder
FromWhat happensWhat
0.03 "g"A heap steeper than its grains can stand slides downheaps
0.04 "g"Marbles and balls are shaken loose (toy cars from 0.05)rolling things
0.12 "g"Dry grains bounce; glitter and confetti are shaken loosegrains, bits
0.2 "g"Puddles show ripplespuddles
0.4 "g"Bubbles on soapy foam and resting soap bubbles start to popbubbles
0.6 "g"Sand turns runny: no slope holds itsand
1.5 "g"Grains are thrown about; puddles throw spraygrains, puddles
What each shaker reaches
ShakerShakes the sheetHighest rung it reaches
Speaker, volume 0.661.32 "g" on a clean figure, at its strongest spot0.6 "g": Sand turns runny: no slope holds it
Speaker, full volume2.00 "g" on a clean figure, at its strongest spot1.5 "g": Grains are thrown about; puddles throw spray
Water drop0.50 "g" where it lands, fading over 14 cells0.4 "g": Bubbles on soapy foam and resting soap bubbles start to pop
Toy car0.70 "g" at full speed, under it0.6 "g": Sand turns runny: no slope holds it
Hammer1.20 "g" at the blow, fading with distance0.6 "g": Sand turns runny: no slope holds it
Table bumped1.50 "g" everywhere at once, for 0.9 s1.5 "g": Grains are thrown about; puddles throw spray
Monster truck2.50 "g" beside it, fading over 400 cells1.5 "g": Grains are thrown about; puddles throw spray

Shakes the sheet

  • Speaker: a whole-sheet figure

  • Hammer and any blow: strong, local

  • Table bumped: everywhere at once, briefly

  • Monster truck: the strongest, fading over a long way

  • Toy car driving, water drops, hole punch: light, local

Shared

  • The shake map: how hard each spot of paper shakes

  • The feel layer: what creatures feel

Reads it

  • Sand, salt, seeds: to the still lines

  • Flour, cocoa, baking soda: to the shaking parts

  • Puddles: ripples; spray when shaken hard

  • Bubbles on soapy foam, and resting soap bubbles: pop when shaken hard enough

  • Glitter, confetti, stickers, marbles, toy cars: shaken loose past their grip

  • Ants, snails, flies, pigeons, cats, dogs: startle at a sudden change

No arrow runs from a shaker straight to a reader. An automated test enforces it for the speaker in both directions, and another checks the table bump's own code, which may not name cups, balls, bits, sticks or cars. One exception remains in the shaking: a blow also kicks googly eyes directly, besides shaking the sheet.

Rows of soapy suds with the speaker in the middle: the bubbles on top have popped round the speaker and in patches either side, and survive in clusters between and above and below
Speaker on soapy suds. Bubbles pop where the sheet shakes hard enough. They are gone round the speaker and either side, and survive in the quieter places where salt gathered in the figure above. A test pins this.
A line of ants on the paper breaking up and scattering away from the speaker
Speaker near a line of ants. Ants are deaf in the game, so they flee because they feel the sheet move, at full sensitivity. A test pins this.

Every sound effect is made by code at the moment it happens

There isn't a single recorded sound effect in the game. A splat is a burst of filtered noise with a falling tone under it. A bonk is a tone gliding down with a short click on top. Fire is noise through a filter, and it gets louder as more of the page burns. The flies' buzz wobbles and grows a little with every fly, up to a quiet ceiling, so a swarm hums instead of roaring. The toy car's motor rises in pitch with its speed. The only audio files are the six songs and the professor's spoken lines.

A sound is also something that happens in the world. When the game plays a bang at a place, the bang writes a patch of noise that creatures hear, as loud as that kind of sound is: a splash is loud, a clink much quieter, a pencil scribble hardly there. The game does this before it checks whether you've muted it, so with the sound off a cat nearby still hears the bang. A dog doesn't startle at its own bark, because each creature takes the noise it made itself off what it hears. The pigeon nearby does startle, by its own hearing, and no code tells it the dog barked. An automated test checks it.

The professor's mouth opens and shuts with the loudness of his voice as it plays. That's two pictures, mouth shut and mouth open, swapped on loudness and each held a moment so they don't flicker. It isn't lip-sync. While he talks, the music dips.

Under the hood
Every sound effect is made by code at the moment it happens

Under the hood

Every sound effect is made by code at the moment it happens

Everything is synthesised on Web Audio. Nearly every one-shot is built from two primitives:

  • A noise burst: a stretch of shared noise, started at a random point and played at a slightly random speed so no two sound alike, through one filter that can glide, with a fast attack and an exponential fade.

  • A tone: one oscillator with an optional pitch glide and the same kind of fade.

50 named one-shots, each rate-limited so dragging doesn't machine-gun them, and 7 loops driven by a level from the simulation: fire, fly buzz, blower, car motor, speaker hum, electric zap, cat purr. Pausing the audio silences every loop at once without restarting any.

The simulation runs in a Web Worker, which has no audio. There a silent stand-in takes every sound call, so creatures still hear it, and notes it down; the list travels with each frame and the main thread plays it.

A sound with a place writes noise that creatures hear. Its loudness depends on the kind of sound and its size, up to a cap; louder sounds reach farther, and the noise dies away within a fraction of a second. The mouth follows the loudness of the voice as it plays.

Try it: four loops, driven by the game The game's own rule

The game's own recipe. Each sound is rebuilt here from the game's recipe, with the same oscillators, filters and numbers, and drawn as its waveform: a whole second and a half on top, a 30-millisecond close-up underneath. Pick a sound and move the slider to the quantity the game feeds it. Tap the waveform to hear three seconds. What to look for: fire is all noise and thickens with the burning; the car and the speaker are regular waves whose spacing shrinks as the pitch rises.

Waveform of a game sound loop, rebuilt from the game's recipe
  • 0 s
  • 1.5 s
  • close-up: 30 ms
  • ▼ close-up
  • Working it out…
  • This browser can't render audio here.
Sound
0.4
fed by
—
of
—
recipe
—
gain
—
pitch
—
crackle ticks
—
drawn at
—
sound
—
How loud each sound is to the creatures on the tableA played sound with a place writes noise that creatures hear, louder and wider for bigger sounds.

A splash is the loudest at 1.2; a bonk and a bark are 1, a flap 0.8, a clink 0.5, a pop 0.4, a rattle 0.2 and a scribble 0.05.

How loud each sound is to the creatures on the table0.000.200.400.600.801.001.20splashbonkbarkflapclinkpoprattlescribbleloudness at size 1How loud each sound is to the creatures on the table0.000.501.00splashbonkbarkflapclinkpoprattlescribbleloudness at size 1
Data table
How loud each sound is to the creatures on the table
loudness
splash1.20
bonk1.00
bark1.00
flap0.80
clink0.50
pop0.40
rattle0.20
scribble0.05

Source: The game's own loudness table. Tuned for play, not measured.

Heavier things knock lower, and a higher drop knocks louder

In the released game, a falling apple plays the same knock whatever height it falls from. In the next update, what you hear when something lands is worked out from the landing itself. How loud goes by the square root of its energy, so the same apple dropped from higher is louder. The pitch goes by the cube root of its weight, so a tin of paint thuds low and a strawberry ticks high. And a soft landing is duller: drop the apple into a deep blob of paint and it's quieter and lower. Automated tests check all three.

Try it: drop something on the paper The game's own rule

next update Pick a thing, set how high it falls from and how much runny paint it lands in. The game's formula picks how loud and at what pitch, and the game's voice recipe for that thing plays it: tap the waveform to hear it. What to look for: the apple from higher is taller in the waveform (every knock is drawn at one fixed scale) but its tone doesn't change; paint underneath makes it shorter and lower. (In the game a deep cushion can also swap the voice for a pat or a splat; this demo keeps each thing's own voice.)

Waveform of the knock a falling thing makes, worked out from its weight, height and landing
  • 0 s
  • 0.5 s
  • close-up: 60 ms
  • ▼ close-up
  • Working it out…
  • This browser can't render audio here.
Drop
70 cells
0.0 cells
lands at
—
voice
—
loud
—
size
—
pitch
—
sound
—
Under the hood: the knock formula
The knock formula

Under the hood

The knock formula

next update The landing's energy comes from the thing's mass and the speed it lands at, E = ½·m·v², and a hammer blow is the reference everything else is measured against. How hard the landing is depends on the thing's own face and on how deep the soft stuff under it is.

Loudness grows with the square root of the energy, up to a cap, and a soft landing takes some of it off. Pitch goes with the cube root of one over the mass, kept within a range, and a soft landing lowers it a little. Every frequency in the thing's voice is scaled by that pitch, and very quiet knocks lose their click too.

The cube root is reasoned, not fitted: for the same shape, ring frequency goes as 1/size, and size as mass⅓. A few of the smallest things share the top of the range.

The salt figures come from 224 shake patterns, and not one sound in the game is a recording

224
Vibration shapes the sheet can mix, with up to fourteen still lines each way.
17
Places on the note slider where a speaker in the middle of a 4:3 page gives a clean figure, calculated from the game's plate formula.
1/64
The shake map's size against the paper's: 3,072 values steer 196,608 cells.
10
Grains and powders that read the shake map, five coarse and five fine, sorted by one rule with its sign flipped.
0
Recorded sound effects. 50 one-shots and 7 loops are built by code as they play.
4 ms
Budget for one tick of a full-size sheet covered in sand under the speaker, pinned by a test.

Connected to the rest of the page

Shared: shake map → what creatures feel

Hold the speaker near a line of ants

They scatter. Ants are deaf in the game; they feel the sheet. Pinned by a test.

Animals with their own minds

Shared: shake map + a jolt

Bump the table

A pigeon takes off, a snail tucks in, ants scatter and sand jumps, from one shake. Pinned by tests.

Animals with their own minds

Shared: shake map, hard enough

Hold the speaker on soapy suds

Bubbles pop where it shakes and survive where it is still. Pinned by a test.

Runny, sticky and oily

Shared: wetness

Wet the sand, then play the speaker

The sand stays where it is: wet grains stick together. Pinned by a test.

Water in paper

Shared: a local shake

Hammer beside a heap of sand

The heap slumps to its own angle. Pinned by a test.

Things that push things

Shared: sound → noise creatures hear

Mute the game, then make a bang

The pet still hears it: the bang happened in the game's world.

Animals with their own minds

Figures appear the moment the note plays, and a few classic shapes are left out, to fit a tablet

A real Chladni plate is a hard problem in mechanics. A child with a tablet needs the figure to appear, change with the note and the press point, and sort sand from flour, within a few milliseconds a frame. Here is what we keep and what we let go.

In real life. A plate's shapes come from a fourth-order equation, with free edges that don't give simple products

In the game. Products of simple cosine stripes with a plate's frequency law: a hybrid of membrane shapes and plate frequencies

Why that's enough. Closed form, no solver, and resonance, press point and more lines higher up all follow. The classic low figures, such as the X on a square plate, are lost: low notes give straight stripes

In real life. Waves build up, travel and ring down

In the game. A steady-state map, on and off instantly

Why that's enough. A sheet rings at hundreds of hertz, far beyond 60 steps a second. No build-up or ring-down, and something heavy lying on the sheet doesn't change the figure

In real life. Acceleration rises with frequency, and a grain needs more than 1 g to leave a plate

In the game. Strength doesn't depend on the note; a game "g" with its own bounce threshold

Why that's enough. Volume stays a simple dial and every clean note is equally rewarding. The units are the game's, not real g

In real life. Grains fly ballistically and land one by one

In the game. A height field: a share of each cell moves one cell per turn, toward lower shaking

Why that's enough. Works at full-page scale on a tablet. Grains don't fly, and speed depends on how hard it shakes, not how steep the slope is

In real life. Fine powder rides air currents above the plate

In the game. The same rule with its sign flipped, at a slower rate; the air isn't read

Why that's enough. The right result for almost nothing. A fan blowing over the sheet doesn't change a flour figure through this rule

In real life. Water on a shaken plate forms Faraday waves, at half the driving rate

In the game. A drawn, slowly moving ripple pattern, as a height offset for the renderer only

Why that's enough. You see a grid of ripples where it shakes. The water itself doesn't move, except that very hard shaking throws spray

In real life. Sound is a pressure wave that travels, echoes and is blocked

In the game. One patch of noise that fades with distance and dies away in a fraction of a second; the speakers play mono

Why that's enough. Creatures need "how loud, and from where". No echoes, no sound shadows, and a bang on the left isn't heard on the left

In real life. The note a speaker plays is the note the plate rings at

In the game. Two scales on one slider: a pleasant range of notes for the ear, a wider range for the model sheet

Why that's enough. Pleasant notes and a full range of figures. Pitch and line count rise together, but the note isn't the sheet's frequency