Thermal camera hub

Where it stands (Tue 20 Oct)

• Live since Fri 16 Oct. The P3 hangs on helping hands behind aifoundry2's card and streams to the live page, which anyone with the link can open. At 11:55 on Tue 20 Oct it had run about 90 hours without a break: 25 fps, picture healthy, app 1.5.4. App 1.5.5, which counts mobile data, is built and waits for a deploy.
• Next to build: a mount. The helping hands drift: a 3° tilt across the view costs 2 mm of focus. Two designs exist on paper, the adjustable mount (Mon 19 Oct) and the fake card (Sun 18 Oct). The sanity check says: measure on the next visit, then print the simplest rigid frame at the measured spot.
• Waiting on a yes from Roman Shaposhnik or Nate Padgett: slot 2 for a few weeks, a black vinyl patch and a contact sensor on the card, card time for the first sweep, and the card to strip.
• What's proven: the chip's footprint shows through the board in a single frame (Fri 9 Oct). Over an M5's keyboard the P3 resolved about 4 mK per frame on a patch and 3.1 mK per pixel with lock-in, and it filmed heat crossing the keys (Thu 15 Oct).
• Not done yet on the ET card: a measurement. There's been no lock-in sweep, no calibration patch and no reference thermometer.
• What the back of the board can show: halves and quadrants of the die, live, at hundreds of millikelvin; not single shires, which are the on-die sensors' job (Physics).

Open questions

• May aifoundry2's empty PCIEX16 (slot 2) hold a camera mount for a few weeks? It's Roman's or Nate's call, and both mount designs depend on it.
• Which card did Roman mean to strip, and on what terms?
• Is aifoundry3's card back as reachable as aifoundry2's?
• Which board revision is in the rigs? It decides where the voltage regulators sit. Friday's photos point to the low-profile production board, about 69 mm tall under a taller heatsink (Friday thermal measurements). One reading of the board's height at its tail settles it.
• Does the P3 really focus down to 8 mm? Nobody has measured it; the manual doesn't say.
• The camera's fixed pattern: a uniform target reads 0.6 °C uneven across the frame (Gough Lui), about 17 times the frame noise. Single frames can't be trusted for small absolute differences; lock-in and difference images remove it.

What you have: in the lab since Fri 16 Oct, the P3, the Realme 14T phone on a USB-C hub with Ethernet and charging, and a helping-hands stand. At home, 1-mil Kapton film (Fri 2 Oct), a SmallRig super-clamp pair and two USB-C adapters (Tue 6 Oct). Gmail, checked Fri 9 Oct, had no order for an arm, a cradle, a contact thermometer, a first-surface mirror, a Raspberry Pi or a riser.

Roman's offers: on Tue 29 Sep in #community-lab, "we can even sacrifice one of the cards and tear off the heatsink"; on Fri 2 Oct, in person, a card to take apart, and the advice to look at the back of the board first. No card or date was named, and nobody followed up. He made #thermals for this work on Tue 6 Oct.

Holding the camera still

Today the P3 hangs on helping hands behind aifoundry2's card. They hold it, but they drift: a 3° tilt across the 40 mm view costs 2 mm of focus. Two designs replace them. Neither is built, and both need a yes for slot 2 from Roman Shaposhnik or Nate Padgett.

The sanity check: measure first, then print the simplest rigid frame at the measured spot.

The adjustable mount (Mon 19 Oct)

Designed Mon 19 Oct 2026 (week 43), 17:02–17:21, from the fake card plan below. Not printed yet. The guide (public) · tailnet · the print and CAD package, a download on the guide (tailnet copy): nine STL files, STEP models, the editable make_mount.py and a drilling template.

A stock steel slot cover in slot 2 carries a printed boom over the gap, and the camera hangs from it in a drop-in cradle. Three slides set where the lens sits: along the card (76–96 mm in from the rear wall), height (37–48 mm above the board) and distance from the card's back (54–62 mm, starting at 60). Two keepers hold the camera's housing on soft pads, so it lifts out for focusing and goes back. Nothing touches the card or the board, and nothing goes into the PCIe connector.

Sanity check: is it a good design?

Checked Tue 20 Oct 2026 (week 43), on your ask, against the package's own dimensions and checks (parameters.json, validation.json, hardware_validation.json) and the numbers on this hub.

A careful first prototype, with more adjustment than this job needs and more joints than a camera that must hold half a pixel should sit on. Print the fit coupon and the cradle now. Measure on the next visit. Then print the frame set to the measured spot, with no slides along the card or up and down.

What it gets right:

• Nothing touches the card or the board, nothing goes into the connector, and nothing is glued to the camera. The housing carries the camera, not the USB plug.
• ASA, a 50 °C stop for the air round the camera (it's rated to 55 °C), and 5 mm kept from the cooler's fins.
• It says what isn't known yet (where the lens sits on the camera, where the chip is, whether the rail's lower tab catches, the cable) and starts with a fit coupon. Its CAD checks found no collision at 27 slide settings, with all 12 screw stacks modelled.

What I'd change:

Gravity isn't the problem: the printed parts weigh about 80 g, and two M3 screws hold that by friction many times over. What matters is whether the view stays put.

The order:

1. Next visit, ten minutes with a ruler and calipers: the measurements below.
2. On the bench: where the lens sits on the camera's face and how far it stands out, with calipers.
3. Print now: the fit coupon (part 07), then the cradle and its keepers (parts 04–06). They carry over to whichever frame stays.
4. Then the frame: regenerate it with the measured numbers and print it as designed with the slides locked once, or better, with the boom, plates and shelf merged into one rigid piece (a change to make_mount.py). Ask Roman Shaposhnik or Nate Padgett for slot 2 first.
5. In software: line the frames up on fixed board features, and log the camera's own temperature.

What to measure on the next visit

Both designs need the same ten minutes with a ruler and calipers, with aifoundry2 powered down:

1. The chip's centre on the card's back: how far in from the rear wall and how high above the motherboard (estimated at 86 and 42 mm).
2. Slot 2's centreline to the card's back (expect about 60 mm) and to the nearest cooler fin (expect about 29 mm, 89 less 60).
3. Slot 2 in the rear rail: is it open, does its screw take a 6-32 or an M3, does a stock cover's lower tab catch, and is slot 1 or 3 free beside it?
4. The space above the gap, up to 90 mm over the motherboard: cables, the card's power lead, the cooler's fan cable.
5. The M.2 adapter: its height, and whether risers are about to go into it. Its ports stand about a centimetre high.
6. On the bench, the P3's lens: where it sits on the camera's face and how far it stands out, so the cradle centres the lens, not the body.

The fake card (Sun 18 Oct)

Your Sat 17 Oct idea: hold the P3 on a fake card in the empty slot beside the card, instead of on the helping hands. Designed Sun 18 Oct; put on this hub Mon 19 Oct 2026 (week 43). Not built yet. The first step is a yes from Roman Shaposhnik or Nate Padgett for the slot.

A printed dummy card in aifoundry2's empty PCIEX16 holds the P3 60 mm from the card's back. The slot is three positions (61 mm) from the card, about the distance the focus simulator picked. The camera rides in a cradle that slides through a window in the plate, so the lens can sit anywhere from 40 to 62 mm from the board. It stays rigid and square to the card by construction, comes out and goes back to the same place, and never touches the card or the connector. About $20–25 in parts, plus the print.

Where the camera sits on it

Expansion slots are 20.32 mm apart (the ATX standard), so slot 2's centreline is 61 mm from the card's, and 60.2 mm from its back. Where the P3 sits on the fake card decides the rest:

60 mm is the setting. The die (26 × 22 mm) only fits whole from 46 mm, and at 60 mm there's still 12 mm of air behind the camera. The distances are to the lens's front; its optical centre sits a few millimetres behind, which shifts every row the same way.

What to build

• The plate: ASA, 3 mm, low profile (no taller than 69 mm), about 125 mm long from the bracket, with a lattice everywhere except round the window.
• The window: 61 × 29 mm, for the P3's 59 × 27 × 17.2 mm body. Centre it about 86 mm in from the rear wall and 42 mm above the motherboard, the 9 Oct estimates for the chip's centre. Give it ±10 mm of slots both ways until those are measured.
• The cradle: start from the back piece of the P3 cover system on MakerWorld, which is meant to be adapted. Two rails run through the window, and two M3 nylon thumbscrews lock it anywhere from 40 to 62 mm. Add a split collar or a Kapton strip to lock the focus ring, and a cable clip.
• The bracket: a stock steel full-height slot cover, drilled for two M3 nylon screws into the plate's flange, so slot 2's own screw holds everything. Line the plate up over the connector's centreline.

Rules:

• Nothing with copper fingers goes into the connector. The fake card isn't seated at all: it hangs from slot 2's screw like any card's bracket, its lower edge a few millimetres above the connector.
• No glue on the card, the board or the camera. The P3 sits in a clamp-on cradle, so it comes out for focusing and calibration. If anything is glued, it's the cradle to the fake card.
• Keep the camera cool. The P3 is rated to 55 °C, and the card's back read 41–47 °C on the live stream, more under load. Print in ASA: PETG sags near 62–68 °C, and PLA is out. Keep the plate open, put one flat baffle between the camera and the fins, and watch the camera's own temperature.
• The cable carries the strain. The USB-C plug is the camera's only structural point, so clamp the cable within 30 mm of it, run it along the plate's top edge and out over the rear wall.
• Airflow stays as it is. Air through the cooler runs front to back, parallel to the plate, and the card's own fans are on its other face.

Two fallbacks: a PCI-slot fan bracket in slot 2 (Akasa AK-MX304-08BK, €7.90) with the P3 on a plate where a fan goes, rigid but a stopgap; or a slot cover with the P3 stuck flat on VHB, which reaches only 40 mm, too close for the whole die, and VHB creeps in warm air.

Prices read Sun 18 Oct 2026.

Steps, in order

1. Ask Roman Shaposhnik or Nate Padgett for slot 2 for a few weeks, and a time to power aifoundry2 down: the fake card takes the board's only x16 slot.
2. Spend ten minutes with the measure on the next visit (the list above).
3. Model it from the MakerWorld cover and print it in ASA.
4. Power down, screw the fake card into slot 2, set the cradle at 60 mm and route the cable out.
5. Boot and let it run warm for 20 minutes. Focus on the capacitors' cold dots with the live focus meter (Focusing the P3), lock the ring and add a witness mark.
6. Tap the frame: if the picture jumps, tighten. Watch the camera's own temperature through one loaded run.

What to measure: the list above, shared by both designs. The full plan, with its sources: its own page (public) · tailnet.

Every page, by kind

(public): anyone with the link, unlisted, like this hub. (private): only you, and why. The Log has them by date.

Live

• Thermal Master P3 live stream (public): the camera behind aifoundry2's card, its own delay, and a ● Record box for CSV.
• Lab history (public): hour, day and week graphs of the three lab machines, the cards' die temperatures among them.

Designs

• Adjustable camera mount (public, with the print and CAD package; tailnet), Mon 19 Oct, and its sanity check.
• A fake card in the open slot (public; tailnet), Sun 18 Oct; its design on this hub.
• Cooling from the back, camera on top (public), Fri 9 Oct: the later route to the front of the chip.
• Imaging the chip: voltage, riser, mount (public), Tue 6 Oct.

Tools and guides

• The focus simulator (public; tailnet): the picture, blur and field of view at any distance.
• Focusing the P3 (public): a live sharpness score and a witness mark.
• Calibrating the P3 (public; tailnet): a contact sensor, a vinyl patch, foil and a flat field.
• Taking data off the P3 (public): watch, record CSV, or pull raw frames on the lab LAN.
• Remote camera setup (public): running the phone in the lab with no visits.

Measurements

• Friday thermal measurements (public): the gap, the card and the slots, to scale.
• Seeing millikelvin heat with the P3 (public): the camera's real sensitivity, over an M5.
• Watching heat flow through the M5 keyboard (public): movies and arrival-time maps.
• Heat spreading on the ET-SoC-1 (public): the on-die sensors, block by block.
• aifoundry2's card gets new fans (public): the cooling the card runs on now.
• aifoundry2's fans at full speed (public), its idle runaway to 138 °C (public) and the card fan that failed (public).
• The lab, 2 October and 16 October (public): the first runaway, and the day the camera went live.

Plans and research

• Pointing a thermal camera at the ET-SoC-1 (public): the 26 Sep plan, 20 experiments.
• Running without the heatsink (public).
• The bit-flip follow-up plan (private: it quotes messages and vault material). Its project now has its own page, the Horace Hub (tailnet; private).
• Every way to measure power and temperature on the card (public).
• Thermal-model files (public): what a spatial model of the chip needs, and what's public.
• Heat per millimetre (public), where the work sits and the effect of overheating (public).
• Spatial temperature: a brief and a compiler that knows where the fan is (public); temperature leaks through the abstractions (public).

The camera itself

• Reverse-engineered Thermal Master P3 (public): the full spec of your unit.
• The P3 on chips: what to read (public), what you ordered and why (public) and order the correct thermal camera (public), the brief it reversed.
• Optics, streaming and mount: research notes (public) and the physics notes (public), Fri 9 Oct: this hub's first sources.
• What happened on 15 October (public): the phone app's biggest day.
• A contact sensor, and the rig without your accounts (private: it lists the rig's accounts and keys), Fri 16 Oct.
• The ET-SoC-1, interactively (public): the die, its shires and where the heat comes from.

Code and your notes

• thermal-android (private GitHub): the phone app; start any change from its AGENTS.md.
• et-soc1-prototyping (public GitHub): the lab repo.
• Project: Thermal Camera PCB and Project: AI Foundry: your Google Docs.
• The AI Foundry hub (spacesheep, private): the lab, the boxes, and the way in through the camera's phone.

Can the P3 see it? The camera is fine; the board sets the limit

Short answer: yes for halves and quadrants of the die, live; no for single shires from the back, with any camera. The P3 resolves far finer detail than the board lets through. What blurs the picture is the copper in the package and the board, which spreads heat sideways about four times faster than it carries it through.

What the P3 covers from inside the gap (256 × 192 pixels, 40° × 30°):

From the gap, at about 65 mm, the P3 frames the die's shadow and most of the 45 mm package, not the whole card; the voltage regulators to the side fall outside. Closer than 45 mm buys nothing (the board's blur is coarser) and the depth in focus shrinks to a millimetre. Whole-card views need a first-surface mirror (bare polished aluminium or gold, never a protected coating, which loses about 30% at 45° in this band), or a longer gap.

What reaches the board's back (a model of the stack from die to board, built for this page and checked three ways; square-wave work, heatsink on):

The camera's noise is 35 mK in one frame. Lock-in (averaging against the known on/off pattern) brings it to 1.3 mK per pixel after a minute and 0.6 mK after five, and to hundredths of a millikelvin over a 1 cm patch.

What follows from the two tables: - Quadrants show live, no averaging needed. A quadrant switched every 20 s swings the back by about 300 mK, nine times the single-frame noise. The 26 Sep plan predicted 21–65 mK for a different pattern (2 s bursts every 24 s). Take that as the floor; the model's central case is a few times higher, and the first run settles it. - A better camera wouldn't help. The P3 out-resolves the board's blur 50–100 times over. A 640 × 480 sensor, a macro lens or a cooled camera buys nothing here. - One shire's heat does arrive, but as a blob 3–5 cm across. If only one known shire is working, a fitted model can still say which one: telling a shire from its neighbour 3.7 mm away has a signal-to-noise ratio around 375 after five minutes at 0.1 Hz. A full map of all 32 shires at once is out of reach: only about 21 of the 32 come apart cleanly. - The heatsink doesn't matter for this. 15–36% of the chip's heat goes into the board, but the fluctuating part on the back is local and changes by about 2% with the heatsink on or off. - The power stages will outshine the chip. A watt in the card's voltage regulators reaches the back about ten times more efficiently than a watt in the die, and the Fri 9 Oct 12:16 image already shows two such hot spots. The control: move the work to the opposite quadrant, and the chip's peak must move about 11 mm. If it doesn't, it's the regulators. - Single shires belong to the on-die sensors. They sit one per shire, at exactly the 3.7 mm pitch the board can't deliver. Even their whole-degree readings can be averaged against a known on/off pattern, as the 26 Sep plan proposed: if the die's slow drift moves them across degree steps, a 0.5 °C swing on one shire stands out within a minute. Roman's raw 12-bit codes (0.061 °C steps) would make that certain.

Coatings, and what they cost. The solder mask is already fine (emissivity about 0.9); the shiny parts are the problem, as the Fri 9 Oct cold dots show. Any coating also slows heat a little:

Polyimide film this thin is partly transparent to the camera: Sheldahl's measured emittance is 0.64 for 1 mil on its own and 0.62 for 1 mil over aluminium (corrected Wed 14 Oct). The timing of the signal (its phase) needs no emissivity at all. Absolute temperatures need a contact thermometer beside the camera's view, which the 26 Sep plan made a must-have; the calibration page has the sensor and the steps.

What "individual" can mean

Next steps

In order. Everything below keeps the heatsink on, so none of it risks a card.

1. ~~Try the browser viewer at home.~~ Done another way: since Fri 16 Oct the phone app streams the P3 to the live page, and since Sat 17 Oct the page records what you see to CSV. The Thermal Master viewer (code, WebUSB) stays a way to drive the camera from a laptop's Chrome. In the Thermal Master app, keep X³ off: it invents detail.
2. ~~Get the camera's own files to us.~~ Done another way: Taking data off the P3 (public) covers watching, saving a CSV, and pulling raw frames on the lab LAN.
3. Mount it. The Mount tab: measure on the next visit, print the fit coupon and the cradle, then a rigid frame at the measured spot. Hold 60–65 mm from the board's back, square to it; keep the camera cool and the cable clamped near its plug. - The alternative: a mirror in the gap. A 50 mm first-surface mirror at 45° ($25–40, bare aluminium or gold) lets the camera sit above the rig at 15–20 cm of path. It then sees the whole package and the regulators, has centimetres of depth in focus, and stays out of the warm air. Nothing is lost, since the board's blur is coarser than the pixels at that range too.
4. A coating and a reference. A square of black electrical vinyl (0.96) on the solder mask in view, or a thin coat of matte black paint, which costs less signal. The 1-mil Kapton is too thin for this (0.62 over metal). Add a contact sensor beside the patch: a TMP117 on USB (±0.2 °C) or an Inkbird probe over Bluetooth (±0.5 °C). Thermocouples are ±1.1 °C at best. Steps, parts and code are on the calibration page. Both need Roman's OK: it's the lab's card.
5. Raw frames for measurement. Pull them from the phone on the lab LAN (the data guide). If that path can't keep up, record on Linux instead, on a Raspberry Pi 5 by the rack or the lab host: - thermal-camera-viewer, a .deb that turns the P3 into an ordinary webcam (/dev/video10) any streamer can serve over the tailnet; - the Python unofficial P3 SDK, which gives radiometric frames; - Josh Dillon's p3-ir-camera, which has lock-in built in.

Five independent drivers exist, so the protocol doesn't rest on one person. On a shared lab box, the webcam route needs a kernel module rebuilt after every kernel update. The frames are 5 MB/s, 8% of USB 2.0. 6. The first measurement: a 45-minute sweep. - Load: one quadrant (8 shires, about 6.8 W of random multiply-adds), switched on and off at 50% duty. - Rates: 0.47, 0.19, 0.093, 0.047 and 0.023 Hz, 5 minutes each, plus 5 minutes idle. These miss the shutter's 90 s rhythm, of which 0.1, 0.2 and 0.5 Hz are exact harmonics. - Predicted swing on the back: about 1.3 K at the slowest rate, falling to 6.5 mK at the fastest. The lag behind the load grows with the square root of the rate, which measures the package's conductance with no emissivity needed. - Pin the clock first. aifoundry2 now idles at 51 °C and runs 63–75 °C loaded, across the governor's 65 °C threshold, so its clock would hunt and the sweep would measure the governor. aifoundry3's governor is latched at 600 MHz, and its card 0 runs Roman's per-sensor firmware. So check whether its back is as reachable as aifoundry2's (same board, same slot) and run there if it is. - Then the control: the same sweep on the opposite quadrant. The peak must move about 11 mm; if it doesn't, the camera is seeing the voltage regulators. - Demodulate afterwards against the frame times and the logged switch times. No hardware sync is needed. 7. The on-die check, no camera. Rerun the 7 Oct single-shire cycles for 5–10 minutes each on aifoundry3's card 0, and average the whole-degree readings against the on/off pattern. If it works, it gives single shires today, before the raw-code firmware. 8. Bring the 26 Sep plan up to date. Today's cooling (0.68 °C/W, 51 °C idle) makes its 84 °C pre-heat gate unreachable, and its power budgets were set for the old fan. Its sources (plan.md, lockin.py) live only on the Intel and in its space: move them into the lab repo (a lab-door job). 9. Later, the front of the chip: cooling from the back, camera on top. A powered cable stands the card on a bench, a cooler on its back carries the heat, the heatsink comes off, and the P3 looks down at the coated lid. Through the lid, the blur floor is the copper's, about one shire at 5 Hz: the only route to shire-scale pictures. Roman's offer of a card to strip stands, but no card was named; ask which one, the loose green board with a copper block on the top shelf or aifoundry1's card 0. - First, the safe test: fit the rear cooler with the heatsink still on, and read the idle drop. - With today's fans, expect 3 °C to 6 °C: about 6 °C means heat flows well into the board and 600 MHz with work on it is in reach; 3 °C or less means the lowest clock only. - The 6 Oct page's 15–20 °C bar no longer applies: it was set for the old, weaker cooling. 10. Write to Josh Dillon, author of the first open P3 driver with lock-in, if he is the Josh Dillon you know (not confirmed).

Who has to say yes: Roman Shaposhnik or Nate Padgett, for anything clamped or taped on the lab's cards or frames, software and a udev rule on a lab host, the card time for the sweep, and the card to strip.

For people to watch live, like a webcam: done since Fri 16 Oct. The live page streams the camera to anyone with the link.

First light, Fri 9 Oct

Your note at 15:33: "I placed the board backwards for easier fan maintenance".

The slot for a mount, Fri 16 Oct

About this hub

AI Foundry. Started Fri 9 Oct 2026 (week 41), 16:29–18:00 PDT, on the Intel (~/git0/aifoundry), on your ask for one place for the thermal-camera work: the goal, everything done since Tue 22 Sep, the first images, whether the Thermal Master P3 can see small heat changes through the board, and the next steps. Built from your posts in the AIFoundry.org Discord, your photos and notes from the lab, every earlier page and report on this, the lab repo, your Gmail orders and new web research. Updated through Mon 19 Oct (the focus simulator, the calibration page, the log, the fake card's design). Rebuilt as tabs Tue 20 Oct 2026 (week 43), with the Log first, on your ask: "the most important tab will be "Log", which gives high-level entries of things that are added … As a user, I'll be clicking on the log a lot." Say "refresh the thermal camera hub" to update it; every update adds an entry at the top of the Log.

The goal

In your Project doc: "thermal modeling of heat dependent on transistors (thermal camera)", the second of three goals.

In your Discord posts: - Sun 20 Sep: "a physics-aware compiler that can arrange computation with knowledge of heat propagation". - Tue 22 Sep: "Need per-tile sensor reading to get spatial heat flow"; then run the card "underclocked without a heatsink… mount a cheap thermal camera… stream… live". - Wed 23 Sep: "there's a heatsink, so maybe onto the opposite side of the PCB?" - Wed 7 Oct: "Camera has to be pointed at the back of PCB", and the camera as "physical verification" of the sensors.

Total life satisfaction: the thermal model is one of the three goals at the top of your Project doc. This rebuild makes the page open on what changed, which is what you come to it for. It doesn't move the measurement itself, which still waits on a mount, a patch and an afternoon of lock-in, and two of those three wait on a yes from the lab.