The Cell Next Door: Reading Errors, Statistical Missteps, and Immunotherapy Links

The 38.7% truth: Misreading adjacent cells in trial tables

[0:00]This is Humanitarians. Here's a sentence from a science video about cancer immunotherapy. Pembrolizumab produces 5-year survival above 50% in metastatic melanoma. It's a good sentence, specific, it names the drug, it names the disease, it has a number in it. Sounds like someone checked. Someone did check. The same

Correlation vs variance: What 0.74 leaves unexplained

[0:16]Project ran a fact check on that sentence, caught it, and wrote down a correction. Not 50%, about 43. The real number is 38.7%, so the claim was wrong, and the correction was also wrong. Both of them are sitting in the same table, in the

Hot, cold, and excluded tumors: Mapping the immune chain

[0:30]Same paper, one cell away from the truth. Hi, this is Tanmay Kulkarni in for Humanitarians AI. That mistake is the subject of this video, and the reason is that it doesn't stay that size. It's also, as it happens, the reason cancer immunotherapy works spectacularly in some tumors and not at all in others. Let's start with the table.

Two broken links, one identical failure: Sting vs CXCR4

[0:50]Keynote 006, 834 patients with advanced melanoma randomized to pembrolizumab or ipilimumab. 5-year survival everyone in the trial 38.7%, so above 50 isn't this row. The fact check's 43 is real, 43.2%, but it belongs to the subgroup who had no previous treatment, not everyone, one column across. And now the thing that

Category errors and asking the right questions in medical research

[1:10]Makes this trial almost unfair. 38.7% alive at 5 years for everybody. 38.7 months of median survival for first-liners, same digits. One is a proportion, the other is a length of time, and they sit a few lines apart on the same page. Nobody made anything up. People read a table, and this table has cells that look like each other. Here's what should bother you. In the same trial, the comparison arm ipilimumab, the previous standard of care, had a median overall survival of 15.9 months, and they kept following these patients. The 10-year results came out in 2024. 34% of the pembrolizumab group were still alive a decade later against 23.6% on ipilimumab. That is an extraordinary result. One of the genuinely great results in modern oncology. The over-claim bought nothing. There was nothing wrong with the truth. Now watch the same move happen to a statistic the whole field quotes. There's a famous one-page letter in the New England Journal Yarchoan and colleagues, 2017, that plots response rate against mutation count across

27 cancer types. The correlation is 0.74, and that letter is cited constantly as mutation count predicts who responds. 0.74 squared is 0.55. Mutation count explains 55% of the difference between cancer types, which means it does not explain 45%. Same table, cell next door. So, we went and checked where that 45% lives. Their own formula predicts pancreatic cancers failure to within half a percentage point. It handles mismatch repair deficient colorectal well. Where it breaks is microsatellite stable colorectal. The formula says about 9% should respond, and in the trial zero of 18 patients did. Not low. None. And the cleanest one is inside a single cancer. In Merkel cell carcinoma, the tumors driven by a virus carry far fewer mutations than the ones driven by sun damage. The low mutation ones responded better. 16 patients and nine, so that's an illustration and not a result, but it points the right way. Once more bigger, a randomized trial high-dose radiation to one tumor site than immunotherapy. The idea being that the wreckage acts like

A vaccine and warms the tumor up. 76 patients with lung cancer. Response rate went from 18% to 36. A doubling, that is how it gets described. The trial set two bars in advance. It needed a response rate of 50% and a P value under 0.1. It came in at 36% with a P value of 0.07. So, it cleared the statistical bar and missed the response bar. The paper's own conclusion, the criteria for clinical benefit were not met, and a doubling is the cell next door to that. Though, I'll come back to this one because it has a complication I don't get to skip. So far every mistake has been arithmetic, a number read next to the right number. Now the interesting one, which isn't about numbers at all, immunotherapy doesn't attack cancer. It takes a break off a T cell that's already attacking the last of those six links. Everyone before it has to have happened first, and that's where hot, cold, and excluded come from. A hot tumor has

T cells inside it. A cold one has none anywhere near it. An excluded one has T cells at the edge, stuck outside, unable to get in. Release the break on a T cell that isn't there, and you've released a break on nothing. Cold tumor has no immune response, start one. There's a pathway called sting that does exactly that when it detects DNA where DNA shouldn't be. So, inject a drug that switches it on straight into the tumor. Phase 1, 47 patients, one confirmed partial response, but read what they measured. They took biopsies before and after. Immune cell infiltration into the tumor, no significant change. Meanwhile, in the bloodstream, exactly what they'd hoped. Inflammatory signaling up, T cell populations expanding. The drug worked, the pathway switched on, the immune system woke up, and it never showed up at the tumor. One footnote, the single patient who did respond had Merkel cell carcinoma, the virus-driven cancer from earlier, the one tumor in that trial that was never cold to begin with. Different target,

Pancreatic cancer is the textbook excluded tumor. T cells reach the edge and stop, held off by dense surrounding tissue and a chemical signal called CXCL12. Block the receptor that reads it, and the door should open. In mice, it did. In people, 37 patients enrolled, 29 evaluable, the CXCR4 blocker plus pembrolizumab, no chemotherapy. CD8 effector T cells inside the tumor increased. Suppressive myeloid cells decreased. Regulatory T cells in circulation decreased. Every single thing the hypothesis predicted confirmed in human tissue. Response rate 3.4%, one patient. The door opened, the T cells walked in, nothing happened. Put those two side by side, they don't agree. The first drug, the immune system activated and the T cells never arrived. The second, the T cells arrived and nothing died. Opposite findings in the same chart, roughly nothing both times, which means it didn't work was never the useful sentence. Two questions were, which link did this actually target, and did the next link move? First drug targeted priming, priming moved, trafficking didn't. It broke at link

Four. Second drug targeted trafficking, trafficking moved measured in tissue, killing didn't. It broke at link six. Two drugs, two completely different broken links, one identical result. So, cold tumor isn't a diagnosis, it's a symptom. And the same symptom has at least two causes, which is exactly why a fix for one does nothing for the other. Right now, run that on me. Twice in this video I told you mutation count leaves 45% unexplained. That's 1 - 0.74 squared, and it's the number sitting right there next to the correlation. But a lot of those response rates come from 10 or 20 patients, and small samples make any correlation look weaker than it really is. We modeled that. Correcting for it, the honest figure is closer to 40%. 45 was the cell next door. I used it twice in a video about not doing that, and I'd have kept it if we hadn't run the model. And a worse one because this time I chose it, that radiation trial. I told you it

Missed the bar it set for itself, which is true. What I didn't tell you is the rest of the paper. The patients who benefited most were the ones whose tumors were PDL1 negative, the colder ones, which is a point for the whole warming up idea inside a trial that formally failed. I had a tidy argument running, and I handed you the half of that trial that fitted it. Same move as reaching for above 50 when the answer is 38.7%. Pick the cell that suits you. And then the one underneath all of them. I've been saying pancreatic cancer is an excluded tumor as though that's a fact about pancreatic cancer. It isn't. Profile the immune cells patient by patient, and pancreatic tumors split into at least two distinct types, which when a patient has predicts how long they live. So, hot, cold, and excluded aren't categories of cancer, they're descriptions of a sample. Ask, is pancreatic cancer cold? And you've asked the wrong shape of question, and I asked it out loud

For most of this video. So, there's the ladder, a number read one cell over, a field reporting the mechanism it moved instead of the outcome it missed, me handing you the half of a trial that suited my argument, and underneath all of it, a category error I couldn't see because I was standing inside it. So, the thing to do with it, pick any immunotherapy readout press release abstract conference slide, doesn't matter. Run the two questions, which link did this target? Did that link move? Is there a measurement in human tissue, not a mouse? And did the next link move? You pass if you can name the link and point at the measurement. You fail if the report hands you the clinical endpoint or the mechanism and never both. That failure is extremely common, and now you can see it. The script that produced every number in this video is in the description. 2 seconds to run, nothing to install. The cell next door, Tanmay Kulkarni, In for Humanitarian's AI, signing off.