Exploring Open-Source AI Development & GitHub Collaboration | Mycroft AI

[0:00]11 episodes of teaching a system to read what executives say. This one asks a question the system had never asked about itself. How long is any of it good for? I'm Anjana. ASIS extracts financial guidance signals from earnings calls using four readers and a five-dimensional triangulator. The prediction layer forecasts guidance direction and a revision predictor uses six models to flag likely changes. Episode 11 added Q and a dodge detection with crossing coder ranking, competitive positioning with a three classifier ensemble, a guidance revision predictor, fine-tuned sentence bird embeddings, and a fin bird ensemble. This week, not every signal ages the same way. Some decay in days, some last for months.

[0:46]Phase 17 measures signal shelf life, catches false positives, reads the structure behind management language, and prunes the feature set. Different signals lose predictive power at different rates. The signal decay module measures correlation with excess returns at horizons from 1 day to 60 days. Each signal type gets a half-life, the point where its predictive value drops below significance. The hedging index might hold for 20 days. Dodge scores may fade in five. The system uses these half-lives to weight recent data appropriately. Fast-decaying signals count for less at longer horizons. High confidence does not guarantee accuracy. The false positive analysis module examines every prediction where confidence exceeded 0.7,

[1:32]But the direction was wrong. It clusters the errors by root cause, ambiguous language, sarcasm misread, section misattribution, context-dependent phrases, or unusual market conditions. Signals matching known patterns get a reduced confidence modifier. At the same time, multi-quarter signal chaining tracks trends across consecutive calls. Three quarters of rising hedging with declining confidence is no longer three independent readings. It's a sustained deterioration signal. A fine-tuned Deberta classifier sorts every management statement into speech act types: promises, commitments, hedged intentions, predictions, deflections, and assertions. The distribution reveals whether management is making concrete commitments or filling time with vague intentions. Discourse structure analysis goes deeper. It maps how the narrative is organized. Front-loading puts bad

[2:28]News first. Varying sandwiches it between positives. Deflection pivots away without addressing the negative. Each pattern is a signal about how management wants you to hear the story. As the feature set grows, not every feature earns its place. Automated feature selection runs three methods: recursive feature elimination, permutation importance, and lasso shrinkage, and flags features that rank in the bottom 10% across all three for removal. Feature interaction discovery identifies synergistic combinations. High dodge score plus rising hedging plus defensive language predicts outcomes better than any feature alone. These interaction features are added to the vector. Embedding drift monitoring tracks whether the sentence BERT space is shifting and flags when refine-tuning is

[3:15]Needed. Phase 17 added signal decay half-lives, false positive filtering, multi-quarter signal chains, transcript quality gates, speech act classification, discourse structure analysis, metaphor tracking, sarcasm detection, automated feature selection, interaction features, and embedding drift monitoring. The system doesn't just extract signals anymore. It measures how long they last, catches when they're wrong, reads the structure behind the language, and prunes its own feature set. Episode 12, how long does a signal last? Let's recap with Claude. 11 episodes built a system that extracts signals. This one makes it skeptical about them. Every signal type now has a measured shelf life. So, a reading that was good yesterday isn't assumed to be

[4:00]Good next month. Every confident prediction that turned out wrong gets clustered by Y, and signals matching a known failure pattern are marked down before they're used. Management language gets read for its structure. What kind of statement was that? And where in the story was the bad news put? And the feature set is pruned by three methods that have to agree, which is the part most teams skip. Your turn. Here's the uncomfortable one. Most of us build a score and then never ask how long it stays true, or go back and look at the times it was confident and wrong. Take a score you already produce and do

[4:35]Both. Paste this into Claude and press hardest on the second part, because a list of your own confident failures, sorted by cause, is the most useful thing you can build this month, and the least pleasant. And Jonna here, thanks for watching.