今日已更新 177 条资讯 | 累计 29911 条内容
关于我们

今日精选

HOT

最新资讯

共 29911 篇
第 292/1496 页
AI 资讯 The Verge AI

What Surrounds Us will make you think a lot about circles

What Surrounds Us takes its title literally. You play as a circle surrounding a hole in its middle - it looks like a donut with frosting. You work together with other sentient moving circles, sometimes helping them joyfully meet with others. And you traverse a large map that's composed entirely of, you guessed it, a […]

Jay Peters 2026-07-25 20:00 6 原文
AI 资讯 Dev.to

ML Without Magic: Building a Tiny Language Model in Pure Node.js and Watching Every Weight Change

Tokenization → embeddings → causal Transformer → LM head → softmax → loss → backpropagation. No TensorFlow, no PyTorch, and no hidden autograd. Repository: tiny-language-model-neuro-js . Most explanations of language models present correct formulas but hide the path between them inside a framework. I wanted the opposite: one small scenario where every scalar is visible and where the terminal clearly shows incorrect answers before learning and correct answers after it. The project now has one command: node src/train.js --generalize --adaptive-teach It requires Node.js 18.19+ and has no dependencies. The result first The model is queried immediately after random initialization: BEFORE TRAINING — random, usually wrong answers > can human read ? model: ? <unk> ... expected: human can read. [WRONG] > can fish swim ? model: ? <unk> ... expected: fish can swim. [WRONG] > can cat read ? model: ? <unk> ... expected: cat cannot read. [WRONG] After pre-training, SFT, and adaptive SFT, the same model produces: FINAL ANSWERS AFTER ADAPTIVE SFT > can human read ? model: human can read. [CORRECT] > can fish swim ? model: fish can swim. [CORRECT] > can bird fly ? model: bird can fly. [CORRECT] > can cat read ? model: cat cannot read. [CORRECT] Rehearsal controls preserved: 14/14. Stable criterion reached 11 times in a row. The initial text varies because initialization is random. The final acceptance criterion does not: all answers must be correct, every target token must have at least 95% probability, and the complete check must pass more than ten times consecutively. What remains after removing the extra modes The code previously contained several debug and training modes. They were useful while experimenting but obscured the main idea. The final version keeps one educational pipeline: text → word tokenization → token IDs → token + position embeddings → two causal Transformer blocks → multi-head self-attention → two-hidden-layer FFN → LM head → softmax → next-token probabilities

Maksim Sekretov 2026-07-25 17:34 12 原文