misk@sopuli.xyz to Technology@lemmy.worldEnglish · edit-221 hours agoApple Intelligence summary botches a headline, causing jitters in BBC newsroomwww.theregister.comexternal-linkmessage-square58fedilinkarrow-up1229arrow-down12
arrow-up1227arrow-down1external-linkApple Intelligence summary botches a headline, causing jitters in BBC newsroomwww.theregister.commisk@sopuli.xyz to Technology@lemmy.worldEnglish · edit-221 hours agomessage-square58fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up5·19 hours agoFor RAG data? It works. But its too slow for the weights. What generative models fundamentally do is run a full pass through the multi-gigabyte weights for every ‘word’ or diffusion step, so even 128-bit DDR5 like you find on desktop CPUs is too slow.
For RAG data? It works.
But its too slow for the weights. What generative models fundamentally do is run a full pass through the multi-gigabyte weights for every ‘word’ or diffusion step, so even 128-bit DDR5 like you find on desktop CPUs is too slow.