WorkingLemmy@lemmy.world to Open Source@lemmy.ml · 1 month agoFOSS infrastructure is under attack by AI companiesthelibre.newsexternal-linkmessage-square30fedilinkarrow-up1390arrow-down11cross-posted to: technology@beehaw.orglinux@programming.devopensource@jlai.lutechnology@lemmy.worldopensource@programming.devOpenSource@europe.pubtechnology@lemmy.worldtechnology@lemmy.worldhackernews@lemmy.bestiver.se
arrow-up1389arrow-down1external-linkFOSS infrastructure is under attack by AI companiesthelibre.newsWorkingLemmy@lemmy.world to Open Source@lemmy.ml · 1 month agomessage-square30fedilinkcross-posted to: technology@beehaw.orglinux@programming.devopensource@jlai.lutechnology@lemmy.worldopensource@programming.devOpenSource@europe.pubtechnology@lemmy.worldtechnology@lemmy.worldhackernews@lemmy.bestiver.se
minus-squarefuckwit_mcbumcrumble@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up9·1 month agoThey’re absolutely not crawling it every time they nee to access the data. That’s an incredible waste of processing power on their end as well. In the case of code though that does change somewhat often. They’d still need to check if the code has been updated at the bare minimum.
minus-squarebeeng@discuss.tchncs.delinkfedilinkarrow-up2·1 month agoHashes for cached content. Anyone know what sort of DB makes sense here?
They’re absolutely not crawling it every time they nee to access the data. That’s an incredible waste of processing power on their end as well.
In the case of code though that does change somewhat often. They’d still need to check if the code has been updated at the bare minimum.
Hashes for cached content. Anyone know what sort of DB makes sense here?