stopthatgirl7@kbin.social to Technology@kbin.social · 11 months agoNow you can block OpenAI’s web crawlerwww.theverge.comexternal-linkmessage-square4fedilinkarrow-up132arrow-down10file-textcross-posted to: technews@radiation.party
arrow-up132arrow-down1external-linkNow you can block OpenAI’s web crawlerwww.theverge.comstopthatgirl7@kbin.social to Technology@kbin.social · 11 months agomessage-square4fedilinkfile-textcross-posted to: technews@radiation.party
minus-squareNaich@kbin.sociallinkfedilinkarrow-up10·edit-211 months agoIn /robots.txt - User-agent: GPTBot Disallow: / If you don’t want to read the article. Presumably they are doing this now because they have scraped everyone already.
minus-squarebionicjoey@lemmy.calinkfedilinkarrow-up1·11 months agoBetter to ask for forgiveness than to beg for permission
In /robots.txt -
User-agent: GPTBot Disallow: /If you don’t want to read the article. Presumably they are doing this now because they have scraped everyone already.
Better to ask for forgiveness than to beg for permission