LIVE

TechEarly Black Friday Brings Steep Price Drops on Apple's Latest iPhone Cases Markets*HOT* Craftsman Versastack 242-Piece Mechanic Tool Set with Hard Case only $99 shipped (Reg. $219!)ScienceZombies are real—just ask the parasites taking over the bodies and brains of insects and rodents ScienceStudy reveals how coral-associated bacteria help corals survive extreme heat BusinessWhat's Next for The Ensign Group, Inc. Shareholders After 8.2% Stock Drop? Glancy Prongay Wolke & Rotter LLP Explains Securities Fraud Class Action Lead Plaintiff Deadline MarketsChoice Hotels expects 2026 adjusted EBITDA of $641M-$651M as U.S. net rooms growth targets ~1% by year-end TechWhite Noise Machines Worthy of Your Nightstand in 2026: Snooz, Hatch, and More BusinessHE Dr Sultan Al Jaber and Sam Altman highlight energy, AI and investment priorities at ADIPEC 2026 opening LocalTrump Floats Pre-Emptive Pardons for Allies if Democrats Take Congress in Midterms BusinessWhat's Next for Webull Corporation Shareholders After 19.09% Stock Drop? Glancy Prongay Wolke & Rotter LLP Explains Securities Fraud Class Action Lead Plaintiff Deadline BusinessVirtual Public School Expert Kim Dunbar Explains Personalized Learning in an Online High School in HelloNation TechNew MacBook Pro with PROLED XDR accessories: Here are five top picks TechX for iPhone and iPad just added over a dozen new app icon options LocalRussia Senses Opportunity in Polish Political Turmoil BusinessMeta’s web crawlers are making life hell for owners of small business websites - Fast Company WorldFake nuclear warning from hacked app sparks unease but not panic in Ukraine MarketsBel Fuse declares $0.06 dividend LocalPBR Ram Challenger Series Championship Returns to South Point Arena Nov. 4–5 - Nevada Business Magazine TechThis Thing Runs On Android?MarketsBel Fuse declares $0.07 dividend LocalTrump administration asks Supreme Court to intervene in California ballots dispute BusinessMartin's Annual "Very Merry Martin's Giveaway" Turns Up the Cheer with a "Holiday Heatwave"LocalSupreme Court asked to intervene in California ballots case - Roll Call TechThis offline Wikipedia project is the perfect introduction to local LLMs - How-To Geek MarketsOption traders moderately bearish in Riot Blockchain with shares down 6.05% - TipRanksTRENDING TICKERS: $EL $NIQ $GIL $IQMX $CRWD $ASTS $ROK $APP $TENX $FLNC $BAC $HIMS $BBNX $VEEV $EG $TMUS $YFOR $MSTR $FCEL $SNDK $DAVA $QBTS $BETR $LINC $NOW $DUOL $BULL $ENSG $LKQ $FETechEarly Black Friday Brings Steep Price Drops on Apple's Latest iPhone Cases Markets*HOT* Craftsman Versastack 242-Piece Mechanic Tool Set with Hard Case only $99 shipped (Reg. $219!)ScienceZombies are real—just ask the parasites taking over the bodies and brains of insects and rodents ScienceStudy reveals how coral-associated bacteria help corals survive extreme heat BusinessWhat's Next for The Ensign Group, Inc. Shareholders After 8.2% Stock Drop? Glancy Prongay Wolke & Rotter LLP Explains Securities Fraud Class Action Lead Plaintiff Deadline MarketsChoice Hotels expects 2026 adjusted EBITDA of $641M-$651M as U.S. net rooms growth targets ~1% by year-end TechWhite Noise Machines Worthy of Your Nightstand in 2026: Snooz, Hatch, and More BusinessHE Dr Sultan Al Jaber and Sam Altman highlight energy, AI and investment priorities at ADIPEC 2026 opening LocalTrump Floats Pre-Emptive Pardons for Allies if Democrats Take Congress in Midterms BusinessWhat's Next for Webull Corporation Shareholders After 19.09% Stock Drop? Glancy Prongay Wolke & Rotter LLP Explains Securities Fraud Class Action Lead Plaintiff Deadline BusinessVirtual Public School Expert Kim Dunbar Explains Personalized Learning in an Online High School in HelloNation TechNew MacBook Pro with PROLED XDR accessories: Here are five top picks TechX for iPhone and iPad just added over a dozen new app icon options LocalRussia Senses Opportunity in Polish Political Turmoil BusinessMeta’s web crawlers are making life hell for owners of small business websites - Fast Company WorldFake nuclear warning from hacked app sparks unease but not panic in Ukraine MarketsBel Fuse declares $0.06 dividend LocalPBR Ram Challenger Series Championship Returns to South Point Arena Nov. 4–5 - Nevada Business Magazine TechThis Thing Runs On Android?MarketsBel Fuse declares $0.07 dividend LocalTrump administration asks Supreme Court to intervene in California ballots dispute BusinessMartin's Annual "Very Merry Martin's Giveaway" Turns Up the Cheer with a "Holiday Heatwave"LocalSupreme Court asked to intervene in California ballots case - Roll Call TechThis offline Wikipedia project is the perfect introduction to local LLMs - How-To Geek MarketsOption traders moderately bearish in Riot Blockchain with shares down 6.05% - TipRanksTRENDING TICKERS: $EL $NIQ $GIL $IQMX $CRWD $ASTS $ROK $APP $TENX $FLNC $BAC $HIMS $BBNX $VEEV $EG $TMUS $YFOR $MSTR $FCEL $SNDK $DAVA $QBTS $BETR $LINC $NOW $DUOL $BULL $ENSG $LKQ $FE

Skip to content

Anthropic, OpenAI, Google, and Microsoft will make it easier for publishers to opt out of being scraped

Source: Nieman Lab · Nov 02, 2026 17:15 UTC

The Quick Take

Right now, if you don’t want your site to be scraped by AI companies, you have to block each company’s crawlers individually. On Friday, Anthropic, OpenAI, Google, and Microsoft said they would support a common robots.txt signal that, per OpenAI, “makes it easier for publishers and website operators to communicate the same preference to multiple...

Read the full story at Nieman Lab →

Ground Level News aggregates headlines from public feeds. This page shows a short summary only; the full article and all rights belong to Nieman Lab.

Related

Early Black Friday Brings Steep Price Drops on Apple's Latest iPhone Cases

Amazon this week has big discounts across Apple's Clear, Silicone, and TechWoven Cases for the iPhone 18 Pro lineup. These deals are part of ongoing early…

MacRumors · 6m ago

White Noise Machines Worthy of Your Nightstand in 2026: Snooz, Hatch, and More

Catch more Z’s with soothing background noise from our favorite sound machines.

WIRED · 10m ago

New MacBook Pro with PROLED XDR accessories: Here are five top picks

The new MacBook Pro with PROLED XDR arrives in stores this Friday. If you’re looking for accessories for the new MacBook Pro, here are our five top…

9to5Mac · 14m ago

X for iPhone and iPad just added over a dozen new app icon options

Since X was called Twitter, the iPhone and iPad app has supported changing the default app icon to something with a bit more flair. Today’s app update…

9to5Mac · 14m ago