Meta’s new commercial terms will allow the scraping of news sites, legal experts say

Meta's updated commercial terms mean business accounts now authorise the crawling of associated websites and the use of that content to train its AI products, with no opt-out and little room for robots.txt to help. Three legal experts who reviewed the wording for Nieman Lab describe a provision worth millions in a licensing market. For publishers, the practical question is not whether to object but how quickly to audit crawler rules, contributor terms and archive rights, because content licensing is becoming a revenue conversation rather than a legal one.
Key Takeaway What it means for your publication
Consent is now bundled with access Meta’s updated commercial terms state that business accounts authorise automated crawling of associated websites and the use of that content to train its AI products. Roughly every news organisation holds a business account.
There is no opt-out switch The terms offer no way to remain on the business products while refusing the scraping permission. The only stated alternative is to stop using those products for commercial purposes.
Robots.txt is no longer a complete answer Meta publishes instructions for blocking its crawlers, but the new language gives it room to argue that contractual terms override those instructions.
Content licensing is the growth opportunity Legal experts put the licensing value of publisher archives in the millions. A functioning market rewards publications with clean rights records and structured archives.
Ownership of audience data is the durable defence Publishers with first-party relationships and clear consent records negotiate from strength. Those dependent on platform distribution absorb the terms of others.
Audit now, negotiate later Review crawler rules, archive rights and subscription terms this quarter. The publishers who document their position early set the price when the market matures.

Publishing technology decisions rarely arrive with a legal deadline attached. This one did. Meta’s updated commercial terms, now applying to business accounts across Facebook, Instagram and WhatsApp, state that the company may use automated software to crawl, index, cache and collect content from domains associated with those accounts, and use that material to train its AI products. Almost every news organisation holds such an account, including the trade press that covers the industry.

For publishing executives the interesting question is not whether Meta has overreached. Three legal experts who reviewed the terms for Nieman Lab have already made that case in detail. The useful question is what you do next, and why content licensing deserves a place in your business plan rather than the legal folder marked for later.

This guide sets out what the terms actually permit, why a working licensing market matters more than a robots.txt entry, and how to structure your publishing platform, archive and reader relationships so that consent becomes something you negotiate. The Publishrs team works with publishers on precisely this question of control.

What Meta’s new commercial terms actually permit

The terms cover business accounts that manage pages and groups, run advertising, send marketing messages and use Meta’s business dashboard. For publishers, the practical effect is that access to the tools used to distribute stories also carries permission to collect them.

Consent by default, with no opt-out window

Legal experts who examined the wording for Nieman Lab described it in unusually blunt terms. Matt Topic, a media and intellectual property attorney at Loevy and Loevy, called it a remarkable provision to bury in terms of service, noting that it gives Meta access to material that would likely cost many millions in a licensing market.

James Grimmelmann, a professor of digital and information law at Cornell Law School, went further. He told Nieman Lab that the wording represents permission to train AI on any content from publisher sites, and possibly to deploy some of that content in competition with the publishers themselves. On the broadest reading, he suggested it would support building a comprehensive archive of journalism.

There is no opt-out mechanism. The terms state that anyone who disagrees should stop all access or use of Meta’s products for business or commercial purposes. For a publisher whose audience development runs through those channels, that is not a choice so much as an ultimatum.

Test credentials and the limits of robots.txt

Meta points out that crawling language was previously scattered across different sets of terms, including those covering generative AI in advertising, and that the update consolidates it. A company spokesperson told Nieman Lab that the change does not alter how Meta crawls publisher sites.

The technical guidance is thinner than the contractual language. Meta’s developer documentation explains how to block its crawlers through robots.txt, the file that tells visiting bots which parts of a site they may access. Legal experts argue the new terms create room for Meta to claim its contract takes precedence over those instructions.

The update also allows Meta to create test credentials, which Grimmelmann described as a technical route to sign up as a fake user and pass registration walls. In practice, three layers of publisher control come under pressure at once:

  • Technical controls, such as robots.txt directives and crawler blocks that are voluntary rather than enforced.
  • Registration walls, which assume identities are genuine, an assumption that test credentials undermine.
  • Contractual terms on your own site, which now compete with a larger counterparty’s terms of service.

Why content licensing is now a publisher growth lever

The commercial asymmetry is the part that should hold a board’s attention. Meta has begun charging some users to post links through its subscription products, while reserving the right to collect the content those links point towards at no cost. That is not a neutral arrangement, and it makes the licensing question concrete.

The market Meta declined to enter

If archives carry licensing value in the millions, then the absence of a deal is a revenue decision made on your behalf. Publishers who treat that as a legal grievance tend to stay where they are. The ones who treat it as a market failure tend to build the thing the market needs, which is a documented, auditable, priced body of rights.

Benjamin Sobel, a law professor at the University of Wisconsin, framed the arrangement through the antitrust idea of tying. If you want access to products that require acceptance of the commercial terms, you must also accept the crawling permissions exacted in them. He compared the move with Google’s approach to its AI search products, where publishers initially had to consent to appear in AI Overviews before an opt-out arrived in August 2026.

That comparison is the encouraging part. Platform terms have shifted before under collective pressure, and the fair use questions surrounding AI training remain unsettled in court. A US federal appeals court recently upheld Thomson Reuters’ copyright claim against a legal AI developer, a result that strengthens the hand of anyone holding a licensing dossier.

What a licensing-ready archive looks like

Most publishers cannot answer basic questions about their own content: which rights they hold, which contributors signed which agreements, which assets came from agencies, and which archive items were syndicated under restrictions. That gap is the real barrier to a licensing conversation. A commercial team cannot sell what the editorial system cannot describe.

A useful archive has four characteristics, and each one starts as an editorial workflow problem rather than a legal one:

  1. Clean provenance for every asset, including contributor contracts, agency terms and syndication limits recorded at the point of publication.
  2. Structured metadata that identifies topic, author, region and date at article level, so a licence can be scoped precisely.
  3. Version control that distinguishes the published record from later corrections and updates.
  4. Exportable records that a buyer, auditor or court can read without translating your content management system.

This is where the choice of publishing platform stops being a matter of editorial convenience. Publishrs’ publishing platform was built around structured content and owned audience data, which is what makes this kind of audit possible without a six-month archaeology project.

How publishers can respond without overreacting

The temptation after news like this is to make a dramatic technical gesture and consider the matter closed. Blocking every crawler in sight satisfies the instinct and damages the distribution that still pays the bills. A measured response works through the layers in order.

Audit the crawler rules you already have

Start by establishing what you currently permit, in writing, and where those permissions live. Many publishers discover that legacy directives were set years ago by a developer who has since left, and that nobody can explain why certain paths are open. That is not a position from which to negotiate.

Then review your subscription and registration terms against the possibility that credentials can be auto-generated. Publish a clear statement of your position on AI crawling, dated and attributable, so that any future dispute starts with your documented intent rather than an inference.

Put consent in the contract, not just the config

Technical controls are advisory. Contracts are enforceable. Publishers with contributor agreements, freelance terms and syndication deals that explicitly set out AI training rights hold an asset that config files cannot provide.

For editorial leaders, that means three practical changes this quarter:

  • Add an explicit AI training clause to every new contributor and freelance agreement, replacing silence with a stated position.
  • Record licence scope at ingestion, so rights travel with the asset instead of living in an inbox.
  • Brief commercial teams on what may and may not be licensed, before a buyer asks rather than afterwards.

The publishing platform decisions that keep rights intact

Underneath the legal argument sits an infrastructure question. Publishers who know what they own, and can prove it, will set the terms of the next negotiation. Those who cannot will accept whatever arrives in their business account settings.

Own the reader relationship

Distribution on someone else’s platform is borrowed reach, and the terms attached to it belong to the lender. That is why audience development has quietly become a rights strategy. A reader who subscribes directly, with a consent record you control, is an asset that no terms of service can reclassify.

First-party data does the same work commercially that a licensing dossier does legally. It lets you describe your audience with precision, price access honestly and demonstrate value to advertisers without relying on a third party’s reporting. Publishrs’ approach to audience data puts that record in the publisher’s hands rather than a platform’s.

Build for the next negotiation

Nobody can predict how the fair use cases will resolve or whether a licensing market will settle into standard rates. What is predictable is that every future negotiation will reward the publisher who can produce a complete, machine-readable record of what it published and on what terms.

Industry bodies including WAN-IFRA and the Reuters Institute have spent the past two years arguing that publisher data and rights are strategic assets rather than administrative overhead. The Meta terms make that argument practical rather than theoretical.

There is also a competitive angle that editorial teams appreciate. Publishers who can prove provenance are easier to trust, and trust is what sponsors, licensees and readers pay for. Digiday has documented how often revenue conversations now turn on exactly that question of demonstrable ownership. Background on the changing shape of the market is available from What’s New in Publishing.

Questions publishers are asking

Does this affect every news organisation, or only large publishers?

Every organisation with a business account on Facebook, Instagram or WhatsApp is covered, and that includes small and independent publishers. Scale does not change the exposure, although it does change the strength of your position in a negotiation.

Can we simply block Meta’s crawlers?

You can publish robots.txt directives that instruct crawlers to stay out, and Meta documents this. The legal experts who reviewed the terms for Nieman Lab suggest the contractual language may be used to argue those instructions no longer apply. Blocking is a starting point, not a complete defence.

Is opting out of the terms possible?

The terms offer no partial opt-out. The stated alternative is to stop using the business products altogether, which for many publishers would mean abandoning a distribution channel their audience development depends on.

What does the Meta One subscription change?

It separates two activities that used to be free and symmetrical. Some accounts now pay to post external links, while the content those links point to can be collected under the commercial terms. That asymmetry is the clearest argument for putting a price on licensing.

Does a court ruling on AI training already exist?

Some do, though the picture is far from settled. A US federal appeals court recently upheld Thomson Reuters’ copyright claim against a legal AI developer, which supports the view that training data has to be paid for. Several other cases between publishers and AI developers are still moving through the courts.

Should we block AI crawlers across the whole site?

Rarely, as a first move. Blocking everything removes you from search and referral traffic as well as training pipelines. A scoped policy that names which crawlers may access which sections, and why, protects value without cutting off the distribution that still earns revenue.

How quickly should a publisher act?

The audit can begin immediately and costs little. Contributor terms and rights records take longer, which is the argument for starting them now rather than when a licensing buyer arrives with a deadline.

Where publishers go from here

The terms are unlikely to be reversed by argument alone. What shifts arrangements like this is commercial pressure from publishers who can demonstrate what their content is worth and who retains the right to say no. That is a position built on documentation, not indignation.

It also rewards publishers who treat their platform as infrastructure rather than a utility bill. If your current system cannot tell you which rights you hold across ten years of archive, the licensing conversation cannot start, and the default answer remains someone else’s terms.

For publishers working through that question, the team at Publishrs is glad to talk through what a rights-aware publishing platform looks like in practice, and which parts of the work can start this month.

This article provides general information about publishing industry trends and best practices. For specific advice about implementing new systems or processes at your publication, we recommend consulting with your technical and editorial teams.

Publishrs.com

The official blog for Publishrs.com – the all in one digital publishing platform

Read More

Sign up for our Newsletter

Get the latest publishing news straight to your inbox