GDPR and Web Scraping: What US Teams Need to Know

The GDPR applies to web scraping whenever you collect personal data about people in the EU, even if your servers, your company and your proxies are all in the United States. Scraping public pages does not exempt you: you need a lawful basis, a defined purpose, minimal collection and respect for the site's opt-outs. Your proxy layer does not change any of that. It does decide whether your collection is clean, scoped and reproducible, and that is where we come in.
This guide is for developers, data scientists, marketers and researchers who run scrapers and want a practical reading of the rules. It is not legal advice. If your project collects personal data about EU residents at scale, have a privacy lawyer or your data protection officer review it.
TL;DR
- The GDPR reaches US organizations that offer goods or services to people in the EU or monitor their behavior, wherever the data is processed.
- Public does not mean free to use. Scraped personal data needs one of six lawful bases, and for scraping that is usually legitimate interest, backed by a documented balancing test.
- The EDPB's July 2026 guidelines on web scraping stress precise collection criteria, filtering, short retention and respect for robots.txt and similar opt-outs.
- Price monitoring, ad verification and most market research can often be designed to collect no personal data at all, which keeps the GDPR out of the picture.
- Proxies are infrastructure. Compliance sits in what you collect, why, and how long you keep it.
Table of Contents
- When the GDPR applies to a scraper
- Which lawful basis can a scraper use?
- What the EDPB expects from your pipeline
- Who is the controller, and who is the processor?
- Beyond the GDPR: terms, copyright and database rights
- What enforcement looks like
- Use cases that sit mostly outside the GDPR
- A compliance checklist for a scraping project
- Where the proxy layer fits
- Our view
- Start a geo-targeted collection project
- FAQ
- Sources
When the GDPR applies to a scraper
The regulation protects people, not servers. The European Commission says it covers a company with an establishment in the EU, and also a company outside the EU that offers goods or services or monitors the behavior of individuals in the EU. A US startup scraping European forum profiles to build a lead list is monitoring and profiling people in the EU. A US team scraping hotel prices is not processing personal data at all.
Two definitions decide most cases:
- Personal data is anything that identifies a person directly or indirectly: names, emails, usernames, photos, reviews tied to a profile, and combinations of fields that point to one individual.
- Processing is almost any operation on it. The EDPB's Guidelines 03/2026 say scraping counts as processing when it involves extraction, cleaning, structuring and storing of personal data, including data that only indirectly identifies someone.
The EDPB (the body of EU data protection authorities that issues common guidance) adopted those guidelines on 7 July 2026 as version 1.0. They are framed around generative AI training, but the scraping principles apply to any project that collects personal data from the web. They build on the EDPB's earlier Opinion 28/2024 on AI models, according to a law firm summary of the July and September 2026 plenaries.
A quick test before you scrape
Ask three questions about each field you plan to collect:
- Could this field, alone or combined with others in my dataset, identify a living person?
- Could that person be in the EU, or is my project aimed at the EU market?
- Do I need the field for the stated purpose?
If the answer to the first is no, you are mostly in terms-of-service and copyright territory rather than GDPR territory. If the first two are yes, keep reading.
Which lawful basis can a scraper use?
The Commission lists six lawful bases: consent, contract, legal obligation, vital interests, public interest and legitimate interests. Scraping rarely fits the first five. You have no relationship with the people on the page, so there is nobody to ask for consent and no contract to perform.
That leaves legitimate interest. France's data protection authority, the CNIL, says that collecting personal data available online through scraping is generally based on legitimate interest, provided the controller adds measures to limit the impact on individuals. The EDPB's Guidelines 1/2024 explain that legitimate interest needs three cumulative conditions, and the assessment must be done before the processing starts:
| Condition | What you document |
|---|---|
| A legitimate interest | The concrete purpose, such as market research or fraud detection |
| Necessity | Why scraping is needed and why less intrusive data would not do |
| Balancing | Why the individuals' rights and expectations do not outweigh your interest |
What the balancing test punishes
The CNIL is direct: processing falls outside people's reasonable expectations if the controller does not exclude websites that object to scraping through robots.txt or CAPTCHAs. It also says sensitive data can be collected only where an Article 9 exception applies, for example where the person has manifestly made it public.
The Dutch regulator goes further. As summarized by Taylor Wessing, the Dutch DPA says that people posting their own data online does not imply consent to scraping. It also says that if you cannot be sure a dataset excludes special category data (health, political views, religion and similar), all of it is treated as if it contained such data. For a data scientist, that second point is the practical one: broad scrapes of social profiles inherit the strictest rules.

If you want a robots.txt routine you can run in code review, our seven robots.txt checks cover it.
What the EDPB expects from your pipeline
The July 2026 guidelines read like a pipeline specification. According to the EDPB text, the controller must meet these requirements:
- Purpose limitation: a defined purpose before collection starts, not after you see what you got.
- Transparency: inform data subjects, with an exception where individual notice is impossible or disproportionate.
- Data minimization: precise collection criteria, filters, exclusion of unnecessary data categories and of sites that oppose scraping.
- Accuracy: reliable sources, timestamps and validation.
- Fairness: no use that people would not reasonably expect.
The guidelines also tell controllers to consider synthetic data, to anonymize or pseudonymize where feasible, and to limit how long data is collected and stored. Where individual notice is impossible or disproportionate, a law firm summary notes that you must make the information public and take further measures, such as publishing a data protection impact assessment (DPIA, a written risk assessment for high-risk processing), anonymizing data or tightening security.
Turning the principles into engineering tasks
| Principle | What it looks like in a scraper |
|---|---|
| Minimization | Select fields at parse time and drop everything else before it reaches storage |
| Exclusion | Honor robots.txt and skip pages that serve CAPTCHAs as an explicit opt-out |
| Accuracy | Store a fetch timestamp and source URL with every record; validate on ingest |
| Retention | Set a time-to-live on raw HTML and a schedule to delete or aggregate |
| Security | Encrypt at rest, restrict access, keep credentials per team member |
| Transparency | Publish a plain-language notice describing what you collect and why |
Accuracy is the principle teams underinvest in. Timestamps and validation also make your numbers better, which is why an AI data pipeline built around source reliability and validation pays off twice: once for regulators, once for model quality.
Who is the controller, and who is the processor?
Roles decide who answers to regulators. The EDPB says roles should be analyzed case by case. As a law firm summary explains, a scraper that follows an AI developer's documented instructions may be a processor while the developer is the controller, and joint controllership can arise where organizations jointly decide purposes and means. The organization doing the scraping is not automatically the controller. If a model developer trains on a dataset someone else scraped, each party is in principle responsible for its own processing.
For your team the lesson is contractual and practical:
- If you scrape for a client, write down whose instructions you follow and who decides the purpose.
- If you buy a scraped dataset, you carry your own obligations for what you do with it, including a lawful basis of your own.
- Keep a record of who can launch jobs and with which credentials, so you can show how the data was collected. Our notes on operating rules for scraping teams cover this.
Beyond the GDPR: terms, copyright and database rights
The GDPR is one layer. Scraping can also be prohibited by a site's terms of use, by copyright, or by database rights. The CNIL notes that scraping may be barred by terms based on database rights or copyright, and that the French text-and-data-mining exception may apply unless rights holders have reserved their rights. Non-personal data such as prices and product specifications is far less regulated under the GDPR, but contract and database law still apply.
For US readers, that means a separate check against US rules on top of the EU analysis. Our overview of US, EU and UK scraping rules walks through the guidance and key cases side by side.
What enforcement looks like
The fines that made scraping a board-level topic involve scraping personal data at scale:
- The Italian authority fined Clearview AI EUR 20 million in February 2022 for unlawful processing, including biometric and geolocation data, and ordered a ban on further collection and erasure of data on people in Italy.
- The case also shows that a US base is no shield: the company had to designate an EU representative.
The Dutch DPA's guidance, again via Taylor Wessing, gives a useful contrast. Lawful examples include scraping public news sites for current affairs, a web shop scraping its own pages for customer reviews, and scraping a public information security forum to understand risks to your own organization. Unlawful examples include building profiles to resell, scraping private social media accounts or forums, and using scraped data to decide insurance eligibility.
The pattern is consistent. Narrow, purpose-bound collection from public sources is defensible. Profile-building about individuals is not.
Use cases that sit mostly outside the GDPR
Most of what our users build never needs personal data:
- Price comparison across regions: product, price, availability and shipping fields. No individuals involved.
- Ad verification: which ad appeared, where, on which page, from which location.
- Local search testing: rankings and listings as seen from a given city.
- Public-sector and open data: datasets published by agencies, which our guide to city-level scraping covers alongside the EDPB's guidance.
In these projects the GDPR job is mostly to keep personal data from sneaking in. Reviews with usernames, seller names on marketplace listings and contact fields on business pages are the usual culprits. Drop them at parse time, and your dataset stays outside the regulation. For a document-level view of what a privacy review looks for, this AI privacy checklist for procurement and security teams shows the questions reviewers ask.
A compliance checklist for a scraping project
- Write the purpose. One paragraph: what you collect, why, and what decisions it feeds.
- Classify the data. List every field and mark personal, sensitive or non-personal.
- Choose the lawful basis. For personal data, document the three-part legitimate interest assessment before the first request.
- Define collection criteria. Domains, page types, fields and date ranges, with filters in code.
- Respect opt-outs. Exclude sites that block scraping through robots.txt or CAPTCHAs, and review their terms.
- Minimize at ingest. Drop unneeded fields before storage and pseudonymize identifiers where you can.
- Set retention. Delete raw data on a schedule and keep only what the purpose requires.
- Plan for rights requests. Know how you would find and erase one person's records.
- Decide on a DPIA. Large-scale or sensitive collection warrants one; involve your data protection officer or counsel.
- Log the pipeline. Timestamps, source URLs, job owners and credentials, per team.
Where the proxy layer fits
A proxy has no say in whether your purpose is lawful. It does shape the quality and scope of what you collect, and a few technical choices make the checklist easier to follow.
Scope by location. MaskLabs offers real carrier IP addresses in 46 US cities, so a price or ad check can be pinned to the city you care about instead of sweeping broad regions. A narrower collection footprint supports the minimization principle.
Authentic traffic, fewer failures. Real mobile carrier IPs make requests look like ordinary visitors, which reduces blocking and throttling. Fewer blocks mean fewer blind retries and fewer pages fetched that you never meant to collect. If you hit walls anyway, our guides on 403 errors, CAPTCHAs and bot detection signals show how to diagnose them. Just remember the CNIL's point above: a CAPTCHA that expresses a site's opposition is a reason to stop, not a puzzle to defeat.
Sticky or rotating sessions. Sticky sessions hold one IP for a session so a multi-page flow completes cleanly. Rotating sessions spread independent requests. Match the mode to the job, and log which you used.
Metered, per-byte billing. You pay for the data you transfer, so tight filters and lean requests show up directly on your usage dashboard.

Credential management for teams. Self-serve credentials by team let you show who ran which job, which supports the logging step in the checklist.
Our view
Most teams that ask us about GDPR are asking the wrong layer. They wonder whether a proxy makes scraping lawful. It does not, and no proxy can. What we can do is give you real US mobile carrier IPs, city-level targeting, sticky and rotating sessions, HTTP, HTTPS and SOCKS5 support, and a usage dashboard, so your collection is precise and repeatable.
The best compliance posture we see on paper is simple: collect only what the purpose needs, stop where a site says no, and keep records. Per-byte metering, team credentials and city targeting fit that approach, because you can see what you pulled, who pulled it and from where.
If your project is price monitoring, ad verification or geotargeted research, design it so personal data never enters the pipeline. Then the regulation has little to say about it, and you get cleaner data too.
Jon
Start a geo-targeted collection project
If your use case is price monitoring, ad verification or location-specific research, you can set up access without a sales call. Plans run from Starter at $30.00 per month, through Basic at $100.00 and Advanced at $500.00, to Scale at $1,000.00 per month, sized by monthly data pool, with prepaid data add-ons billed per GB and a free trial with limited data. See the details on the MaskLabs pricing page, or look at our US mobile proxies on real carrier IPs.

FAQ
Does the GDPR apply if my company and servers are in the US?
Yes, if you offer goods or services to people in the EU or monitor their behavior. The Commission says the regulation applies regardless of where the data is processed. Scraping only prices or product data with no personal fields generally falls outside it.
Is scraping publicly available personal data allowed?
Public availability does not remove the need for a lawful basis. The CNIL says scraping generally rests on legitimate interest, with extra safeguards such as excluding sites that object. Dutch guidance adds that posting data online is not consent to scraping. Document your balancing test before you start.
Do I need a DPIA for a scraping project?
The EDPB lists publishing a DPIA among measures that can compensate when individuals cannot be notified directly. Large-scale collection of personal data, or any chance of sensitive data, is a strong reason to run one. Ask your data protection officer or privacy counsel to confirm for your project.
Does using a proxy make scraping more or less compliant?
Neither. Compliance depends on what you collect, your lawful basis, minimization, retention and respect for opt-outs. A proxy is the network path. Used well, with city targeting and per-byte metering, it helps you keep collection narrow and records clear.
What changed in the EU guidance recently?
The EDPB adopted Guidelines 03/2026 on web scraping on 7 July 2026 as version 1.0, and it adopted four sets of guidelines across its July and September 2026 plenaries. Check the EDPB site for the final text, since version 1.0 was released for consultation.
Sources
- Application of the GDPR, European Commission
- Legal grounds for processing data, European Commission
- CNIL focus sheet on legitimate interest and web scraping
- EDPB Guidelines 1/2024 on legitimate interest
- EDPB Guidelines 03/2026 on web scraping
- Elections and web scraping: the Dutch DPA's guidance, Taylor Wessing
- EDPB guidelines on web scraping for generative AI, HLC
- EDPB guidelines on web scraping for generative AI, HLC Canada edition
- EDPB adopts four sets of guidelines, Stibbe
- AI data pipeline stages for data engineers
- Spreadsheet AI privacy checklist