How to Scrape Compass Real Estate Listings With Browse AI
Compass’s terms prohibit scraping its listings. Learn what Browse AI can do on permitted sites, why a Compass recipe is inappropriate, and how to source listing data responsibly.
Direct answer: Do not use Browse AI to scrape Compass listings unless Compass has expressly authorized that activity in writing. Compass’s published Terms of Service prohibit manual or automated crawling or spidering of its pages and harvesting or scraping its content. Browse AI’s ability to extract data from some websites does not grant permission to collect Compass data. Read Compass’s Terms of Service.
This guide explains the restriction, describes Browse AI’s general workflow for websites where automated extraction is allowed, and outlines safer ways to obtain real estate data. It does not provide a Compass robot recipe or claim that one works. No Compass-specific extraction was performed.
1. Check permission before collecting listing data
Compass’s Terms of Service, last updated July 27, 2026, grant users access to content for personal, non-commercial purposes subject to the terms. They prohibit crawling or spidering site pages and scraping or harvesting Compass content. The terms also prohibit using Compass content to train, fine-tune, validate, or develop AI models, or to create derivative datasets, embeddings, or similar representations, without express written authorization.
Some listing data may also be governed by MLS, IDX, or VOW rules. Those rules can limit use to personal, non-commercial purposes and require accurate data and broker attribution. Listing descriptions, photos, contact details, and other content can have additional rights attached. Do not assume that publicly visible content is free to reuse.
Browse AI makes the same general point: the source site’s terms and applicable rules govern what users may do with extracted data. Its tool’s capabilities are not evidence that a particular collection is permitted. See Browse AI’s guidance on rights in scraped data.
2. How Browse AI works on sites where extraction is permitted
Browse AI documents two general ways to create a robot. These are product workflows for sources whose terms and licenses allow your planned use; they are not instructions or a verified setup for Compass.
Table Studio: data already visible on a page
- Choose a source page you are authorized to collect from and identify the fields you need.
- In Browse AI, start a new robot and provide the page URL as the Origin URL.
- Use Table Studio to generate a structured table from the page.
- Review the selected fields and sample rows for missing, duplicated, or incorrectly associated values before saving.
- Configure exports or integrations only in ways allowed by the source’s terms, your data license, and applicable privacy rules.
Browse AI describes Table Studio as a way to build a data table from a URL. Its first-robot guide covers that general process.
Robot Studio: a permitted page needs interaction
For a permitted site where the desired data appears only after ordinary page interactions, Browse AI’s Robot Studio uses visual, point-and-click training. Its documentation describes selecting lists or specific text, interacting with dynamic page elements, and monitoring changes on a schedule. Configure navigation and monitoring only when the source permits automated access and the frequency you intend to use.
For either approach, keep a record of the source, the permission or license basis, permitted fields and uses, attribution requirements, and any collection limits. Browse AI’s responsible-use guidance also advises checking terms, robots.txt, rate limits, privacy obligations, and copyright considerations. A robots.txt file is not a substitute for permission where the site’s terms prohibit scraping.
3. Browse AI compatibility limits
Even when extraction is permitted, technical behavior can make a robot incomplete or unreliable. Browse AI’s compatibility documentation notes several limitations:
- Dynamic content: data that appears after scripts run or after interaction may require a different training flow and can change over time.
- Login and high-security sites: some sites with strong access controls may not work reliably. Do not try to get around a site’s access restrictions.
- MFA or 2FA: Browse AI says these are not supported.
- CAPTCHAs: some standard CAPTCHA types are supported, but not every custom CAPTCHA. Do not attempt to bypass access controls.
- Virtual lists: lists that render only the currently visible rows while scrolling are identified as unsupported.
- iFrames: direct interaction with iFrames is not supported; a separately accessible frame URL may sometimes be usable when authorized.
- Page changes: a site redesign, changed labels, or a different result layout can invalidate selections and require the robot to be reviewed.
These are general product limitations, not a prediction of Compass compatibility. Browse AI’s website compatibility guide does not establish an authorized or working Compass extraction.
4. Safer ways to obtain real estate listing data
If your project needs listing data, use a source and license that expressly cover the intended collection and use. Before building an integration, compare candidate sources against these questions:
| Check | What to confirm |
|---|---|
| Permission and license | Does the agreement permit automated access and the fields you need? |
| Use scope | Is your personal, internal, or commercial use allowed? May you store, display, or redistribute the data? |
| Coverage | Does the source cover the relevant geography and MLS areas? |
| Completeness and attribution | Which fields are included, how are they maintained, and what broker or source attribution is required? |
| Freshness and integration | How often does the data update, and what effort is required to ingest and maintain it? |
For Compass-specific data, seek written authorization from the relevant rights holder before automated collection. Alternatively, find a provider whose documented license covers your intended use and geographic scope. The materials reviewed here do not establish a particular authorized Compass feed or provider, so verify the source and its terms directly.
5. Troubleshooting authorized Browse AI robots
| Symptom | Likely cause | What to do |
|---|---|---|
| Robot cannot reach or access the page | Login requirements, access controls, or site security | Confirm that you have permission and use an officially authorized access method. Do not evade a block or CAPTCHA. |
| Rows are missing from a list | Virtualized rows, pagination, lazy rendering, or changed page structure | Check whether the list is supported and whether the source offers a permitted export or feed. Recheck the robot after source changes. |
| Fields are blank or assigned to the wrong row | Training selection did not match the page structure, or dynamic content was not ready | On an authorized source, retrain the selection and inspect representative output rows before relying on the data. |
| Robot stops at a second-factor prompt | MFA/2FA is unsupported | Use a source-approved data access path that does not require unsupported automation. |
| CAPTCHA or bot check appears | The site is limiting or challenging automated access | Stop and confirm permission with the site. Do not attempt to bypass the control. |
| Previously working robot returns changed results | Source layout, labels, content, or access behavior changed | Review the source’s current terms and robot output; retrain only if continued collection is authorized. |
6. Screenshots for authorized documentation
If your permitted workflow also needs a screenshot of a page for documentation, visual QA, or a report, ScreenshotNeo is a website screenshot API and MCP server. A screenshot captures a visual page; it does not extract structured listing fields or grant permission to collect or reuse Compass content. Use it only with a URL and purpose you are authorized to access.
Or skip the browser setup
For an authorized page, ScreenshotNeo can return a screenshot in one request. See the ScreenshotNeo API documentation.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpimport requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);- Cookie banners are accepted and removed before capture; known consent platforms, newsletter popups, and chat widgets can be removed, and each step can be turned off.
- Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed; response headers report the page verdict and billing status.
- An MCP server provides
take_screenshot,get_page_info, andcapture_pdftools for AI agents and MCP clients. - The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 shots.
Sign up free for 1,000 screenshots a month, with no card required.
7. Cost, performance, and reliability considerations
For any permitted data source, start with only the fields and update frequency the project needs. More frequent runs can add maintenance and load while increasing the chance that changed pages produce incomplete records. Respect source rate limits and stop if access is denied or the site asks you to stop.
Validate records before using them: check for missing fields, duplicate listings, stale status or price values, inconsistent addresses, and attribution requirements. Keep source timestamps and provenance where your license allows. Do not treat a successful robot run as proof of completeness, accuracy, or permission.
The right cost comparison depends on license fees, coverage, freshness, integration work, and allowed use. Do not estimate Compass extraction costs or performance from Browse AI’s general capabilities; no Compass-specific run or benchmark is established here. For screenshots, ScreenshotNeo charges only for clean shots under the supplied product terms; its pricing is Free for 1,000 per month, Starter $5 for 3,000, Growth $15 for 15,000, Pro $39 for 60,000, Scale $99 for 250,000, and Business $249 for 1,000,000. Yearly billing gives two months free, and every feature is on every plan.
Frequently asked questions
Does Browse AI have documented instructions for scraping Compass?
The reviewed Browse AI materials describe general extraction workflows, not a Compass-specific setup. They do not show that a Compass robot is permitted or works.
Does a public listing page mean I can reuse its photos and descriptions?
No. Visibility does not itself grant reuse rights. Compass’s terms restrict use of its content, and photos and descriptions may carry additional copyright or license restrictions.
Can I use collected listings to train an AI model?
Compass’s terms prohibit using its services or content for AI training, validation, development, or related derivative datasets without express written authorization.
Is ScreenshotNeo a listing scraper?
No. It captures page images or PDFs. It does not turn listings into structured records or authorize access to their content.
