2025, Oct 17 04:00

Why Selenium performance logs miss cross-origin iframe traffic and how to capture every domain with Playwright

Learn why Selenium performance logs ignore cross-origin iframe (OOPIF) network requests and how to capture all domains using Playwright for full coverage.

When you embed a Google Sheets document via an iframe and try to inventory every network request the page makes, it’s tempting to rely on Selenium’s performance logs. The surprise comes when the list of domains you collect contains only the top-level host, while Chrome DevTools clearly shows multiple requests to other origins. Below is a concise walkthrough of why this happens and how to collect a complete set of domains in this scenario.

Environment

ChromeDriver 138.0.7204.168, Chrome 138.0.7204.184, Selenium 4.34.2, Windows 11.

Reproducing the problem

A page embeds a published spreadsheet through an iframe:

<html>
  <head>
  </head>
  <body>
    <iframe src="https://docs.google.com/spreadsheets/d/e/2PACX-1vQaJwXqci0KQzDjzs8kvy--p80OZY1n30t4NWRh2qkU3pJqAdB4ZJEc79ohh4OkuifHWOHBRi0Z0yKS/pubhtml?gid=0&single=true&widget=true&headers=false"></iframe>
  </body>
</html>

A Selenium script attempts to capture every domain the page reaches out to using the “performance” log:

from urllib.parse import urlparse
import json
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
import time
opt = Options()
opt.set_capability('goog:loggingPrefs', {'performance': 'ALL'})
svc = Service(executable_path='C:\\Users\\qi\\PycharmProjects\\PythonLearn\\DomainCollector\\chromedriver.exe')
browser = webdriver.Chrome(service=svc, options=opt)
local_url = "file:///C:/Users/qi/Desktop/a.html"
browser.get(local_url)
time.sleep(5)
hosts = set()
perf_entries = browser.get_log('performance')
for evt in perf_entries:
    raw = evt['message']
    try:
        payload = json.loads(raw)
        evt_type = payload['message']['method']
        if evt_type == 'Network.requestWillBeSent':
            req_url = payload['message']['params']['request']['url']
            host = urlparse(req_url).netloc
            if host:
                hosts.add(host)
    except Exception:
        continue
print(f"domains:{hosts}")

The output lists only the top-level domain:

domains:{'docs.google.com'}

At the same time, the browser’s DevTools shows additional traffic to other Google origins and static asset hosts.

What’s really happening

ChromeDriver’s “performance” log does not reliably include network traffic originating from cross‑origin iframes. In modern Chrome, those are out‑of‑process iframes (OOPIF). Setting goog:loggingPrefs with performance=ALL affects the log level, not the scope of captured targets. Even when perfLoggingPrefs.enableNetwork is enabled, the performance log remains largely tied to the top‑level target and will miss network requests issued by cross‑origin frames. That’s why the script above sees docs.google.com but not the additional domains the iframe loads.

Working alternative that captures all requests

A practical way to capture all outgoing requests in this case is to switch to an approach that tracks network events across the entire page, including cross‑origin iframes. Using Playwright for this task, the following script collects all domains observed during page load:

import asyncio
from urllib.parse import urlparse
from playwright.async_api import async_playwright
hosts2 = set()
def on_request(req):
    h = urlparse(req.url).netloc
    if h:
        hosts2.add(h)
async def run():
    async with async_playwright() as pw:
        engine = await pw.chromium.launch(executable_path='..\\DomainCollector\\chrome-win64\\chrome.exe', headless=False)
        tab = await engine.new_page()
        tab.on("request", on_request)
        await tab.goto("file:///C:/Users/qi/Desktop/a.html")
        await tab.wait_for_load_state("networkidle")
asyncio.run(run())
print(f"domains:{hosts2}")

The result includes all observed hosts such as ssl.gstatic.com, fonts.gstatic.com, fonts.googleapis.com, and docs.google.com.

Why this matters

If you build monitoring, crawling, or security tooling that depends on enumerating external resources, missing cross‑origin iframe traffic leads to blind spots. Embedded content is ubiquitous, and large platforms often spread assets across multiple domains. Relying solely on top‑level network events can skew metrics, hide third‑party calls, and break policy enforcement based on origin analysis.

Takeaways

When your target page includes cross‑origin iframes, ChromeDriver’s performance log alone won’t give a complete view of the network. The mechanism is geared toward the primary target and does not consistently cover OOPIF traffic. For scenarios that require full capture, use tooling that observes requests across all frames on the page. In the example above, Playwright provides that coverage out of the box and returns the expected set of domains. Keep this in mind when designing data collection pipelines around embedded content to avoid underreporting and to maintain accurate visibility.

The article is based on a question from StackOverflow by tinyhare and an answer by tinyhare.

google-chrome iframe python selenium-webdriver