Use this when a vendor, bank, or government portal only offers data as a download behind a login. The session authenticates like a real user, triggers the export, and hands the file to your pipeline.
Executable starters
portal-download.ts
import { chromium } from 'playwright';const apiKey = process.env.BROWSERCITY_API_KEY;const portalUser = process.env.PORTAL_USER;const portalPassword = process.env.PORTAL_PASSWORD;if (!apiKey || !portalUser || !portalPassword) { throw new Error('Set BROWSERCITY_API_KEY, PORTAL_USER, and PORTAL_PASSWORD first.');}const { endpoint, token, id } = await fetch('https://api.browser.city/v1/sessions', { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ browser: 'chromium' }),}).then((r) => r.json());try { const browser = await chromium.connect(endpoint, { headers: { Authorization: `Bearer ${token}` }, }); const page = browser.contexts().at(0)!.pages().at(0)!; await page.goto('https://example.com/portal/login'); await page.fill('#username', portalUser); await page.fill('#password', portalPassword); await page.click('button[type="submit"]'); await page.waitForSelector('.dashboard'); await page.goto('https://example.com/portal/reports/monthly'); const downloadPromise = page.waitForEvent('download'); await page.click('#export-csv'); const download = await downloadPromise; await download.saveAs('./monthly-report.csv'); console.log(`Saved ${download.suggestedFilename()}`);} finally { await fetch(`https://api.browser.city/v1/sessions/${id}`, { method: 'DELETE', headers: { Authorization: `Bearer ${apiKey}` }, });}import osimport requestsfrom playwright.sync_api import sync_playwrightAPI_KEY = os.environ["BROWSERCITY_API_KEY"]PORTAL_USER = os.environ["PORTAL_USER"]PORTAL_PASSWORD = os.environ["PORTAL_PASSWORD"]auth = {"Authorization": f"Bearer {API_KEY}"}session = requests.post( "https://api.browser.city/v1/sessions", headers=auth, json={"browser": "chromium"}).json()try: with sync_playwright() as playwright: browser = playwright.chromium.connect( session["endpoint"], headers={"Authorization": f"Bearer {session['token']}"}, ) page = browser.contexts[0].pages[0] page.goto("https://example.com/portal/login") page.fill("#username", PORTAL_USER) page.fill("#password", PORTAL_PASSWORD) page.click('button[type="submit"]') page.wait_for_selector(".dashboard") page.goto("https://example.com/portal/reports/monthly") with page.expect_download() as download_info: page.click("#export-csv") download = download_info.value download.save_as("./monthly-report.csv") print(f"Saved {download.suggested_filename}")finally: requests.delete(f"https://api.browser.city/v1/sessions/{session['id']}", headers=auth)
Production hardening checklist
- Keep portal credentials in a secret manager and rotate them like any service credential.
- Verify the downloaded file (size, header row, checksum) before the pipeline consumes it.
- Handle the “already exported today” and “report not ready yet” portal states explicitly.
- Only automate portals where your account’s terms permit it; this template is for your own data.
Cost and plan notes
The session itself is short; the download’s size is the variable. A 50 MB export costs meaningfully more over residential egress than datacenter, so pick the cheapest egress the portal accepts and model the file size in /pricing-calculator.