.png&w=828&q=75)
birddog puts a leash on AI scraping agents. One context manager wraps an agent that hits the web and gives you: 1. Domain allowlist with wildcard subdomains — deny everything outside it 2. Per-domain rate caps via token bucket, one bucket per host 3. JSONL audit log with one line per fetch (url, status, bytes, ms) 4. Bright Data Web Unlocker proxy as an opt-in flag 5. Streamlit dashboard showing per-host bytes, denials, and p50 latency LLM agents don't know what a sane scraping cadence looks like. They'll hammer a site, follow links into spammy subdomains, and burn through a Bright Data quota in a single run. birddog stops that. Use it as a context manager: bd = Birddog( allowed_domains={"docs.brightdata.com", "*.example.com"}, per_domain_qps=1.0, audit_path="runs/scrape.jsonl", bright_data={"host": "brd.superproxy.io:33335", "username": "...", "password": "..."}, ) with bd.session("research-bot") as s: r = s.fetch("https://docs.brightdata.com/api") s.fetch("https://evil.example/exfil") # DomainDeniedError, logged The audit log is JSONL — one event per fetch, including domain_denied and rate_limited events. The bundled Streamlit dashboard reads the log and shows total fetches, denials, bytes, and a per-host breakdown with p50 latency. Built for research bots, price trackers, and RAG ingest jobs that hit live sites. Pairs with agentleash for USD budget caps on the same agent. Includes a Jupyter notebook walkthrough and two runnable example scripts. 10 tests, MIT license.
31 May 2026
.png&w=828&q=75)
GeminiLens is a small open-source library plus dashboard that gives Gemini agent builders the four numbers they care about during development: cost per call, p95 latency, output drift over time, and a full audit log of every external host an agent's tools reached. It wraps a Vertex AI Gemini client without forcing you onto a hosted backend. A Streamlit dashboard renders the traces locally, and a one-line exporter pushes them to Dynatrace or TrueFoundry for production monitoring. Why we built it: every Gemini agent project goes through the same loop. You ship the agent, then you spend a week wiring per-call cost, then another week building an "is the output getting worse" check, then a month wishing you had logged the URLs your tools actually hit. GeminiLens collapses those three things into one library plus one dashboard, so you can keep building the agent instead of building observability around it. Built on google-genai for Vertex AI Gemini 2.5 calls, httpx for the egress allowlist, Streamlit and pandas for the dashboard. The cost table is hand-curated from Google's published Gemini pricing and is checked into the repo so reviewers can see what we charge against. Math is pure stdlib, so nothing has to be mocked in tests. The repo ships with an example agent, a recorded 2-minute demo, a one-command Cloud Run deploy, and exporters for Dynatrace and TrueFoundry. Apache-2.0.
19 May 2026