Data Sources & API Status

Everything feeding this dashboard — what's live, what's partial, and what we're still working on. All live sources refresh daily at 5am ET via Firebase Cloud Functions.

Live Sources
14
Partial
2
Planned
5
Total Entries
25,862
Refresh Architecture
Queues: Firebase Cloud Function refreshQueuesDaily runs at 5am ET. Scrapes all 7 ISOs + EIA in parallel (~2 min). Writes to Firestore queue_entries + computes iso_stats with byYear/byFuelType breakdowns.
News: Cloud Function scrapeNewsDaily runs at 6am ET. Scrapes 7 RSS feeds, filters by relevance, auto-tags with ISO/sentiment.
Infrastructure: HIFLD data seeded manually via npm run seed:infrastructure. Rarely changes.
Data Centers: Hardcoded in src/lib/data/data-centers.ts. Updated manually from public announcements.

Interconnection Queues(12)

PJM Queue API
9,251 entries
LIVE
Endpoint: https://services.pjm.com/PJMPlanningApi/api/Queue/GetFiltere...
Format: JSON (POST)
Auth: Public API key in header (api-subscription-key)
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapePJM()
Fields: queueNumber, projectName, state, county, fuelType, capacityMW, status, projectSubmittedDate, projectedInServiceDate, transmissionOwner
~9,200 entries. Most complete ISO API. Swagger docs available.
MISO GI Queue API
3,744 entries
LIVE
Endpoint: https://www.misoenergy.org/api/giqueue/getprojects
Format: JSON (GET)
Auth: None (public), browser-like User-Agent required
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapeMISO()
Fields: projectNumber, state, county, fuelType, summerNetMW, winterNetMW, applicationStatus, studyCycle, studyPhase, poiName
~3,700 entries. No project names exposed. studyCycle used for submission year (e.g., "DPP-2018-APR"). 403 errors if User-Agent is not browser-like.
SPP Queue CSV
2,833 entries
LIVE
Endpoint: https://opsportal.spp.org/Studies/GenerateSummaryCSV
Format: CSV
Auth: None (public)
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapeSPP()
Fields: GI Number, State, County, Fuel Type, Capacity, Status, Request Received, In-Service Date, Date Withdrawn
~2,800 entries. Clean CSV with header row. Also available: GenerateActiveCSV (active only).
ERCOT GIS Report (Excel)
1,874 entries
LIVE
Endpoint: https://www.ercot.com/misapp/servlets/IceDocListJsonWS?repor...
Format: Excel (.xlsx) via 2-step: list docs JSON → download Excel
Auth: None (public MIS endpoint)
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapeERCOT()
Fields: INR, Project Name, GIM Study Phase, County, Fuel, Technology, Capacity (MW), Projected COD, Screening Study Started
~1,800 entries across Large Gen + Small Gen sheets. 30 rows of notes/disclaimers before data. Header detection required. Published monthly by ERCOT.
CAISO Public Queue Report (Excel)
2,278 entries (dates through 2023)
PARTIAL
Endpoint: https://www.caiso.com/PublishedDocuments/PublicQueueReport.x...
Format: Excel (.xlsx)
Auth: None (public)
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapeCAISO()
Fields: Queue Position, Project Name, Queue Date, Application Status, Fuel-1/2/3, MW-1/2/3, Net MWs to Grid, County, State, POI
~2,300 entries across 3 sheets (Active, Completed, Withdrawn). Dates are Excel serial numbers. Queue dates only go through early 2023 — Cluster 15/16 entries may not be in the public report yet.
NYISO Interconnection Queue (Excel)
2,021 entries
LIVE
Endpoint: https://www.nyiso.com/documents/20142/1407078/NYISO-Intercon...
Format: Excel (.xlsx)
Auth: None (public)
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapeNYISO()
Fields: Queue Pos., Project Name, Date of IR, SP (MW), WP (MW), Type/Fuel, County, State, POI, IA Tender Date
~2,000 entries across 7 sheets (Queue, Cluster, Withdrawn, In Service, etc.). Column name "Queue Pos." has trailing period.
ISO-NE IRTT Queue (HTML)
1,751 entries (many missing dates)
PARTIAL
Endpoint: https://irtt.iso-ne.com/reports/external
Format: HTML table (ASP.NET)
Auth: None, but requires 2-step cookie handling (ASP.NET session)
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapeISONE()
Fields: QP, Fuel Type, Net MW, Summer MW, County, ST, POI, Project Status
~1,750 entries. Requires cookie-based fetch (AspxAutoDetectCookieSupport). Column "Requested" used for submission date but often empty. Status conveyed via <img> tags that must be converted to text.
EIA Form 860 (Planned Generators)
2,000 entries
LIVE
Endpoint: https://api.eia.gov/v2/electricity/facility-fuel/data/
Format: JSON (GET)
Auth: API key required (free registration)
Refresh: Daily at 5am ET via Cloud Function
Code: scripts/lib/iso-queue-scraper.ts → scrapeEIAPlannedGenerators()
Fields: plantCode, plantName, stateDescription, fuelTypeDescription, nameplate_capacity_mw, period
Supplements ISO queue data with EIA facility records. ~2,000 entries used. Excluded from year-over-year charts (not queue submissions). Mapped to ISOs via state.
LBNL Queues (Direct Download)
PLANNED
Endpoint: https://emp.lbl.gov/queues (Excel download)
Format: Excel
Auth: Returns 403 for automated requests (manual download only)
Refresh: Not automated
Fields: Aggregated queue data across all ISOs, standardized format
LBNL compiles all ISO queues into a single standardized dataset. Would be the gold standard but download is blocked for automated requests. Could use as annual validation.
Duke Energy Carolinas Queue
PLANNED
Endpoint: Unknown (utility-specific portal)
Format: Unknown
Auth: Unknown
Refresh: Not automated
Fields: Queue entries for largest non-ISO utility (50+ GW)
Duke Carolinas has the largest non-ISO queue in the country. No public API known. Would dramatically improve Southeast coverage.
Southern Company Queue
PLANNED
Endpoint: Unknown (OASIS)
Format: Unknown
Auth: Unknown
Refresh: Not automated
Fields: Queue entries for GA/AL/MS territory
Southern Company (Georgia Power, Alabama Power, Mississippi Power) does not publish queue data in a machine-readable format.
Western Non-ISO Utility Queues
PLANNED
Endpoint: Various (BPA, PacifiCorp, NV Energy, APS, etc.)
Format: Various
Auth: Various
Refresh: Not automated
Fields: Queue entries from individual Western utilities
Each Western utility manages its own queue. BPA, PacifiCorp (Berkshire), NV Energy, APS/SRP have separate portals. No centralized source.

Grid Infrastructure (HIFLD)(3)

HIFLD Transmission Lines
LIVE
Endpoint: https://services1.arcgis.com/Hp6G80Pky0om6HgA/arcgis/rest/se...
Format: GeoJSON via ArcGIS REST API
Auth: None (public HIFLD)
Refresh: Manual via npm run seed:infrastructure
Code: scripts/lib/hifld-scraper.ts → scrapeTransmissionLines()
Fields: voltage, owner, status, sub1, sub2, lengthMiles, coordinates
Homeland Infrastructure Foundation-Level Data. Covers all US transmission lines 69kV+. Layered on map by voltage class (115/230/345/500/765 kV).
HIFLD Substations
LIVE
Endpoint: https://services1.arcgis.com/Hp6G80Pky0om6HgA/arcgis/rest/se...
Format: GeoJSON via ArcGIS REST API
Auth: None (public HIFLD)
Refresh: Manual via npm run seed:infrastructure
Code: scripts/lib/hifld-scraper.ts → scrapeSubstations()
Fields: name, city, state, county, maxVoltage, owner, status, lat, lng
All US electrical substations with location and voltage data.
HIFLD Power Plants
LIVE
Endpoint: https://services1.arcgis.com/Hp6G80Pky0om6HgA/arcgis/rest/se...
Format: GeoJSON via ArcGIS REST API
Auth: None (public HIFLD)
Refresh: Manual via npm run seed:infrastructure
Code: scripts/lib/hifld-scraper.ts → scrapePowerPlants()
Fields: name, city, state, fuelType, totalMW, operator, status, lat, lng
All US power plants with capacity, fuel type, and location.

Data Centers(1)

Data Centers (Curated)
60+ facilities (grid + BTM)
LIVE
Endpoint: N/A (hardcoded from public sources)
Format: TypeScript array
Auth: N/A
Refresh: Manual updates from press releases, SEC filings, utility filings
Code: src/lib/data/data-centers.ts
Fields: name, operator, capacityMW, state, status, year, campus, connectionType, powerSource, regulatoryStatus
60+ facilities including hyperscale (AWS, Google, Meta, Microsoft), colocation (Equinix, Digital Realty, QTS), AI-focused (CoreWeave, Crusoe, Lambda), and behind-the-meter (Amazon/Susquehanna, Microsoft/TMI, Google/Kairos SMR). BTM facilities tagged with power source and FERC regulatory context.

News & Commentary(1)

RSS News Feeds
LIVE
Endpoint: Multiple RSS feeds (Utility Dive, EIA, Renewable Energy Worl...
Format: RSS/XML
Auth: None
Refresh: Daily at 6am ET via Cloud Function (scrapeNewsDaily)
Code: functions/src/index.ts → scrapeAndWriteNews()
Fields: title, source, url, publishedAt, summary, iso, tags, sentiment
7 RSS feeds scraped. Articles filtered by energy/grid relevance keywords. Auto-tagged with ISO, sentiment, and topic tags. Articles older than 90 days auto-deleted.

Reference & Curated Data(4)

LBNL Queues Report (2024)
LIVE
Endpoint: https://emp.lbl.gov/queues
Format: Manual download (Excel/PDF)
Auth: N/A (public report)
Refresh: Annual (manual update when new report published)
Code: src/lib/data/reference-data.ts
Fields: Per-ISO: totalProjects, totalCapacityGW, activeProjects, byFuelType, yearlySubmissions
Lawrence Berkeley National Lab annual interconnection queue analysis. Used as baseline reference for ISO scores, completion rates, withdrawal rates, and processing times. Fallback when Firestore unavailable.
Bottleneck Data
20+ locations
LIVE
Endpoint: N/A (curated from ISO filings, FERC dockets, industry report...
Format: TypeScript array
Auth: N/A
Refresh: Manual updates
Code: src/lib/data/bottleneck-data.ts
Fields: location, state, lat, lng, severity, pendingMW, pendingProjects, constraintType, description
20+ bottleneck locations across all 9 regions. Severity-coded (critical/high/medium). Rendered as map markers and sidebar cards.
Study Cluster Data
30+ clusters
LIVE
Endpoint: N/A (curated from ISO filings, stakeholder presentations, FE...
Format: TypeScript array
Auth: N/A
Refresh: Manual updates when new clusters announced
Code: src/lib/data/cluster-data.ts
Fields: name, iso, phase, totalProjects, activeProjects, totalCapacityMW, fuelMix, applicationWindow, studyTools, keyMilestones, withdrawalRate, avgCostPerMW
30+ study clusters across 7 ISOs. Phase tooltips explain the interconnection process. Cluster detail view shows timeline, fuel mix, costs.
FERC eLibrary (Docket Filings)
PLANNED
Endpoint: https://elibrary.ferc.gov/eLibrary/
Format: HTML / PDF
Auth: None (public)
Refresh: Not automated
Fields: FERC orders, interconnection agreements, queue reform filings
Could scrape for regulatory updates (Order 2023 compliance, BTM rulings). Would enhance the regulatory timeline feature.
Known Gaps & Limitations
  • CAISO dates: Public Excel only contains entries through early 2023. Cluster 15/16 entries (2023+) not yet in the report.
  • ISO-NE dates: "Requested" column is often empty — many entries lack submission dates.
  • MISO dates: API has no submission date field. Year inferred from studyCycle (e.g., "DPP-2018-APR").
  • Non-ISO queues: Southeast (Duke, Southern Co, TVA, FPL) and Western (BPA, PacifiCorp, NV Energy) utilities don't publish machine-readable queue data. Major blind spot.
  • Behind-the-meter: BTM data centers are curated manually from press releases. No API exists for this data.
  • Coordinates: Queue entry lat/lng are approximate (randomized within state/county center). Not exact project locations.
  • LBNL download: Returns 403 for automated requests. Annual report used as reference baseline only.
Last updated: March 2026. To add a source, update src/app/sources/page.tsx SOURCES array and add a "// SOURCE: see /sources page" comment in your scraper function.