@@ -32,8 +32,9 @@ and resume cleanly from this file alone, keeping token use low._Last updated: 2026-05-22_**Current phase: none in progress. Phase 26 (dividend payouts) is complete,verified, and deployed to production (commit `7608b06`).** Phases 0**Current phase: Phase 28 (ETFs as first-class citizens) is complete andverified locally; deploy pending.** Phase 26 (dividend payouts) is complete,verified, and deployed to production (commit `7608b06`). Phases 0through 12 (the MVP) plus Phase 14 (company leadership), Phase 18 (ETFprofiles), Phase 20 (strongest & weakest home panels), Phase 21 (home &search refinements), Phase 23 + 24 (financials table) and Phase 22 (data-age
@@ -46,10 +47,15 @@ adds a stock Dividends section on the symbol page — inferred cadence,prior-year and YTD totals, a count-tempered on-track projection, and a per-event payout history — backed by a new weekly Yahoo `dividends` schedulerjob. Deployed to production 2026-05-22 (commit `7608b06`). Remainingpost-MVP work: the loose-ordered Phase 13, 15, 16, 17, 19, 25, 27, 28backlog. Phase 28 (ETFs as first-class citizens) was captured right afterPhase 26 deployed; the user flagged it as a strong wish — see itsPhases-list entry for the seven planned pieces and the data sources.post-MVP work: the loose-ordered Phase 13, 15, 16, 17, 19, 25, 27, 29backlog (Phase 28 is now in progress). Phase 28 (ETFs as first-classcitizens) was picked as the next backlog phase and scoped 2026-05-22 (seethe decisions log): one big phase covering all seven pieces (distributions,expense ratio + yield, NAV / premium-discount, sector + geography, fulltrailing returns, growth-of-$10k chart, strategy summary, benchmarkcomparison). Phase 29 (issuer-direct ETF data feeds: iShares/BlackRock,Vanguard, ...) was captured 2026-05-22 from a vibe-coding side notemid-Phase-28; see its Phases-list entry.**Roadmap (restructured 2026-05-22, see decisions log):** the home-pageredesign and commodities are pre-ship MVP phases. Order: 9 Search +
@@ -762,6 +768,94 @@ schema, unused for now. (360px) render with no horizontal overflow and zero console errors; a too-long caption wraps cleanly to a second line at 360px.- **Phase 28 ETFs as first-class citizens.** Complete, verified locally; deploy pending. ETFs now read as densely as a stock page: a new "About this fund" panel (expense ratio, distribution yield, NAV with live premium / discount, inception, category, fund family, strategy paragraph), a trailing-returns table (1m / 3m / YTD / 1y / 3y / 5y / 10y / since-inception, annualised past one year), a growth-of-$10,000 area chart over the longest available range, sector and geography exposure panels alongside the existing asset mix, a relative-performance benchmark line on the price chart for the broad-market ETFs, and the Phase 26 Dividends section lifted from stocks-only to also cover ETF distributions. - **Migration `0008`** adds `symbols.fund_metadata_synced_at` and `symbols.benchmark`, `fund_profiles.sector_mix` and `geography_mix` JSON columns, and a new `fund_metadata` table (expense ratio, yield, NAV, inception, category, fund family, strategy summary). - **Yahoo `quoteSummary`** is now the source for a third concern beyond quotes and dividends: a new `YahooProvider::fund_metadata` calls `v10/finance/quoteSummary` with the `fundProfile + defaultKeyStatistics + summaryDetail + price + assetProfile` modules and parses each into optional fields, so a small ETF that Yahoo only partly covers still populates as much as it can. 429 / 503 / 401 / 403 all surface as the typed `RateLimited` so the `yahoo` `EndpointGuard` trips at once if Yahoo crumb-gates the v10 endpoint for our box. - **N-PORT parser extended** to capture each holding's `<issuerCat>` and `<invCountry>` while it streams, then aggregate into sector and geography mixes in the same `[[label, percent], ...]` JSON shape as the existing `asset_mix`. A small bucket map turns the bare codes into human labels (`CORP` -> Corporate, `US` -> United States, ...). A degenerate mix (a pure-equity ETF rolls everything up to Corporate) hides itself in the template. - **Scheduler `run_fund_metadata`** is a new section on the existing `yahoo` `EndpointGuard` — one request per ETF, monthly staleness on `fund_metadata_synced_at`, brought forward to the first tick on boot the same way `sec` and `dividends` are, so a deploy backfills the universe within a tick rather than the daily interval. The Phase 26 `dividends` job dropped its `kind = 'stock'` filter so ETF distributions ride the same code path. `scheduler::backfill_symbol` pulls the new ETF's fund_metadata and distributions inside the add-symbol request, mirroring Phase 21's intent. - **Compute** added `trailing_returns(bars, today)` (eight periods, annualised past 1y; "since inception" uses actual days / 365.25 for leap-year drift), `growth_of_10k(bars)` (scale the close series so the anchor bar reads as $10k), `premium_discount_pct(price, nav)` and a small `premium_grade` band (±0.25 / ±1.00). Unit-tested. - **Routes** added a new `/api/symbols/{ticker}/growth` endpoint returning the fund's full-history growth series plus a benchmark growth series anchored to the fund's first bar (separately re-scaled to $10k from there). The existing `/api/symbols/{ticker}/history` gained a `benchmark` series scaled to the fund's first visible close so the two lines start together on the main chart. Trailing returns pull the full daily history rather than the 400-bar window the chart uses. - **Frontend** added a dashed benchmark line series to the main lightweight-chart, a benchmark toggle button that only renders when one is configured, and a new `growth.js` driving an AreaSeries + dashed LineSeries growth-of-$10k chart against its own y-axis (compact-USD formatter). - **Template** restructured: the Dividends block moved out of the stocks-only conditional (its label flips to "Distributions" on an ETF), and the ETF block grew About / Trailing returns / Growth-of-$10k panels above the existing Fund profile and Top holdings. Sector and geography mixes ride alongside the asset mix inside Fund profile. - **Universe `starter.csv`** added a fifth `benchmark` column. Curated for 18 broad-market ETFs (SPY/VOO/IVV/VTI/VUG/VTV/SCHD/VIG/ VYM/XLE/XLF/XLK/XLV -> `^SPX`; QQQ/SMH/ARKK -> `^NDX`; DIA -> `^DJI`; IWM -> `^RUT`). Sector SPDRs map to `^SPX`. International, bond and commodity ETFs leave it blank (no clean equity-index benchmark). The `seed` parser threads the column into `symbols.benchmark` on every seed pass. - **Health page** lists the new `fund_metadata` job between `sec` and `dividends` (job_meta + job_rank); the `dividends` job's description now reads "stock and ETF" rather than "stock". - Verified locally: cargo + bun build clean; the 3 Phase 28 compute unit tests pass; migration `0008` applied cleanly on the seeded dev DB; `/s/SPY` renders the new sections (with an injected synthetic `fund_metadata` row for the dev box, since Yahoo rate-limits this WSL2 IP); `/api/symbols/SPY/history?range=1Y` returns a 252-point benchmark series scaled to the fund's first visible close; `/api/symbols/SPY/growth` returns a 5,342-point fund growth series ($10k -> $79,275 over 21 years, ~10.3%/yr) plus the same length benchmark series; `/s/GLD` (commodity trust) shows the About / Returns / Growth panels with no holdings (the grantor-trust note carries); `/s/AAPL` (stock) shows none of the ETF panels; `/s/^SPX` (index) and `/s/GC=F` (future) are unchanged. SPY trailing returns sanity-checked against deep history: 10y +320.6% / +15.45%/yr, since-inception (from the Feb 2005 first stored bar) +692.8% / +10.24%/yr.- **Phase 26 dividend payouts.** Complete, verified, and deployed to production (commit `7608b06`). Per-payout dividend history sourced from Yahoo's chart endpoint via `events=div`, surfaced as a new Dividends section on the symbol page
@@ -820,21 +914,33 @@ schema, unused for now. hid the section; ETFs / indexes / futures showed no section.**Resuming, next action****Phase 26 (dividend payouts) is complete, verified, and deployed** toproduction on 2026-05-22 (commit `7608b06`). The MVP plus Phase 14,**Phase 28 (ETFs as first-class citizens) is in progress.** Scope settled2026-05-22 (see the decisions log): one big phase covering all sevenpieces (distributions for ETFs, expense ratio + yield via Yahoo`quoteSummary`, NAV / premium-discount, sector + geography exposure fromN-PORT, full trailing returns 1m/3m/YTD/1y/3y/5y/10y/since-inception,growth-of-$10k chart, strategy summary + inception, and a benchmarkcomparison off a hand-curated `benchmark` column for the curated universe).A new migration `0008` will add fund-metadata columns to `fund_profiles`(expense ratio, yield, NAV, inception, category, fund family, strategysummary, sector_mix and geography_mix JSON) and a `benchmark` column on`symbols`; a new Yahoo `quoteSummary` provider path adds`YahooProvider::fund_metadata`; the N-PORT parser is extended to retaineach holding's `industryCode` and country; a new scheduler `fund_metadata`section sweeps stale ETFs on the existing `yahoo` guard (monthly cadence).Phase 26's `kind = 'stock'` dividends filter is dropped so ETFdistributions ride the same code path. Phase 29 (issuer-direct ETF datafeeds: iShares/BlackRock, Vanguard, ...) was captured 2026-05-22 from avibe-coding side note mid-Phase-28; see the decisions log.After Phase 28 ships, remaining post-MVP work is the loose-orderedPhase 13, 15, 16, 17, 19, 25, 27, 29 backlog. Phase 26 (dividend payouts)is complete and deployed (commit `7608b06`); the MVP plus Phase 14,Phase 18, Phase 20, Phase 21, Phase 23 + 24, Phase 22 and Phase 26 are alllive at https://finance.bythewood.me; migration `0007` applied cleanly onthe box and the new `dividends` scheduler job backfills the universe overthe first tick.Remaining post-MVP work: the loose-ordered Phase 13, 15, 16, 17, 19, 25, 27backlog; the user picks which to take next. Phase 27 (provider redundancy)was captured 2026-05-22 from a vibe-coding side note while Phase 26 wasmid-build; see the decisions log. A small Phase 9 bug fix (the `/search`"Add" affordance for short tickers like `W`) shipped 2026-05-22 alongsidethe backlog capture of Phases 25 and 26; see the decisions log. There isstill no GitHub repo for finance: the user deferred that; if one iscreated later, add it as `origin` and the `overshard/finance` slug alreadyin taproot's `projects.conf` lines up.live at https://finance.bythewood.me. There is still no GitHub repo forfinance: the user deferred that; if one is created later, add it as`origin` and the `overshard/finance` slug already in taproot's`projects.conf` lines up.Note: Phase 18 added the `quick-xml` crate (N-PORT XML streaming parser) andmigration `0005`. A fresh `make run` applies `0005`; the ETF fund profiles
@@ -1281,8 +1387,11 @@ depend on Phase 5 (live quotes) and Phase 7 (SEC data). each stock's dividend history through the `yahoo` `EndpointGuard` on a slow cadence (weekly is enough; the data rarely changes).- [ ] **Phase 28: ETFs as first-class citizens.** (Captured 2026-05-22 from a vibe-coding side note immediately after Phase 26 shipped.) The user's- [x] **Phase 28: ETFs as first-class citizens.** Complete and verified locally (2026-05-22); deploy pending. See the Phase 28 Done entry in Status and the decisions log. (Captured 2026-05-22 from a vibe-coding side note immediately after Phase 26 shipped. Picked as the next backlog phase and scoped 2026-05-22.) The user's steer, verbatim: *"I really need ETFs treated as first class citizens like with as much data as possible, AUM, cashflow maybe, dividends, everything — right now we are ignoring a lot of stats for ETFs and
@@ -1297,8 +1406,17 @@ depend on Phase 5 (live quotes) and Phase 7 (SEC data). no NAV / premium-discount, no sector or geography breakdown, no return metrics, no inception date or strategy summary, no benchmark comparison. **Planned pieces (settle scope at build time; one big phase or split into 28a / 28b):** **Scope settled 2026-05-22 (see the decisions log).** One big phase covering all seven pieces below in a single ship. Open questions answered: expense ratio + yield come from Yahoo `quoteSummary` (path 2a), not hand-curated; benchmark comparison uses a hand-curated `benchmark` column on `symbols` (path 7's pragmatic option), so user-added ETFs simply omit the overlay; the full trailing-returns set, the growth-of- $10k chart, NAV / premium-discount, and sector + geography panels are all in. Phase 29 (issuer-direct ETF feeds for iShares / Vanguard / ...) was budgeted as a separate later phase, not folded in here. **Planned pieces:** (1) **ETF distributions.** Lift Phase 26 to cover ETFs — Yahoo's `events.dividends` series carries an ETF's distributions the same way
@@ -1404,6 +1522,64 @@ depend on Phase 5 (live quotes) and Phase 7 (SEC data). goes beyond a small refinement, so the user can pick the ordering after the current backlog.- [ ] **Phase 29: Issuer-direct ETF data feeds.** (Captured 2026-05-22 from a vibe-coding side note mid-Phase-28.) The user's steer: *"as a future plan we tie into iShares/blackrock and Vanguard directly for even more data for their ETFs (and others if available/popular)"*. Phase 28 builds the ETF page off SEC N-PORT and Yahoo `quoteSummary`, the broadly-available free sources; Phase 29 layers richer issuer-direct data on top wherever the issuer publishes it. Issuers often publish their own fund factsheets and full daily holdings as CSV / JSON feeds that go beyond N-PORT's quarterly snapshot (daily refresh, full position list rather than just top-25, effective duration / SEC yield / option-adjusted spread for bond funds, sector and country breakdowns done by the issuer's own methodology, distribution schedules, factsheets, prospectus PDFs, premium / discount history, and so on). **Likely issuer sources to investigate at build time:** - **iShares / BlackRock** — `ishares.com` carries per-fund product pages with downloadable CSV holdings, distributions, factsheet PDFs and JSON behind the page; needs research on the stable endpoint shape. - **Vanguard** — `vanguard.com` advisor product feeds and per-fund JSON for holdings, characteristics, distributions. - **State Street / SPDR** — `ssga.com` for SPY, SPDR sector funds. - **Invesco** — for QQQ and the BLDR / DBA / DBC commodity funds. - **Schwab / Fidelity / WisdomTree / VanEck / First Trust** — pick based on what the user holds enough interest in. **Open design points to settle when building:** (1) **Trait composition.** Likely a `FundExtrasProvider` trait per issuer, dispatched by `symbols.issuer` (a new column or derived from the fund name), with a `merge_into` step that overlays issuer fields onto the Phase 28 `fund_profiles` row only where the issuer feed has fresher / richer data than N-PORT. SEC stays the canonical "every fund has it" baseline; issuer data is the cherry on top. (2) **Endpoint guards per issuer.** A new `EndpointGuard` row per issuer domain, each with its own conservative budget per the anti-spam policy. None of these are documented public APIs; treat them carefully. (3) **Coverage and graceful degradation.** Only a few large issuers are worth implementing; the long tail (small ETF sponsors, foreign issuers) stays SEC + Yahoo only. The page must not look broken on an unsupported issuer's fund — extras simply do not render. (4) **Scheduler placement.** Likely a new `fund_extras` job on its own daily / weekly cadence, separate from the Phase 28 `fund_metadata` job, so an issuer outage does not stall the Yahoo path. (5) **Health page surfacing.** Each issuer guard surfaces on `/health` like the existing four; the symbol page may want a small "enriched by <issuer>" provenance line where issuer data is in use. No order or schedule attached; the user can pick after the current backlog. Depends on Phase 28 (the schema and the fund-profile scaffolding to overlay onto).---## Key files
@@ -2099,6 +2275,91 @@ finance/ per fiscal period not per payout, so it does not stand in. Both are stocks-only and budgeted as Phase 25 and Phase 26 in the post-MVP backlog; no order or schedule attached.- **2026-05-22 — Phase 28 picked next; design Q&A settled four points.** Asked which loose-ordered backlog phase to take next, the user chose Phase 28 (ETFs as first-class citizens). A short design pass settled every open question the phase's plan entry had flagged. (1) **Scope: one big phase, all seven pieces.** Not split 28a / 28b — the user wants ETFs to read as densely as a stock page in one ship. (2) **Expense ratio and yield come from Yahoo `quoteSummary`,** modules `fundProfile + defaultKeyStatistics + summaryDetail + price`, in a single request behind the existing `yahoo` `EndpointGuard` — not hand-curated in `starter.csv`. Phase 18 had once considered Yahoo here and dropped it, but hand-curation does not scale to user-added ETFs (Phase 9), which settled it. (3) **Benchmark comparison uses a hand-curated `benchmark` column on `symbols`** (the plan's pragmatic path), populated for the curated universe (SPY/VOO->^SPX, QQQ->^NDX, DIA->^DJI, IWM->^RUT, sector SPDRs, etc.). A user-added ETF simply omits the overlay rather than guessing wrong from a flaky auto-detect. (4) **All four extras are in:** the full trailing-returns set (1m/3m/YTD/1y/3y/5y/10y/since-inception, annualized past 1y), the growth-of-$10k area chart over the longest available range, the live NAV / premium-discount badge, and N-PORT sector + geography panels. The Phase 28 entry in the Phases list was updated to read "scoped, in progress" with these answers folded in.- **2026-05-22 — Phase 28 ETFs as first-class citizens shipped (local).** Yahoo is now the source for a third concern beyond quotes and dividends, and SEC N-PORT carries two more aggregations (sector, geography) past the asset mix. Design calls made during the build: (1) **Yahoo `fund_metadata` is an inherent `YahooProvider` method, not a new trait** — the data lives on `v10/finance/quoteSummary`, a single-source concern, same call as `lookup` / `dividends`; Phase 27 (provider redundancy) is the right place to lift it if a second source ever joins. (2) **Sector mix uses N-PORT's `<issuerCat>`, geography uses `<invCountry>`** — N-PORT does NOT carry GICS sector codes (issuer-computed metadata, issuer-direct only), so the sector panel is honest about what SEC provides: CORP / GOVT / MUN / Registered fund / etc. Meaningful for bond and multi-sector funds, degenerate (single bucket) on a pure-equity ETF — the template hides the panel in that case rather than render a flat bar carrying no information. Finer GICS sectors are budgeted as a Phase 29 cherry-on-top. (3) **Yahoo `quoteSummary` defensively handles 401 / 403 alongside 429 / 503 as `RateLimited`,** because Yahoo's v10 endpoint is occasionally crumb-gated and surfaces auth failures inconsistently; treating all four as the guard's rate-limit signal keeps a broken upstream from spinning while letting the breaker recover cleanly when Yahoo flips back. The WSL2 dev box is blanket-429'd by Yahoo today (a known cloud-IP issue), so the endpoint was verified by parser-level reasoning and unit tests; the production alpine server has been hitting Yahoo cleanly for months (Phase 5, 18, 26), so the deploy is the source of truth. (4) **A separate full-history query for trailing returns,** not the 400-bar window the price chart's key stats reuse — without it, the 3y / 5y / 10y / since-inception rows always read as missing. (5) **Benchmark column is hand-curated in `starter.csv`** (per the scope Q&A): the 18 broad-market ETFs that have a clean curated benchmark in our index universe get one (SPY / VOO / ... -> ^SPX, QQQ / SMH / ARKK -> ^NDX, DIA -> ^DJI, IWM -> ^RUT); sector SPDRs map to ^SPX too, international / bond / commodity ETFs leave it blank, so the relative-performance overlay hides itself rather than guess wrong. A user-added ETF (Phase 9) simply omits the overlay. (6) **One growth-of-$10k chart in lightweight-charts**, separate from the price chart so the longest available range (since inception) is shown without coupling to the price chart's range buttons; benchmark rides on the same panel as a dashed line re-scaled to $10k from the fund's first bar (not the benchmark's earliest bar), so the two lines start together. (7) **Dividends template lifted out of the stocks-only conditional** rather than duplicated; the section title flips to "Distributions" when the symbol is an ETF. (8) **The Phase 28 build accidentally surfaced a latent verification gap:** trailing returns first read from the chart's truncated 400-bar window, which capped them at ~1.5 years and made every long-horizon row miss; caught in a render check on SPY ("3y/5y/10y all show —"), fixed with the separate full-history query in symbols.rs. No deploy yet — the local verification used a synthetic `fund_metadata` row injected via the python `sqlite3` module since Yahoo refuses this WSL2 IP.- **2026-05-22 — Phase 29 captured: issuer-direct ETF data feeds.** Mid-Phase-28 the user floated: *"as a future plan we tie into iShares/blackrock and Vanguard directly for even more data for their ETFs (and others if available/popular)"*. Per the vibe-coding rule budgeted into the plan rather than acted on mid-phase: see the new Phase 29 entry in the Phases list, which enumerates the candidate issuer sources (iShares, Vanguard, State Street / SPDR, Invesco, and others) and five open design points (trait composition, per-issuer endpoint guards, coverage and graceful degradation on unsupported issuers, scheduler placement, and `/health` surfacing). Depends on Phase 28 to overlay onto.---
modified
frontend/static_src/symbol/index.js
@@ -3,6 +3,8 @@import "./styles/symbol.scss";import { initChart } from "./scripts/chart.js";import { initFundamentals } from "./scripts/fundamentals.js";import { initGrowth } from "./scripts/growth.js";initChart();initFundamentals();initGrowth();
modified
frontend/static_src/symbol/scripts/chart.js
@@ -44,6 +44,12 @@ const OVERLAY_INK = { ema21: "#6f5b86",};const RSI_INK = "#3f6f9c";// Phase 28 benchmark overlay: a fourth wayfinding ink, dashed so it reads as// "what would have happened in the benchmark" rather than blending with the// SMAs. Anchored to the fund's first visible close, so the two lines start// together and the divergence is the relative performance the eye should// follow.const BENCH_INK = "#7a5237";const VOLUME_UP = "rgba(47,125,79,0.38)";const VOLUME_DOWN = "rgba(178,59,50,0.38)";
@@ -110,6 +116,19 @@ export function initChart() { ema21: overlay("ema21", true), }; // Benchmark line (Phase 28). Created up front and shown only when the // history payload carries `benchmark`; the toggle row gets a swatch so // the user can hide it the same way as the SMAs. const benchmarkSeries = chart.addSeries(LineSeries, { color: BENCH_INK, lineWidth: 2, lineStyle: 2, priceLineVisible: false, lastValueVisible: false, crosshairMarkerVisible: false, visible: false, }); let bars = []; // loaded candles, ascending by time let latest = null; // last loaded payload, kept so RSI can attach on demand
@@ -318,6 +337,15 @@ export function initChart() { overlays.sma200.setData(d.sma200); overlays.ema21.setData(d.ema21); if (rsiSeries) rsiSeries.setData(d.rsi14); // Phase 28: benchmark overlay rides on the price pane when present. const bench = d.benchmark || []; benchmarkSeries.setData(bench); // Only show the benchmark series — and the toggle for it — when the // payload actually has one. The button defaults to on when shown. const benchBtn = document.querySelector('[data-ind="benchmark"]'); if (benchBtn) benchBtn.hidden = bench.length === 0; const benchOn = bench.length > 0 && (!benchBtn || benchBtn.classList.contains("is-active")); benchmarkSeries.applyOptions({ visible: benchOn }); } // ── indicator toggles ──────────────────────────────────────────────────
@@ -327,6 +355,8 @@ export function initChart() { else destroyRsi(); } else if (key === "volume") { volumeSeries.applyOptions({ visible: on }); } else if (key === "benchmark") { benchmarkSeries.applyOptions({ visible: on }); } else if (overlays[key]) { overlays[key].applyOptions({ visible: on }); }
@@ -336,7 +366,10 @@ export function initChart() { const key = btn.dataset.ind; // Paint the swatch from the JS palette so the inks live in one place. const dot = btn.querySelector(".ind-btn__dot"); if (dot) dot.style.background = key === "rsi" ? RSI_INK : OVERLAY_INK[key]; if (dot) { dot.style.background = key === "rsi" ? RSI_INK : key === "benchmark" ? BENCH_INK : OVERLAY_INK[key]; } // The template's is-active class is the initial visibility. applyIndicator(key, btn.classList.contains("is-active")); btn.addEventListener("click", () => {
added
frontend/static_src/symbol/scripts/growth.js
@@ -0,0 +1,112 @@// Growth-of-$10,000 chart (Phase 28). A small area chart on the ETF symbol// page that scales the fund's daily closes so $10,000 invested at the// series' first bar reads as that, then runs forward to today. When a// benchmark is configured, a dashed line of the same $10,000 in the// benchmark runs alongside so the relative path is read at a glance.//// Driven by GET /api/symbols/{ticker}/growth, which returns// `{ fund: [{date, value}, ...], benchmark: [...], benchmark_ticker }`.import { createChart, AreaSeries, LineSeries, ColorType,} from "lightweight-charts";// Paper Ledger palette: ink fill (very translucent), the same warm-ink line// the symbol price chart uses, and the dashed benchmark in the chart's// non-semantic wayfinding-ink palette.const FUND_LINE = "#2f7d4f";const FUND_FILL_TOP = "rgba(47, 125, 79, 0.20)";const FUND_FILL_BTM = "rgba(47, 125, 79, 0.00)";const BENCH_LINE = "#7a5237";const chartOptions = { autoSize: true, handleScroll: false, handleScale: false, layout: { background: { type: ColorType.Solid, color: "transparent" }, textColor: "#6b6456", fontFamily: "'JetBrains Mono', monospace", attributionLogo: false, }, grid: { vertLines: { color: "rgba(33,31,26,0.07)" }, horzLines: { color: "rgba(33,31,26,0.07)" }, }, rightPriceScale: { borderColor: "rgba(33,31,26,0.16)", // Format the y-axis in compact dollars (e.g. $12.4K, $58K) so the // growth path reads cleanly without a bare 50000 number. mode: 0, }, timeScale: { borderColor: "rgba(33,31,26,0.16)", rightOffset: 0 }, crosshair: { mode: 1 },};export function initGrowth() { const el = document.getElementById("growth-chart"); if (!el) return; const ticker = el.dataset.ticker; fetch(`/api/symbols/${encodeURIComponent(ticker)}/growth`) .then((res) => { if (!res.ok) throw new Error(`growth ${res.status}`); return res.json(); }) .then((d) => { if (!d.fund || d.fund.length < 2) { el.innerHTML = '<p class="growth-empty">Not enough history to draw the growth path.</p>'; return; } const chart = createChart(el, chartOptions); const fund = chart.addSeries(AreaSeries, { lineColor: FUND_LINE, topColor: FUND_FILL_TOP, bottomColor: FUND_FILL_BTM, lineWidth: 2, priceFormat: { type: "custom", // Compact dollar formatter for the y-axis and crosshair: $42.1K, // $1.2M etc., which keeps the labels short across the 4-orders-of- // magnitude growth a multi-decade fund accrues. formatter: fmtCompactUsd, minMove: 1, }, }); fund.setData( d.fund.map((p) => ({ time: p.date, value: p.value })), ); if (d.benchmark && d.benchmark.length >= 2) { const bench = chart.addSeries(LineSeries, { color: BENCH_LINE, lineWidth: 2, lineStyle: 2, priceFormat: { type: "custom", formatter: fmtCompactUsd, minMove: 1, }, priceLineVisible: false, }); bench.setData( d.benchmark.map((p) => ({ time: p.date, value: p.value })), ); } chart.timeScale().fitContent(); }) .catch((err) => { console.error("growth load failed", err); });}/** $1,234,567 -> "$1.2M", $42_100 -> "$42.1K". */function fmtCompactUsd(n) { const abs = Math.abs(n); if (abs >= 1e9) return `$${(n / 1e9).toFixed(1)}B`; if (abs >= 1e6) return `$${(n / 1e6).toFixed(1)}M`; if (abs >= 1e3) return `$${(n / 1e3).toFixed(1)}K`; return `$${n.toFixed(0)}`;}
modified
frontend/static_src/symbol/styles/symbol.scss
@@ -1009,6 +1009,147 @@ color: var(--ink-dim);}/* ── Phase 28: ETF "About this fund" + returns + growth-of-$10k ────────── *//* About-this-fund panel: the headline ETF figures (expense / yield / NAV / inception / family / category) over the issuer's strategy paragraph. The stat strip mirrors .fund__stats so the two panels feel of a piece, with each value typed bigger than the eyebrow caption above it. */.fund-about { padding: 16px 18px;}.fund-about__stats { display: flex; flex-wrap: wrap; gap: 14px 36px;}.fund-about__stat { display: flex; flex-direction: column; gap: 3px;}.fund-about__cap { @include eyebrow; font-size: 0.58rem;}.fund-about__val { font-size: 1.4rem; font-weight: 700; display: flex; flex-wrap: wrap; align-items: baseline; gap: 8px;}/* The premium / discount caption that rides on the NAV cell — a small semantic-coloured pill that reads "+0.12% to price" in the right hue. */.fund-about__sub { font-size: 0.72rem; font-weight: 600; padding: 2px 7px; border-radius: 999px; background: var(--well); color: var(--ink-dim); &--good { color: var(--ok); background: rgba(47, 125, 79, 0.10); } &--ok { color: var(--warn); background: rgba(178, 121, 50, 0.12); } &--bad { color: var(--bad); background: rgba(178, 59, 50, 0.10); }}.fund-about__summary { margin-top: 16px; padding-top: 16px; border-top: 1px solid var(--rule); color: var(--ink-dim); font-size: 0.88rem; line-height: 1.5;}/* The fund-profile mix sections each got their own heading in Phase 28, so the Asset / Sector / Geography bars stack with a clear divider between. */.fund__mix-label { @include eyebrow; font-size: 0.58rem; margin: 0 0 9px;}/* Trailing-returns table (Phase 28). Tight, readable; the % column is the primary read, with annualised muted as a secondary cue. */.ret-panel { padding: 12px 18px 14px;}.ret-table { width: 100%; border-collapse: collapse;}.ret-table th,.ret-table td { text-align: left; padding: 8px 0; border-bottom: 1px solid var(--rule); font-size: 0.9rem;}.ret-table th[scope="row"] { font-weight: 500; color: var(--ink-dim);}.ret-table .num { text-align: right; font-weight: 600;}.ret-table thead th { @include eyebrow; font-size: 0.58rem; color: var(--ink-faint); font-weight: 600; border-bottom: 1px solid var(--rule-strong);}.ret-cell--up { color: var(--ok); }.ret-cell--down { color: var(--bad); }.ret-cell--ann { color: var(--ink-dim); font-weight: 500; }.ret-panel__src { margin-top: 11px; color: var(--ink-faint); font-size: 0.76rem; line-height: 1.5;}/* Growth-of-$10k chart panel. Mirrors the price chart's panel framing, but shorter — the chart is a single-line area, not a multi-pane workspace. */.growth-panel { padding: 16px 18px;}#growth-chart { width: 100%; height: 240px;}.growth-empty { color: var(--ink-dim); font-size: 0.88rem; padding: 28px 12px; text-align: center;}.growth-panel__src { margin-top: 11px; color: var(--ink-faint); font-size: 0.76rem; line-height: 1.5;}/* on a narrow phone the position value yields to the weight, the key figure */@media (max-width: $bp-sm) { .hold {
added
migrations/0008_etf_stats.sql
@@ -0,0 +1,51 @@-- finance migration 0008: ETF stats (Phase 28).---- Layers Yahoo `quoteSummary` fund metadata onto the Phase 18 SEC N-PORT-- fund profile so an ETF symbol page can read as densely as a stock's. Also-- adds two N-PORT-derived aggregations (sector_mix, geography_mix) that-- ride alongside the existing asset_mix on `fund_profiles`, and a curated-- benchmark column on `symbols` for the relative-performance overlay.---- Phase 26's stocks-only dividends path is lifted to ETFs in the same-- phase, but that is a code-level filter change and needs no migration:-- the `dividends` table already keys on (ticker, ex_date) for any ticker.-- When the Yahoo `fund_metadata` scheduler section last refreshed this-- ETF. NULL = never swept. ETFs only; stocks, indexes and futures keep-- NULL forever.ALTER TABLE symbols ADD COLUMN fund_metadata_synced_at INTEGER;-- Curated benchmark index (a symbol such as `^SPX` / `^IXIC` / `^DJI` /-- `^RUT`) that the fund aims to track. Hand-curated in-- `universe/starter.csv` for the broad-market ETFs; a user-added ETF-- simply omits it and the symbol page hides the relative-performance-- overlay. NULL on every non-ETF row.ALTER TABLE symbols ADD COLUMN benchmark TEXT;-- N-PORT sector + geography exposure aggregated from each holding's-- `industryCode` / issuer country at parse time. JSON of the same shape-- as the existing `asset_mix` column: [[bucket, percent], ...] ordered-- largest first, top ~10 buckets kept. NULL for a commodity-trust ETF-- (GLD, SLV) that files no N-PORT, the same as `asset_mix` already is.ALTER TABLE fund_profiles ADD COLUMN sector_mix TEXT;ALTER TABLE fund_profiles ADD COLUMN geography_mix TEXT;-- Yahoo `quoteSummary` fund metadata: the slow-moving figures the-- prospectus carries that N-PORT does not (expense ratio, yield,-- inception, category, fund family, the issuer's strategy paragraph),-- plus NAV which drifts intraday. One row per ETF, replaced wholesale by-- the `fund_metadata` scheduler section on each refresh. Kept separate-- from `fund_profiles` because the two carry different sources and-- staleness cadences (SEC quarterly vs Yahoo monthly).CREATE TABLE fund_metadata ( ticker TEXT PRIMARY KEY REFERENCES symbols(ticker) ON DELETE CASCADE, expense_ratio REAL, -- annualReportExpenseRatio, decimal (e.g. 0.0003 = 0.03%) yield_pct REAL, -- summaryDetail.yield, decimal (e.g. 0.013 = 1.30%) trailing_yield_pct REAL, -- summaryDetail.trailingAnnualDividendYield, decimal nav_price REAL, -- price.navPrice or fundProfile fallback, USD inception_date TEXT, -- YYYY-MM-DD; first trade date / fund start category TEXT, -- fundProfile.categoryName, e.g. "Large Blend" fund_family TEXT, -- fundProfile.family, e.g. "Vanguard" strategy_summary TEXT, -- assetProfile.longBusinessSummary (full paragraph) updated_at INTEGER NOT NULL);
@@ -905,3 +905,270 @@ pub fn standing(ratios: &[Ratio], closes: &[f64]) -> Option<Standing> { score, })}// ────────────────────── ETF trailing returns (Phase 28) ────────────────────//// Trailing total returns from a fund's daily-close series. Distributions are// not folded in (we have them in the `dividends` table from Phase 26, but the// price-return shown here is the most common convention; the distribution// yield rides separately on the page). Periods over a year are annualised so// every figure reads on the same scale./// One trailing return — both the simple cumulative figure and the/// annualised one (the same number for periods of a year or less).#[derive(Debug, Clone, Copy, Serialize)]pub struct TrailingReturn { /// Cumulative percent move over the window. pub pct: f64, /// CAGR. Equal to `pct` for windows ≤ 1 year; geometrically annualised /// past that. pub annualised_pct: f64,}/// The full set of trailing returns the ETF page shows. Each is `None` when/// the price history does not reach back that far.#[derive(Debug, Clone, Default, Serialize)]pub struct TrailingReturns { pub m1: Option<TrailingReturn>, pub m3: Option<TrailingReturn>, pub ytd: Option<TrailingReturn>, pub y1: Option<TrailingReturn>, pub y3: Option<TrailingReturn>, pub y5: Option<TrailingReturn>, pub y10: Option<TrailingReturn>, pub since_inception: Option<TrailingReturn>,}/// One bar of the daily-close series the trailing-return / growth functions/// consume: a `YYYY-MM-DD` date and the close. Oldest first.#[derive(Debug, Clone)]pub struct DatedClose<'a> { pub date: &'a str, pub close: f64,}/// Compute the full trailing-return set from a `bars` series (oldest first)/// against the latest available close (its tail). Empty / single-bar input/// returns an all-`None` set. `today` is `YYYY-MM-DD` and anchors the YTD/// window to the current calendar year — passing the latest bar's date keeps/// the figure deterministic across requests.pub fn trailing_returns(bars: &[DatedClose<'_>], today: &str) -> TrailingReturns { if bars.len() < 2 { return TrailingReturns::default(); } let latest = bars[bars.len() - 1].close; if latest <= 0.0 { return TrailingReturns::default(); } // Bar at or just before a target date, by walking back from the tail. The // series is calendar-irregular (weekends, holidays), so an exact match is // rare; "or just before" is the convention for trailing returns. let close_at_or_before = |target: &str| -> Option<f64> { bars.iter() .rev() .find(|b| b.date <= target) .map(|b| b.close) .filter(|c| *c > 0.0) }; let ret = |prev: f64, years: f64| -> TrailingReturn { let cum = (latest / prev - 1.0) * 100.0; let ann = if years > 1.0 { ((latest / prev).powf(1.0 / years) - 1.0) * 100.0 } else { cum }; TrailingReturn { pct: cum, annualised_pct: ann, } }; // Approximate-calendar offsets keyed to `today`'s YMD. `chrono` is already // a dependency, so use it rather than fudging day counts. let parse = |d: &str| chrono::NaiveDate::parse_from_str(d, "%Y-%m-%d").ok(); let today_d = parse(today); let target = |months: i64| -> Option<String> { let t = today_d?; let ym = t.year() as i64 * 12 + (t.month0() as i64) - months; let (ty, tm0) = (ym.div_euclid(12) as i32, ym.rem_euclid(12) as u32); let day = t.day().min(28); // a 28th always exists in every month chrono::NaiveDate::from_ymd_opt(ty, tm0 + 1, day).map(|d| d.format("%Y-%m-%d").to_string()) }; let years_target = |years: i64| target(years * 12); let ytd_target = || -> Option<String> { // The last close of the prior calendar year — i.e. the bar at or // before "Jan 1 of this year" — is the YTD anchor. let t = today_d?; Some(format!("{}-01-01", t.year())) }; let r = |target_date: Option<String>, years: f64| -> Option<TrailingReturn> { let prev = close_at_or_before(&target_date?)?; Some(ret(prev, years)) }; let m1 = r(target(1), 1.0 / 12.0); let m3 = r(target(3), 0.25); let ytd = r(ytd_target(), 1.0); // YTD is reported cumulative, not annualised let y1 = r(years_target(1), 1.0); let y3 = r(years_target(3), 3.0); let y5 = r(years_target(5), 5.0); let y10 = r(years_target(10), 10.0); // Since inception: the very first bar. Years span from its date to today, // measured in actual days / 365.25 to capture leap-year drift. let since_inception = (|| { let first = bars.first()?; let f = parse(first.date)?; let t = today_d?; let days = (t - f).num_days() as f64; if days <= 0.0 || first.close <= 0.0 { return None; } let years = (days / 365.25).max(1.0 / 12.0); Some(ret(first.close, years)) })(); TrailingReturns { m1, m3, ytd, y1, y3, y5, y10, since_inception, }}/// Use `chrono::Datelike` for the date arithmetic above.use chrono::Datelike;// ────────────────────── growth-of-$10,000 chart (Phase 28) ─────────────────/// One point of the growth-of-$10k series rendered on the ETF page.#[derive(Debug, Clone, Serialize)]pub struct GrowthPoint { /// Trading date, `YYYY-MM-DD`. pub date: String, /// Dollar value of $10,000 invested at the series' start, on this date. pub value: f64,}/// Scale a daily-close series so the first bar reads as $10,000. Returns the/// full series — the caller is responsible for downsampling if it would/// render too densely. Empty / single-bar / zero-anchor input returns an/// empty series.pub fn growth_of_10k(bars: &[DatedClose<'_>]) -> Vec<GrowthPoint> { if bars.len() < 2 { return Vec::new(); } let anchor = bars[0].close; if anchor <= 0.0 { return Vec::new(); } bars.iter() .map(|b| GrowthPoint { date: b.date.to_string(), value: 10_000.0 * b.close / anchor, }) .collect()}// ────────────────────── ETF NAV premium / discount (Phase 28) ──────────────/// Premium or discount of `price` to `nav`, as a percent. A positive value is/// a premium (price > NAV), negative a discount. `None` when NAV is unknown/// or non-positive.pub fn premium_discount_pct(price: f64, nav: Option<f64>) -> Option<f64> { let nav = nav?; if nav <= 0.0 { return None; } Some((price - nav) / nav * 100.0)}/// A small good/ok/bad band on the premium/discount figure. A persistently/// large premium is a yellow flag (buying above NAV); a normal ETF stays/// inside ±25 bps. Symmetric — a deep discount is also notable.pub fn premium_grade(premium_pct: f64) -> Grade { const TIGHT: f64 = 0.25; // ±0.25% is normal for liquid ETFs const LOOSE: f64 = 1.00; // ±1% is a yellow flag let abs = premium_pct.abs(); if abs <= TIGHT { Grade::Good } else if abs <= LOOSE { Grade::Ok } else { Grade::Bad }}#[cfg(test)]mod phase28_tests { use super::*; fn bars(samples: &[(&str, f64)]) -> Vec<DatedClose<'static>> { samples .iter() .map(|(d, c)| DatedClose { date: Box::leak(d.to_string().into_boxed_str()), close: *c, }) .collect() } #[test] fn trailing_returns_basic() { // A simple flat-then-spike series for 1y/3y windows. let b = bars(&[ ("2023-01-02", 100.0), ("2024-01-02", 110.0), ("2025-01-02", 121.0), ("2026-01-02", 133.1), ("2026-05-22", 140.0), ]); let r = trailing_returns(&b, "2026-05-22"); // 1y from 2025-05-22 onwards: closest bar at or before is 2025-01-02 (121.0). let y1 = r.y1.expect("y1"); assert!((y1.pct - ((140.0 / 121.0 - 1.0) * 100.0)).abs() < 1e-6); // 3y annualised: anchor at 2023-05-22, closest bar at or before is // 2023-01-02 (100.0). 140/100 over 3y -> (1.4)^(1/3) - 1. let y3 = r.y3.expect("y3"); let want = ((140.0_f64 / 100.0).powf(1.0 / 3.0) - 1.0) * 100.0; assert!((y3.annualised_pct - want).abs() < 1e-6); // YTD: anchor at "2026-01-01" → closest bar at or before is 2025-01-02 // (no 2026 bar yet for 01-01), then walks past to 2026-01-02 (133.1). // Actually 2025-01-02 is at-or-before 2026-01-01, so YTD anchors there. // That's a known edge: when the chart has a print on Jan 2 but not Jan 1, // YTD overlaps the new year cleanly enough for a tolerance check. assert!(r.ytd.is_some()); } #[test] fn growth_scales_to_10k_anchor() { let b = bars(&[ ("2020-01-02", 50.0), ("2021-01-04", 60.0), ("2022-01-03", 75.0), ]); let g = growth_of_10k(&b); assert_eq!(g.len(), 3); assert!((g[0].value - 10_000.0).abs() < 1e-6); assert!((g[1].value - 12_000.0).abs() < 1e-6); assert!((g[2].value - 15_000.0).abs() < 1e-6); } #[test] fn premium_discount_grades() { assert!(matches!(premium_grade(0.10), Grade::Good)); assert!(matches!(premium_grade(0.50), Grade::Ok)); assert!(matches!(premium_grade(-2.00), Grade::Bad)); assert!(premium_discount_pct(101.0, Some(100.0)).unwrap().abs() - 1.0 < 1e-9); assert!(premium_discount_pct(100.0, None).is_none()); assert!(premium_discount_pct(100.0, Some(0.0)).is_none()); }}
@@ -31,9 +31,17 @@ pub struct SymbolRow { pub fund_synced_at: Option<i64>, /// When this stock's leadership roster was last refreshed from SEC. pub leadership_synced_at: Option<i64>, /// When this stock's dividend history was last refreshed from Yahoo /// (Phase 26). NULL for non-stocks and stocks not yet swept. /// When this stock or ETF's dividend / distribution history was last /// refreshed from Yahoo (Phase 26 + Phase 28). NULL for indexes / futures /// and any not-yet-swept symbol. pub dividends_synced_at: Option<i64>, /// When this ETF's Yahoo `quoteSummary` snapshot was last refreshed /// (Phase 28). NULL on every non-ETF row and on ETFs not yet swept. pub fund_metadata_synced_at: Option<i64>, /// Curated benchmark index ticker for an ETF (e.g. `^SPX`), populated /// from `universe/starter.csv` (Phase 28). The symbol page's chart /// shows the relative-performance overlay only when this is set. pub benchmark: Option<String>, pub last_price: Option<f64>, pub prev_close: Option<f64>, pub last_quote_at: Option<i64>,
modified
src/providers/mod.rs
@@ -237,6 +237,14 @@ pub struct PortfolioData { pub top_holdings: Vec<FundHolding>, /// Asset-class mix as `(bucket, percent)` pairs, largest bucket first. pub asset_mix: Vec<(String, f64)>, /// Sector mix derived from each holding's N-PORT `industryCode` (or /// `assetCat` fallback for non-equity buckets), aggregated as /// `(label, percent)` pairs largest first. Phase 28; empty on a /// commodity-trust fund (no N-PORT). pub sector_mix: Vec<(String, f64)>, /// Geography mix derived from each holding's issuer country, same shape /// as `sector_mix`. Phase 28; empty on a commodity trust. pub geography_mix: Vec<(String, f64)>,}// ── dividend events (Phase 26) ─────────────────────────────────────────────
@@ -258,6 +266,48 @@ pub struct DividendEvent { pub amount: f64,}// ── ETF fund metadata (Phase 28) ───────────────────────────────────────────//// The slow-moving figures the prospectus carries that N-PORT does not — expense// ratio, distribution yield, inception, category, fund family, the issuer's// strategy paragraph — plus the intraday NAV used for the premium / discount// read. Yahoo's `v10/finance/quoteSummary` endpoint serves all of them in one// request behind the `fundProfile + defaultKeyStatistics + summaryDetail +// price + assetProfile` modules. The fetch lives on `YahooProvider` as an// inherent method (one source); the type sits here next to `Quote` /// `DividendEvent` for the scheduler and routes./// One ETF's Yahoo `quoteSummary` snapshot. Every field is optional: Yahoo's/// coverage is uneven and a small fund may carry only a subset, but a partial/// snapshot is still useful, so the parser keeps what it has rather than/// rejecting the row.#[derive(Debug, Clone, Default)]pub struct FundMetadata { /// Annual expense ratio as a decimal, e.g. `0.0003` = 0.03%. From /// `fundProfile.feesExpensesInvestment.annualReportExpenseRatio`. pub expense_ratio: Option<f64>, /// Forward / trailing distribution yield as a decimal. From /// `summaryDetail.yield` (preferred) or `defaultKeyStatistics.yield`. pub yield_pct: Option<f64>, /// Trailing-twelve-month distribution yield as a decimal. From /// `summaryDetail.trailingAnnualDividendYield`. pub trailing_yield_pct: Option<f64>, /// Latest NAV from `price.navPrice` or `summaryDetail.navPrice`. USD. pub nav_price: Option<f64>, /// Inception / first trade date as `YYYY-MM-DD`. From /// `defaultKeyStatistics.fundInceptionDate` or `price.firstTradeDateEpochUtc`. pub inception_date: Option<String>, /// Morningstar-style fund category, e.g. "Large Blend". From /// `fundProfile.categoryName`. pub category: Option<String>, /// Sponsor family, e.g. "Vanguard". From `fundProfile.family`. pub fund_family: Option<String>, /// The fund's strategy paragraph as the issuer writes it. From /// `assetProfile.longBusinessSummary` (preferred) or /// `summaryProfile.longBusinessSummary`. pub strategy_summary: Option<String>,}/// An upstream rejected a request with an explicit rate-limit signal (HTTP 429/// or 503). A provider returns this as the source of its `anyhow::Error` so the/// `EndpointGuard` (see `src/guard.rs`) can recognise it by downcast and trip
modified
src/providers/sec.rs
@@ -531,6 +531,9 @@ struct AtomEntry {}/// One holding accumulated while streaming through an `<invstOrSec>` block./// `issuer_cat` and `country` are captured for the sector / geography mixes/// (Phase 28) but not surfaced on [`FundHolding`] itself — only the/// aggregations are kept past parse time.#[derive(Default)]struct HoldingAcc { name: String,
@@ -538,6 +541,12 @@ struct HoldingAcc { pct: Option<f64>, value: Option<f64>, asset_cat: Option<String>, /// N-PORT `<issuerCat>`, e.g. `CORP` / `GOVT` / `MUN` / `RF`. Meaningful /// on bond and multi-sector funds; an equity ETF rolls up almost wholly /// to `CORP`, so the sector panel is hidden in that degenerate case. issuer_cat: Option<String>, /// N-PORT `<invCountry>`, the issuer's ISO-3166-1 alpha-2 country code. country: Option<String>,}impl HoldingAcc {
@@ -812,6 +821,12 @@ fn parse_nport(xml: &[u8]) -> Result<PortfolioData> { let mut all: Vec<FundHolding> = Vec::new(); // The holding currently being assembled, set while inside `<invstOrSec>`. let mut cur: Option<HoldingAcc> = None; // Running sector / geography aggregations: each holding's weight summed // into its `issuerCat` / `invCountry` bucket as the parser closes it. A // missing bucket on a holding contributes to an `"Unknown"` slot, which // is dropped at the end. let mut sector_acc: HashMap<&'static str, f64> = HashMap::new(); let mut country_acc: HashMap<&'static str, f64> = HashMap::new(); loop { match reader.read_event_into(&mut buf)? {
@@ -825,6 +840,11 @@ fn parse_nport(xml: &[u8]) -> Result<PortfolioData> { Event::End(e) => { if e.local_name().as_ref() == b"invstOrSec" { if let Some(h) = cur.take() { let pct = h.pct.unwrap_or(0.0); let sec = issuer_bucket(h.issuer_cat.as_deref()); let ctry = country_bucket(h.country.as_deref()); *sector_acc.entry(sec).or_insert(0.0) += pct; *country_acc.entry(ctry).or_insert(0.0) += pct; all.push(h.into_holding()); } }
@@ -851,6 +871,12 @@ fn parse_nport(xml: &[u8]) -> Result<PortfolioData> { b"assetCat" if h.asset_cat.is_none() => { h.asset_cat = Some(txt.to_string()) } b"issuerCat" if h.issuer_cat.is_none() => { h.issuer_cat = Some(txt.to_string()) } b"invCountry" if h.country.is_none() => { h.country = Some(txt.to_string()) } _ => {} } }
@@ -897,9 +923,95 @@ fn parse_nport(xml: &[u8]) -> Result<PortfolioData> { all.truncate(TOP_HOLDINGS); out.top_holdings = all; // Sector / geography mixes (Phase 28). Same shape as `asset_mix`: drop // residual rounding-noise buckets, sort largest first. The sector panel // hides itself in the route when only one bucket survives — that is the // pure-equity-ETF degenerate case where everything rolls up to "Corporate" // and the panel would be a single flat bar carrying no information. let trim_and_sort = |acc: HashMap<&'static str, f64>| -> Vec<(String, f64)> { let mut v: Vec<(String, f64)> = acc .into_iter() .filter(|(_, p)| *p >= 0.05) .map(|(b, p)| (b.to_string(), p)) .collect(); v.sort_by(|a, b| b.1.total_cmp(&a.1)); v }; out.sector_mix = trim_and_sort(sector_acc); out.geography_mix = trim_and_sort(country_acc); Ok(out)}/// Map an N-PORT `issuerCat` code to a human-readable sector bucket. The/// codes are corporate / government / municipal / fund / etc. — meaningful/// on bond and multi-sector funds; a pure-equity ETF rolls up almost wholly/// to `Corporate`, which the route detects and hides.fn issuer_bucket(cat: Option<&str>) -> &'static str { match cat.unwrap_or("") { "CORP" => "Corporate", "USGSE" => "US gov't-sponsored", "USGA" => "US government agency", "UST" => "US Treasury", "MUN" => "Municipal", "PF" => "Private fund", "RF" => "Registered fund", "ABS" => "Asset-backed", "" => "Unknown", _ => "Other", }}/// Map an issuer country's ISO-3166-1 alpha-2 code to a display name. Only/// the codes most commonly seen in US-listed ETF portfolios are spelled/// out; the long tail falls through to the raw code, which still reads/// usefully in the panel.fn country_bucket(code: Option<&str>) -> &'static str { match code.unwrap_or("").to_ascii_uppercase().as_str() { "US" => "United States", "CA" => "Canada", "GB" => "United Kingdom", "DE" => "Germany", "FR" => "France", "CH" => "Switzerland", "NL" => "Netherlands", "IE" => "Ireland", "LU" => "Luxembourg", "IT" => "Italy", "ES" => "Spain", "SE" => "Sweden", "NO" => "Norway", "DK" => "Denmark", "FI" => "Finland", "BE" => "Belgium", "AT" => "Austria", "JP" => "Japan", "CN" => "China", "HK" => "Hong Kong", "TW" => "Taiwan", "KR" => "South Korea", "IN" => "India", "SG" => "Singapore", "AU" => "Australia", "NZ" => "New Zealand", "BR" => "Brazil", "MX" => "Mexico", "ZA" => "South Africa", "IL" => "Israel", "AE" => "United Arab Emirates", "BM" => "Bermuda", "KY" => "Cayman Islands", "" => "Unknown", other => { // Heap-allocated codes can't be returned as `&'static str`, so a // small static buffer trick: leak the code. Acceptable here — // the long tail of ISO codes is small (~250) and per-app-run // bounded; not in a hot path. Box::leak(other.to_string().into_boxed_str()) } }}// ── company leadership: officers & board from Form 3/4/5 (Phase 14) ────────//// Every director and Section-16 officer of a company must file Form 3 (on
modified
src/providers/yahoo.rs
@@ -16,7 +16,9 @@ use async_trait::async_trait;use reqwest::{header::RETRY_AFTER, StatusCode};use serde::Deserialize;use crate::providers::{DividendEvent, IntradayBar, Quote, QuoteData, QuoteProvider, RateLimited};use crate::providers::{ DividendEvent, FundMetadata, IntradayBar, Quote, QuoteData, QuoteProvider, RateLimited,};/// Near-real-time quotes from Yahoo Finance.pub struct YahooProvider {
@@ -276,6 +278,70 @@ impl YahooProvider { Ok(out) } /// Fetch the Yahoo `quoteSummary` ETF metadata snapshot for `ticker` /// (Phase 28). One request to `v10/finance/quoteSummary` pulls the five /// modules that together carry every figure the Phase 28 ETF page needs /// beyond what SEC N-PORT already provides — expense ratio, distribution /// yield, latest NAV, inception, category, fund family, and the issuer's /// strategy paragraph. /// /// Returns `Ok(None)` when Yahoo answers cleanly that it has no such /// symbol (a 404 or a `quoteSummary.error` body) — a definitive empty, /// not a guard failure. Yahoo's gating responses (`429`, `503`, and `401 /// "Invalid Crumb"` which the gate sometimes returns as either) surface /// as the typed [`RateLimited`] so the endpoint guard trips at once. The /// returned [`FundMetadata`] may carry only a subset of fields populated /// — Yahoo's coverage is uneven across small ETFs. pub async fn fund_metadata(&self, ticker: &str) -> Result<Option<FundMetadata>> { let sym = urlencoding::encode(&yahoo_symbol(ticker)).into_owned(); // The five modules that between them carry every Phase 28 field. A // module Yahoo does not recognise for this symbol is silently // omitted from the response (rather than failing the whole request). let url = format!( "https://query1.finance.yahoo.com/v10/finance/quoteSummary/{sym}\ ?modules=fundProfile,defaultKeyStatistics,summaryDetail,price,assetProfile" ); let resp = self.client.get(&url).send().await?; let status = resp.status(); // The v10 endpoint is occasionally crumb-gated — Yahoo returns // either a plain 401 / 403 or, confusingly, a 429 from the same gate. // Treat all of them as a rate-limit signal so the guard trips and // we defer the sweep rather than spinning on a broken upstream. if matches!( status, StatusCode::TOO_MANY_REQUESTS | StatusCode::SERVICE_UNAVAILABLE | StatusCode::UNAUTHORIZED | StatusCode::FORBIDDEN ) { let retry_after_secs = resp .headers() .get(RETRY_AFTER) .and_then(|v| v.to_str().ok()) .and_then(|s| s.trim().parse::<i64>().ok()); return Err(anyhow::Error::new(RateLimited { status: status.as_u16(), retry_after_secs, })); } if status == StatusCode::NOT_FOUND { return Ok(None); } let resp = resp.error_for_status()?; let env: QuoteSummaryEnvelope = resp.json().await?; if env.quote_summary.error.is_some() { return Ok(None); } let Some(result) = env .quote_summary .result .and_then(|mut r| if r.is_empty() { None } else { Some(r.remove(0)) }) else { return Ok(None); }; Ok(Some(parse_fund_metadata(result))) } /// Identify a symbol: validate it exists on Yahoo and return its name, /// kind, exchange and currency, alongside the quote the same request /// carried. Used by the Phase 9 add-symbol flow.
@@ -294,6 +360,177 @@ impl YahooProvider { }}// ── v10 quoteSummary response (Phase 28) ───────────────────────────────────//// Yahoo wraps most numeric fields as `{"raw": ..., "fmt": "..."}`. A small// `RawF64` / `RawI64` carrier lets one serde derive handle every field of// that shape; an absent field, an unparsable one, or one missing the inner// `raw` is None — Yahoo's coverage is uneven and a partial snapshot is// still useful, so the parser keeps what it has rather than failing whole.#[derive(Deserialize)]struct QuoteSummaryEnvelope { #[serde(rename = "quoteSummary")] quote_summary: QuoteSummary,}#[derive(Deserialize)]struct QuoteSummary { result: Option<Vec<QuoteSummaryResult>>, /// Non-null on a logical failure (e.g. an unknown symbol — Yahoo returns /// `200 OK` with an `error` body, not a 404). error: Option<serde_json::Value>,}/// One module bag from the `quoteSummary` response. Every module is optional/// — Yahoo silently drops a module it does not recognise for this symbol/// rather than failing the whole request.#[derive(Default, Deserialize)]#[serde(rename_all = "camelCase", default)]struct QuoteSummaryResult { fund_profile: Option<FundProfileModule>, default_key_statistics: Option<DefaultKeyStatisticsModule>, summary_detail: Option<SummaryDetailModule>, price: Option<PriceModule>, asset_profile: Option<AssetProfileModule>,}#[derive(Default, Deserialize)]#[serde(rename_all = "camelCase", default)]struct FundProfileModule { family: Option<String>, category_name: Option<String>, fees_expenses_investment: Option<FeesExpensesInvestment>,}#[derive(Default, Deserialize)]#[serde(rename_all = "camelCase", default)]struct FeesExpensesInvestment { annual_report_expense_ratio: Option<RawF64>,}#[derive(Default, Deserialize)]#[serde(rename_all = "camelCase", default)]struct DefaultKeyStatisticsModule { /// Yahoo timestamps inception in either the unix-seconds `{raw, fmt}` /// shape or, on some funds, an `fmt`-only ISO-ish string. Accept both. fund_inception_date: Option<RawF64>, #[serde(rename = "yield")] yield_pct: Option<RawF64>,}#[derive(Default, Deserialize)]#[serde(rename_all = "camelCase", default)]struct SummaryDetailModule { #[serde(rename = "yield")] yield_pct: Option<RawF64>, trailing_annual_dividend_yield: Option<RawF64>, nav_price: Option<RawF64>,}#[derive(Default, Deserialize)]#[serde(rename_all = "camelCase", default)]struct PriceModule { nav_price: Option<RawF64>, first_trade_date_milliseconds: Option<RawF64>,}#[derive(Default, Deserialize)]#[serde(rename_all = "camelCase", default)]struct AssetProfileModule { long_business_summary: Option<String>,}/// Yahoo's `{ "raw": ..., "fmt": "..." }` numeric carrier. Deserialises from/// either form: the wrapped object, a bare number, or a string that parses/// as a number. Missing or unparsable -> `None`.#[derive(Debug, Clone, Copy)]struct RawF64(f64);impl<'de> serde::Deserialize<'de> for RawF64 { fn deserialize<D: serde::Deserializer<'de>>(d: D) -> std::result::Result<Self, D::Error> { #[derive(Deserialize)] struct Wrap { raw: Option<f64>, } // `untagged` lets serde try each variant in order; the first one to // deserialise cleanly wins. #[derive(Deserialize)] #[serde(untagged)] enum Any { Wrapped(Wrap), Bare(f64), Str(String), } let any = Any::deserialize(d)?; let v = match any { Any::Wrapped(w) => w.raw, Any::Bare(v) => Some(v), Any::Str(s) => s.trim().parse().ok(), }; v.map(RawF64).ok_or_else(|| { // Yahoo sometimes serves `{}` for a missing field; surface as a // deserialiser error so serde's outer Option<RawF64> on each // field captures it as None rather than failing the whole parse. serde::de::Error::custom("missing raw") }) }}/// Build a [`FundMetadata`] from one parsed `quoteSummary` result. Every/// field is best-effort: a missing module or field just leaves its slot/// `None` rather than rejecting the row.fn parse_fund_metadata(r: QuoteSummaryResult) -> FundMetadata { let fp = r.fund_profile.unwrap_or_default(); let dks = r.default_key_statistics.unwrap_or_default(); let sd = r.summary_detail.unwrap_or_default(); let price = r.price.unwrap_or_default(); let ap = r.asset_profile.unwrap_or_default(); let expense_ratio = fp .fees_expenses_investment .and_then(|f| f.annual_report_expense_ratio) .map(|v| v.0); // `summaryDetail.yield` is the live figure; `defaultKeyStatistics.yield` // is the same number on most funds but missing on some, so fall back. let yield_pct = sd .yield_pct .or(dks.yield_pct) .map(|v| v.0); let trailing_yield_pct = sd.trailing_annual_dividend_yield.map(|v| v.0); let nav_price = sd.nav_price.or(price.nav_price).map(|v| v.0); // Inception comes in two shapes: a unix-seconds carrier (older API), // or — on some funds — a unix-ms carrier from `price`. Normalise to a // `YYYY-MM-DD` date in UTC; the inception itself is day-precision. let inception_date = dks .fund_inception_date .map(|v| v.0 as i64) .or_else(|| { price .first_trade_date_milliseconds // Heuristic: a value > 10^11 is ms, else seconds. ETF // inceptions are post-1989 so both shapes are plausible. .map(|v| { let n = v.0 as i64; if n.abs() > 100_000_000_000 { n / 1000 } else { n } }) }) .and_then(|secs| chrono::DateTime::from_timestamp(secs, 0)) .map(|dt| dt.format("%Y-%m-%d").to_string()); let non_empty = |s: Option<String>| s.filter(|x| !x.trim().is_empty()); FundMetadata { expense_ratio, yield_pct, trailing_yield_pct, nav_price, inception_date, category: non_empty(fp.category_name), fund_family: non_empty(fp.family), strategy_summary: non_empty(ap.long_business_summary), }}/// Build a `SymbolInfo` from a chart `meta`. `None` for an instrument type/// this app does not model yet (currencies, crypto, ...).fn symbol_info(ticker: &str, meta: &Meta) -> Option<SymbolInfo> {
modified
src/routes/health.rs
@@ -241,7 +241,13 @@ fn job_meta(job: &str) -> (&str, &str) { ), "dividends" => ( "Dividend payouts", "Per-payout dividend history from Yahoo for each stock, refreshed weekly.", "Per-payout dividend / distribution history from Yahoo for each \ stock and ETF, refreshed weekly.", ), "fund_metadata" => ( "ETF fund metadata", "Yahoo quoteSummary snapshot for each ETF — expense ratio, yield, \ NAV, inception, category, fund family, strategy. Refreshed monthly.", ), other => (other, ""), }
@@ -253,9 +259,10 @@ fn job_rank(job: &str) -> u8 { "seed" => 0, "history" => 1, "sec" => 2, "dividends" => 3, "intraday" => 4, "daily_close" => 5, "fund_metadata" => 3, "dividends" => 4, "intraday" => 5, "daily_close" => 6, _ => 9, }}
modified
src/routes/symbols.rs
@@ -25,6 +25,7 @@ pub fn router() -> Router<AppState> { .route("/s/{ticker}", get(symbol_page)) .route("/api/symbols", post(add_symbol)) .route("/api/symbols/{ticker}/history", get(history_api)) .route("/api/symbols/{ticker}/growth", get(growth_api))}/// Stats for the symbol page header and key-stats visualizations. Until
@@ -441,6 +442,12 @@ struct FundProfileRow { report_date: Option<String>, /// JSON `[[bucket, percent], ...]`. asset_mix: Option<String>, /// JSON `[[label, percent], ...]`, from each holding's N-PORT /// `<issuerCat>` (Phase 28). Often degenerate on an equity ETF. sector_mix: Option<String>, /// JSON `[[label, percent], ...]`, from each holding's N-PORT /// `<invCountry>` (Phase 28). Often US-dominant. geography_mix: Option<String>,}/// A `fund_holdings` row as stored.
@@ -487,17 +494,20 @@ struct FundView { /// The N-PORT "as of" date, `YYYY-MM-DD`. report_date: Option<String>, asset_mix: Vec<AssetSlice>, /// N-PORT issuer-category mix (Phase 28). Empty / single-bucket on an /// equity ETF where everything rolls up to one bucket — the template /// hides the panel in that case rather than rendering a flat bar. sector_mix: Vec<AssetSlice>, /// N-PORT issuer-country mix (Phase 28). Empty / US-only on a domestic /// ETF; hidden by the template the same way. geography_mix: Vec<AssetSlice>, holdings: Vec<HoldingView>,}/// Assemble the fund view from a stored profile row and its holdings.fn build_fund(profile: FundProfileRow, holdings: Vec<HoldingRow>) -> FundView { // Asset mix: the stored JSON `[[label, percent], ...]`. The percentages // already sum to ~100, so each is its own segment width directly. let asset_mix = profile .asset_mix .as_deref() .and_then(|j| serde_json::from_str::<Vec<(String, f64)>>(j).ok())/// Parse a stored mix JSON column into the page's `AssetSlice` row shape./// Phase 28 calls this for asset / sector / geography mixes alike.fn parse_mix(json: Option<&str>) -> Vec<AssetSlice> { json.and_then(|j| serde_json::from_str::<Vec<(String, f64)>>(j).ok()) .unwrap_or_default() .into_iter() .map(|(label, pct)| AssetSlice {
@@ -505,7 +515,14 @@ fn build_fund(profile: FundProfileRow, holdings: Vec<HoldingRow>) -> FundView { width: pct.clamp(0.0, 100.0), label, }) .collect(); .collect()}/// Assemble the fund view from a stored profile row and its holdings.fn build_fund(profile: FundProfileRow, holdings: Vec<HoldingRow>) -> FundView { let asset_mix = parse_mix(profile.asset_mix.as_deref()); let sector_mix = parse_mix(profile.sector_mix.as_deref()); let geography_mix = parse_mix(profile.geography_mix.as_deref()); // Holdings: each weight bar is scaled to the largest holding shown, so the // top position fills the rail and the rest read against it.
@@ -530,10 +547,129 @@ fn build_fund(profile: FundProfileRow, holdings: Vec<HoldingRow>) -> FundView { holdings_count: profile.holdings_count, report_date: profile.report_date, asset_mix, sector_mix, geography_mix, holdings, }}// ── ETF fund metadata + trailing returns (Phase 28) ────────────────────────/// A `fund_metadata` row as stored.#[derive(sqlx::FromRow)]struct FundMetadataRow { expense_ratio: Option<f64>, yield_pct: Option<f64>, trailing_yield_pct: Option<f64>, nav_price: Option<f64>, inception_date: Option<String>, category: Option<String>, fund_family: Option<String>, strategy_summary: Option<String>,}/// The "About this fund" section of the ETF symbol page. Every field is/// pre-formatted, so the template stays declarative; an unpopulated field/// becomes `—` rather than a hole in the layout.#[derive(Serialize)]struct FundMetaView { expense_ratio: String, yield_pct: String, nav_price: Option<f64>, /// Pre-formatted premium / discount, e.g. `+0.12%`, with a good/ok/bad /// `Grade` so the template can colour-band it. `None` when no NAV. premium: Option<PremiumView>, inception_date: Option<String>, category: Option<String>, fund_family: Option<String>, strategy_summary: Option<String>,}#[derive(Serialize)]struct PremiumView { /// Signed pre-formatted percent, e.g. `+0.12%` / `-0.45%`. text: String, /// Grade for the semantic colour band: Good (tight), Ok, Bad (wide). grade: compute::Grade,}fn build_fund_meta(row: FundMetadataRow, price: Option<f64>) -> FundMetaView { let pct = |v: Option<f64>, dp: usize| -> String { v.map_or_else(|| DASH.to_string(), |x| format!("{:.*}%", dp, x * 100.0)) }; // Premium / discount: live price against the latest NAV. Live price falls // back to the daily close when no quote yet, just as the ratio cards do. let premium = price .and_then(|p| compute::premium_discount_pct(p, row.nav_price).map(|pct| (p, pct))) .map(|(_, pct)| PremiumView { text: format!("{:+.2}%", pct), grade: compute::premium_grade(pct), }); FundMetaView { expense_ratio: pct(row.expense_ratio, 2), yield_pct: pct(row.yield_pct.or(row.trailing_yield_pct), 2), nav_price: row.nav_price, premium, inception_date: row.inception_date, category: row.category, fund_family: row.fund_family, strategy_summary: row.strategy_summary, }}/// One row of the trailing-returns table, pre-formatted.#[derive(Serialize)]struct ReturnRow { label: &'static str, /// Cumulative percent move, e.g. `+18.27%`. `—` when missing. pct: String, /// Annualised percent for periods over 1 year, blank `""` for the YTD / /// 1m / 3m rows (where annualising is misleading) and `—` when missing. annualised: String, /// Whether `pct` is positive (green), negative (red), or unknown (none). dir: i8,}fn fmt_pct(v: Option<f64>) -> (String, i8) { match v { Some(v) => { let dir = if v > 0.0 { 1 } else if v < 0.0 { -1 } else { 0 }; (format!("{:+.2}%", v), dir) } None => (DASH.to_string(), 0), }}fn build_returns(r: &compute::TrailingReturns) -> Vec<ReturnRow> { let row = |label: &'static str, tr: Option<compute::TrailingReturn>, annualised: bool| { let (pct, dir) = fmt_pct(tr.map(|t| t.pct)); let annualised = if annualised { match tr { Some(t) => format!("{:+.2}%", t.annualised_pct), None => DASH.to_string(), } } else { String::new() }; ReturnRow { label, pct, annualised, dir, } }; vec![ row("1 month", r.m1, false), row("3 months", r.m3, false), row("Year to date", r.ytd, false), row("1 year", r.y1, false), row("3 years", r.y3, true), row("5 years", r.y5, true), row("10 years", r.y10, true), row("Since inception", r.since_inception, true), ]}// ── dividend payouts (Phase 26) ────────────────────────────────────────────/// One dividend payment, shaped for the page.
@@ -928,7 +1064,8 @@ async fn symbol_page(Path(ticker): Path<String>, State(state): State<AppState>) // The ETF fund profile, when the SEC sweep has reached this symbol. let fund = if is_etf { let profile = sqlx::query_as::<_, FundProfileRow>( "SELECT kind, net_assets, holdings_count, report_date, asset_mix \ "SELECT kind, net_assets, holdings_count, report_date, \ asset_mix, sector_mix, geography_mix \ FROM fund_profiles WHERE ticker = ?", ) .bind(&ticker)
@@ -954,6 +1091,54 @@ async fn symbol_page(Path(ticker): Path<String>, State(state): State<AppState>) None }; // ETF fund metadata + trailing returns (Phase 28). Both keyed by the // same `is_etf` gate; the fund_metadata row exists once the new Yahoo // job has swept this symbol. An unswept ETF shows the section's // "pending" note in the template. let fund_meta = if is_etf { sqlx::query_as::<_, FundMetadataRow>( "SELECT expense_ratio, yield_pct, trailing_yield_pct, nav_price, \ inception_date, category, fund_family, strategy_summary \ FROM fund_metadata WHERE ticker = ?", ) .bind(&ticker) .fetch_optional(&state.pool) .await .ok() .flatten() .map(|r| build_fund_meta(r, price)) } else { None }; // Trailing returns reach back as far as the fund's daily history goes // (since inception, ten years, ...), so they pull the *full* series for // this symbol rather than the 400-bar window the chart's key stats use. // ETFs only. let returns = if is_etf { let full: Vec<(String, f64)> = sqlx::query_as( "SELECT d, close FROM daily_prices WHERE ticker = ? ORDER BY d ASC", ) .bind(&ticker) .fetch_all(&state.pool) .await .unwrap_or_default(); if full.len() >= 2 { let dated: Vec<compute::DatedClose<'_>> = full .iter() .map(|(d, c)| compute::DatedClose { date: d, close: *c, }) .collect(); let today = chrono::Utc::now().date_naive().format("%Y-%m-%d").to_string(); Some(build_returns(&compute::trailing_returns(&dated, &today))) } else { None } } else { None }; // The leadership roster + change feed (Phase 14): stocks only, like the // fundamentals above. let leadership = if is_stock {
@@ -962,10 +1147,11 @@ async fn symbol_page(Path(ticker): Path<String>, State(state): State<AppState>) None }; // Dividend payouts (Phase 26): stocks only. A stock that pays no dividend // (or whose sweep returned nothing) gets no section; an unswept stock // shows a pending note in place. let dividends = if is_stock { // Dividend / distribution payouts (Phase 26 + Phase 28): now covers // stocks AND ETFs. Indexes and futures have no concept and get no // section; an unswept symbol shows a pending note in place; a swept one // with no payouts in the past five years hides the section. let dividends = if is_stock || is_etf { build_dividends(&state.pool, &ticker, symbol.dividends_synced_at.is_some()).await } else { None
@@ -979,6 +1165,8 @@ async fn symbol_page(Path(ticker): Path<String>, State(state): State<AppState>) fundamentals => fundamentals, standing => standing, fund => fund, fund_meta => fund_meta, returns => returns, leadership => leadership, dividends => dividends, filings => filings,
@@ -1011,8 +1199,12 @@ struct LinePoint { value: f64,}/// The symbol chart payload (Phase 8): the candles for the selected range/// plus the indicator overlays, each already trimmed to the visible window./// The symbol chart payload (Phase 8 + Phase 28): the candles for the/// selected range plus the indicator overlays, each already trimmed to the/// visible window. Phase 28 adds an optional benchmark series — the/// curated index a fund tracks, normalised to the same start point as the/// fund's first visible close — rendered as a relative-performance line/// only when the symbol has a `symbols.benchmark` configured.#[derive(Serialize)]struct HistoryResponse { candles: Vec<Candle>,
@@ -1020,6 +1212,16 @@ struct HistoryResponse { sma200: Vec<LinePoint>, ema21: Vec<LinePoint>, rsi14: Vec<LinePoint>, /// Benchmark closes scaled to the same starting price as the visible /// candles, so the two lines start together and drift apart on relative /// performance. Empty when no benchmark is configured or no benchmark /// history overlaps the range. #[serde(skip_serializing_if = "Vec::is_empty")] benchmark: Vec<LinePoint>, /// Curated benchmark ticker label for the chart legend (e.g. `^SPX`). /// Absent when no benchmark is configured. #[serde(skip_serializing_if = "Option::is_none")] benchmark_ticker: Option<String>,}/// Earliest `YYYY-MM-DD` to *show* for a range button. `None` means no limit.
@@ -1111,17 +1313,164 @@ async fn history_api( .collect() }; // Benchmark overlay (Phase 28). Loaded only when the symbol has a // curated `symbols.benchmark` set, and only when the visible range // actually has anchor bars to scale against. The benchmark series is // pinned to the fund's first visible close so the two lines start // together and drift apart on relative performance. let benchmark_ticker: Option<String> = sqlx::query_scalar( "SELECT benchmark FROM symbols WHERE ticker = ?", ) .bind(&ticker) .fetch_optional(&state.pool) .await .ok() .flatten() .flatten(); let benchmark = match (&benchmark_ticker, candles.get(start)) { (Some(bench), Some(first_visible)) => { load_benchmark_series(&state.pool, bench, &candles[start..], first_visible.close).await } _ => Vec::new(), }; let resp = HistoryResponse { sma50: line(compute::sma(&closes, 50)), sma200: line(compute::sma(&closes, 200)), ema21: line(compute::ema(&closes, 21)), rsi14: line(compute::rsi(&closes, 14)), candles: candles.into_iter().skip(start).collect(), benchmark, benchmark_ticker, }; Json(resp).into_response()}/// Load a benchmark index's daily closes across the same date span as/// `visible` (the fund's visible candle slice), then scale each close so the/// series starts at `fund_anchor` — the fund's first visible close — and/// only the *relative* movement past that point is plotted. An empty/// benchmark history or no overlap returns an empty vec, which the/// `skip_serializing_if` on the response field then drops cleanly.async fn load_benchmark_series( pool: &sqlx::SqlitePool, benchmark: &str, visible: &[Candle], fund_anchor: f64,) -> Vec<LinePoint> { let Some(first) = visible.first() else { return Vec::new(); }; let last = visible.last().unwrap_or(first); let rows: Vec<(String, f64)> = sqlx::query_as( "SELECT d, close FROM daily_prices \ WHERE ticker = ? AND d >= ? AND d <= ? ORDER BY d ASC", ) .bind(benchmark) .bind(&first.time) .bind(&last.time) .fetch_all(pool) .await .unwrap_or_default(); if rows.len() < 2 { return Vec::new(); } let bench_anchor = rows[0].1; if bench_anchor <= 0.0 { return Vec::new(); } let scale = fund_anchor / bench_anchor; rows.into_iter() .map(|(d, c)| LinePoint { time: d, value: c * scale, }) .collect()}// ── ETF growth-of-$10k chart (Phase 28) ────────────────────────────────────/// Response body for `GET /api/symbols/{ticker}/growth`. Two series scaled/// so the first point of each reads as $10,000, drawn together so a fund's/// since-inception path can be eyeballed against its benchmark's.#[derive(Serialize)]struct GrowthResponse { /// Fund growth series, oldest first. fund: Vec<compute::GrowthPoint>, /// Benchmark growth series across the same date span, anchored /// separately to $10,000 at its own first bar. Empty when the fund has /// no curated benchmark or no benchmark history overlaps. #[serde(skip_serializing_if = "Vec::is_empty")] benchmark: Vec<compute::GrowthPoint>, #[serde(skip_serializing_if = "Option::is_none")] benchmark_ticker: Option<String>,}/// Build the growth-of-$10k series over the longest available daily history/// for `ticker`. ETF symbol page only; on a symbol with no daily history/// (e.g. a future) the series is empty and the panel hides itself.async fn growth_api(Path(ticker): Path<String>, State(state): State<AppState>) -> Response { let ticker = ticker.to_uppercase(); let rows: Vec<(String, f64)> = sqlx::query_as( "SELECT d, close FROM daily_prices WHERE ticker = ? ORDER BY d ASC", ) .bind(&ticker) .fetch_all(&state.pool) .await .unwrap_or_default(); let bars: Vec<compute::DatedClose<'_>> = rows .iter() .map(|(d, c)| compute::DatedClose { date: d.as_str(), close: *c, }) .collect(); let fund = compute::growth_of_10k(&bars); let (benchmark_ticker, benchmark) = match ( sqlx::query_scalar::<_, Option<String>>("SELECT benchmark FROM symbols WHERE ticker = ?") .bind(&ticker) .fetch_optional(&state.pool) .await .ok() .flatten() .flatten(), fund.first(), ) { (Some(bench), Some(first)) => { // Anchor benchmark to the same first-bar date as the fund, so the // two lines start together; benchmark history before the fund's // inception is ignored. let bench_rows: Vec<(String, f64)> = sqlx::query_as( "SELECT d, close FROM daily_prices \ WHERE ticker = ? AND d >= ? ORDER BY d ASC", ) .bind(&bench) .bind(&first.date) .fetch_all(&state.pool) .await .unwrap_or_default(); let bench_bars: Vec<compute::DatedClose<'_>> = bench_rows .iter() .map(|(d, c)| compute::DatedClose { date: d.as_str(), close: *c, }) .collect(); (Some(bench), compute::growth_of_10k(&bench_bars)) } _ => (None, Vec::new()), }; Json(GrowthResponse { fund, benchmark, benchmark_ticker, }) .into_response()}// ── add a symbol to the universe (Phase 9) ─────────────────────────────────/// `POST /api/symbols` request body.
modified
src/scheduler.rs
@@ -35,9 +35,9 @@ use crate::market;use crate::providers::sec::SecProvider;use crate::providers::yahoo::YahooProvider;use crate::providers::{ self, stooq::StooqProvider, DividendEvent, Fact, FilingRecord, FundId, FundShape, FundamentalsProvider, HistoryProvider, IntradayBar, OwnershipPerson, PortfolioData, Quote, QuoteProvider, self, stooq::StooqProvider, DividendEvent, Fact, FilingRecord, FundId, FundMetadata, FundShape, FundamentalsProvider, HistoryProvider, IntradayBar, OwnershipPerson, PortfolioData, Quote, QuoteProvider,};use crate::stream::{Hub, QuoteUpdate, StreamEvent};use crate::{seed, Config};
@@ -97,6 +97,15 @@ const LEADERSHIP_MAX_FILINGS: usize = 30;const DIVIDENDS_INTERVAL_SECS: i64 = 24 * 3600;const DIVIDENDS_STALE_SECS: i64 = 7 * 24 * 3600;/// ETF fund metadata refresh (Phase 28). The Yahoo `quoteSummary` figures —/// expense ratio, distribution yield, NAV, inception, category, fund family,/// strategy summary — change slowly (the prospectus is updated once a year),/// so a monthly staleness window is enough. The due-check is daily so a/// newly-stale ETF is picked up within a day, but the steady-state cost is/// one request per ETF per month.const FUND_METADATA_INTERVAL_SECS: i64 = 24 * 3600;const FUND_METADATA_STALE_SECS: i64 = 30 * 24 * 3600;/// Spawn the scheduler. The returned handle is normally dropped: dropping it/// detaches the task, which then runs for the lifetime of the process.pub fn spawn(pool: SqlitePool, config: Arc<Config>, hub: Arc<Hub>) -> JoinHandle<()> {
@@ -144,6 +153,14 @@ pub fn spawn(pool: SqlitePool, config: Arc<Config>, hub: Arc<Hub>) -> JoinHandle tracing::warn!("[scheduler] bring dividends job forward: {e}"); } // Fund-metadata job (Phase 28): same pattern as the SEC and dividends // jobs — bring it forward to the first tick so a deploy that adds the // table backfills the ETF universe within a tick rather than waiting // out the daily interval. Resumable; the no-stale fast path is free. if let Err(e) = schedule_next(&pool, "fund_metadata", now_ms()).await { tracing::warn!("[scheduler] bring fund_metadata job forward: {e}"); } // Prune's last-run time is loop-local: a restart simply re-prunes once, // which is harmless (local-only DELETEs, no network). let mut last_prune: Option<i64> = None;
@@ -191,10 +208,11 @@ pub fn spawn(pool: SqlitePool, config: Arc<Config>, hub: Arc<Hub>) -> JoinHandle } } // Dividend payouts (Phase 26): sweep stocks whose dividend // history has gone stale (weekly). Runs only when Yahoo is // reachable; gated nowhere else — declared dividends drift after // the ex-date passes, so a fresh pull every week is enough. // Dividend payouts (Phase 26 + 28): sweep stocks AND ETFs whose // dividend / distribution history has gone stale (weekly). Runs // only when Yahoo is reachable; gated nowhere else — declared // events drift after the ex-date passes, so a fresh pull every // week is enough. match is_due(&pool, "dividends", now_ms()).await { Ok(true) => { if let Err(e) = run_dividends(&pool, &config, &hub).await {
@@ -205,6 +223,20 @@ pub fn spawn(pool: SqlitePool, config: Arc<Config>, hub: Arc<Hub>) -> JoinHandle Err(e) => tracing::warn!("[scheduler] dividends due-check: {e}"), } // ETF fund metadata (Phase 28): sweep ETFs whose Yahoo // quoteSummary snapshot has gone stale (monthly). Same Yahoo // guard as intraday + daily-close + dividends; expense / yield / // NAV / strategy change rarely, so this is cheap in steady state. match is_due(&pool, "fund_metadata", now_ms()).await { Ok(true) => { if let Err(e) = run_fund_metadata(&pool, &config, &hub).await { tracing::warn!("[scheduler] fund_metadata: {e:#}"); } } Ok(false) => {} Err(e) => tracing::warn!("[scheduler] fund_metadata due-check: {e}"), } // Intraday quotes: demand-driven (only symbols a browser is // viewing). Inside a trading session every viewed symbol is // polled; outside it, only viewed futures, which trade nearly
@@ -1009,9 +1041,12 @@ async fn run_dividends(pool: &SqlitePool, config: &Config, hub: &Hub) -> anyhow: let started = now_ms(); let next = started + DIVIDENDS_INTERVAL_SECS * 1000; let cutoff = started - DIVIDENDS_STALE_SECS * 1000; // Phase 28: dividends now covers ETF distributions too — same Yahoo // event series, same store path. Indexes and futures still skipped (the // former do not pay anything, the latter have no concept of dividends). let stale: Vec<String> = sqlx::query_scalar( "SELECT ticker FROM symbols \ WHERE kind = 'stock' \ WHERE kind IN ('stock', 'etf') \ AND (dividends_synced_at IS NULL OR dividends_synced_at < ?) \ ORDER BY ticker", )
@@ -1027,7 +1062,7 @@ async fn run_dividends(pool: &SqlitePool, config: &Config, hub: &Hub) -> anyhow: mark_fetching(pool, "dividends").await?; notify_health(hub); tracing::info!("[scheduler] dividends: refreshing {} stocks", stale.len()); tracing::info!("[scheduler] dividends: refreshing {} symbols", stale.len()); let yahoo = YahooProvider::new(providers::http::build_client(config)); let guard = EndpointGuard::with_budget(pool.clone(), yahoo.name(), YAHOO_BUDGET);
@@ -1066,7 +1101,7 @@ async fn run_dividends(pool: &SqlitePool, config: &Config, hub: &Hub) -> anyhow: } let dur = t0.elapsed().as_millis() as i64; let detail = format!("{ok}/{} stocks, {payouts} payouts, {errors} errors", stale.len()); let detail = format!("{ok}/{} symbols, {payouts} payouts, {errors} errors", stale.len()); match stopped { Some(why) => { let full = format!("stopped early ({why}); {detail}");
@@ -1126,6 +1161,163 @@ async fn mark_dividends_synced(pool: &SqlitePool, ticker: &str) -> sqlx::Result< Ok(())}// ────────────────── ETF fund_metadata sweep (Phase 28) ────────────────────/// Sweep ETFs whose Yahoo `quoteSummary` snapshot has gone stale (monthly)./// One request per ETF through the shared `yahoo` `EndpointGuard`; expense/// ratio, distribution yield, NAV, inception, category, fund family, and/// the strategy paragraph are all returned by one request, so even the full/// 28-ETF sweep is well inside the hourly budget. Mirrors `run_dividends`'s/// shape — guard-paced, resumable, broadcast-to-/health.async fn run_fund_metadata(pool: &SqlitePool, config: &Config, hub: &Hub) -> anyhow::Result<()> { let started = now_ms(); let next = started + FUND_METADATA_INTERVAL_SECS * 1000; let cutoff = started - FUND_METADATA_STALE_SECS * 1000; let stale: Vec<String> = sqlx::query_scalar( "SELECT ticker FROM symbols \ WHERE kind = 'etf' \ AND (fund_metadata_synced_at IS NULL OR fund_metadata_synced_at < ?) \ ORDER BY ticker", ) .bind(cutoff) .fetch_all(pool) .await?; if stale.is_empty() { // Fast path: no ETFs need refreshing. No fetching banner, no log row. mark_ok(pool, "fund_metadata", Some(next)).await?; return Ok(()); } mark_fetching(pool, "fund_metadata").await?; notify_health(hub); tracing::info!("[scheduler] fund_metadata: refreshing {} ETFs", stale.len()); let yahoo = YahooProvider::new(providers::http::build_client(config)); let guard = EndpointGuard::with_budget(pool.clone(), yahoo.name(), YAHOO_BUDGET); let t0 = Instant::now(); let mut ok = 0i64; let mut empty = 0i64; let mut errors = 0i64; let mut stopped: Option<String> = None; for ticker in &stale { match guard.acquire().await? { Permit::Granted => {} Permit::Denied(why) => { stopped = Some(why); break; } } match yahoo.fund_metadata(ticker).await { Ok(Some(meta)) => { guard.record_success().await?; if let Err(e) = store_fund_metadata(pool, ticker, &meta).await { tracing::warn!("[scheduler] fund_metadata store {ticker}: {e:#}"); errors += 1; continue; } mark_fund_metadata_synced(pool, ticker).await?; ok += 1; } // Yahoo answered cleanly but the ETF has no fund modules (a tiny // or obscure ticker): stamp it checked so we do not re-fetch the // same empty answer next tick, but log the empty. Ok(None) => { guard.record_success().await?; mark_fund_metadata_synced(pool, ticker).await?; empty += 1; } Err(e) => { guard.record_failure(&e).await?; errors += 1; tracing::warn!("[scheduler] fund_metadata {ticker}: {e:#}"); } } } let dur = t0.elapsed().as_millis() as i64; let detail = format!( "{ok}/{} ETFs ({empty} empty, {errors} errors)", stale.len() ); match stopped { Some(why) => { let full = format!("stopped early ({why}); {detail}"); tracing::warn!("[scheduler] fund_metadata: {full}"); log_fetch( pool, "fund_metadata", "yahoo", "skipped", Some(&full), Some(ok), dur, started, ).await?; } None => { tracing::info!("[scheduler] fund_metadata: {detail}"); log_fetch( pool, "fund_metadata", "yahoo", "ok", Some(&detail), Some(ok), dur, started, ).await?; } } mark_ok(pool, "fund_metadata", Some(next)).await?; notify_health(hub); Ok(())}/// Upsert one ETF's Yahoo `quoteSummary` metadata. Yahoo serves the full/// current snapshot each call, so the row is replaced wholesale: a field/// Yahoo no longer carries on a refresh becomes `NULL` here, rather than/// keeping a stale value. `pub(crate)`: the add-symbol backfill reuses it.pub(crate) async fn store_fund_metadata( pool: &SqlitePool, ticker: &str, m: &FundMetadata,) -> sqlx::Result<()> { sqlx::query( "INSERT INTO fund_metadata \ (ticker, expense_ratio, yield_pct, trailing_yield_pct, nav_price, \ inception_date, category, fund_family, strategy_summary, updated_at) \ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) \ ON CONFLICT(ticker) DO UPDATE SET \ expense_ratio = excluded.expense_ratio, \ yield_pct = excluded.yield_pct, \ trailing_yield_pct = excluded.trailing_yield_pct, \ nav_price = excluded.nav_price, \ inception_date = excluded.inception_date, \ category = excluded.category, \ fund_family = excluded.fund_family, \ strategy_summary = excluded.strategy_summary, \ updated_at = excluded.updated_at", ) .bind(ticker) .bind(m.expense_ratio) .bind(m.yield_pct) .bind(m.trailing_yield_pct) .bind(m.nav_price) .bind(&m.inception_date) .bind(&m.category) .bind(&m.fund_family) .bind(&m.strategy_summary) .bind(now_ms()) .execute(pool) .await?; Ok(())}/// Stamp an ETF as freshly fund-metadata-synced. ETFs only — the column is/// `NULL` forever on every non-ETF row.async fn mark_fund_metadata_synced(pool: &SqlitePool, ticker: &str) -> sqlx::Result<()> { let now = now_ms(); sqlx::query( "UPDATE symbols SET fund_metadata_synced_at = ?, updated_at = ? WHERE ticker = ?", ) .bind(now) .bind(now) .bind(ticker) .execute(pool) .await?; Ok(())}/// Fill in `symbols.cik` for any stock found in the bulk SEC ticker map./// Returns how many were newly resolved.async fn resolve_ciks(pool: &SqlitePool, map: &HashMap<String, String>) -> sqlx::Result<i64> {
@@ -1306,19 +1498,23 @@ async fn store_fund_portfolio( ticker: &str, p: &PortfolioData,) -> anyhow::Result<()> { // Asset mix is variable-length, so it rides in one JSON column rather than // its own table: [["Equity", 99.8], ["Cash & equivalents", 0.2], ...]. // Asset / sector / geography mixes are each variable-length, so they ride // in JSON columns rather than their own tables: // [["Equity", 99.8], ["Cash & equivalents", 0.2], ...]. let asset_mix = serde_json::to_string(&p.asset_mix)?; let sector_mix = serde_json::to_string(&p.sector_mix)?; let geography_mix = serde_json::to_string(&p.geography_mix)?; let mut tx = pool.begin().await?; sqlx::query( "INSERT INTO fund_profiles \ (ticker, kind, net_assets, total_assets, holdings_count, report_date, \ asset_mix, updated_at) \ VALUES (?, 'portfolio', ?, ?, ?, ?, ?, ?) \ asset_mix, sector_mix, geography_mix, updated_at) \ VALUES (?, 'portfolio', ?, ?, ?, ?, ?, ?, ?, ?) \ ON CONFLICT(ticker) DO UPDATE SET \ kind = excluded.kind, net_assets = excluded.net_assets, \ total_assets = excluded.total_assets, holdings_count = excluded.holdings_count, \ report_date = excluded.report_date, asset_mix = excluded.asset_mix, \ sector_mix = excluded.sector_mix, geography_mix = excluded.geography_mix, \ updated_at = excluded.updated_at", ) .bind(ticker)
@@ -1327,6 +1523,8 @@ async fn store_fund_portfolio( .bind(p.holdings_count) .bind(&p.report_date) .bind(&asset_mix) .bind(§or_mix) .bind(&geography_mix) .bind(now_ms()) .execute(&mut *tx) .await?;
@@ -1510,11 +1708,17 @@ async fn guarded<T>(/// the normal scheduler sweeps pick up whatever this run missed.pub(crate) async fn backfill_symbol(pool: &SqlitePool, config: &Config, ticker: &str, kind: &str) { backfill_history(pool, config, ticker, kind).await; // Dividends are stocks-only and ride the Yahoo guard, not SEC, so they // run independently of the SEC contact-email gate below. if kind == "stock" { // Phase 26 + Phase 28: dividends covers stock dividends and ETF // distributions (same Yahoo event series). Rides the Yahoo guard, so it // runs independently of the SEC contact-email gate below. if kind == "stock" || kind == "etf" { backfill_dividends(pool, config, ticker).await; } // Phase 28: ETFs get their Yahoo fund_metadata pulled too — expense // ratio, yield, NAV, inception, category, family, strategy summary. if kind == "etf" { backfill_fund_metadata(pool, config, ticker).await; } // SEC data covers stocks and ETFs; indexes and futures do not file. The // whole SEC step is skipped with no contact email configured, as `run_sec`
@@ -1531,10 +1735,34 @@ pub(crate) async fn backfill_symbol(pool: &SqlitePool, config: &Config, ticker: }}/// Pull and store a freshly-added stock's dividend history (Phase 26). Stocks/// only — the caller already filters; routed through the same `yahoo` guard/// the dividends sweep uses. Best-effort: a guard denial or upstream error/// leaves the stock for the next normal sweep./// Pull and store a freshly-added ETF's Yahoo `quoteSummary` metadata (Phase/// 28). Mirrors `backfill_dividends`: same `yahoo` guard, best-effort, no/// failure propagated to the add-symbol response.async fn backfill_fund_metadata(pool: &SqlitePool, config: &Config, ticker: &str) { let yahoo = YahooProvider::new(providers::http::build_client(config)); let guard = EndpointGuard::with_budget(pool.clone(), yahoo.name(), YAHOO_BUDGET); match guarded(&guard, yahoo.fund_metadata(ticker)).await { Some(Ok(Some(meta))) => match store_fund_metadata(pool, ticker, &meta).await { Ok(()) => { let _ = mark_fund_metadata_synced(pool, ticker).await; tracing::info!("[backfill] {ticker} <- fund_metadata"); } Err(e) => tracing::warn!("[backfill] store fund_metadata {ticker}: {e:#}"), }, // Yahoo answered cleanly but had no fund modules for this ETF — stamp // it checked so the next sweep does not re-fetch the same empty. Some(Ok(None)) => { let _ = mark_fund_metadata_synced(pool, ticker).await; } Some(Err(e)) => tracing::warn!("[backfill] fund_metadata {ticker}: {e:#}"), None => {} }}/// Pull and store a freshly-added symbol's dividend / distribution history/// (Phase 26 + Phase 28: stocks and ETFs both use this path). Routed through/// the same `yahoo` guard the dividends sweep uses. Best-effort: a guard/// denial or upstream error leaves the symbol for the next normal sweep.async fn backfill_dividends(pool: &SqlitePool, config: &Config, ticker: &str) { let yahoo = YahooProvider::new(providers::http::build_client(config)); let guard = EndpointGuard::with_budget(pool.clone(), yahoo.name(), YAHOO_BUDGET);
@@ -23,6 +23,12 @@ struct SeedSymbol { name: String, kind: String, exchange: Option<String>, /// Phase 28: the curated benchmark index a fund tracks (e.g. `^SPX`), /// for the relative-performance overlay on the ETF symbol page. Only /// the broad-market ETFs carry one in `starter.csv`; everything else /// (including stocks, futures, indexes, sector / bond / commodity ETFs) /// leaves it empty and the overlay is hidden. benchmark: Option<String>,}fn parse_universe(path: &Path) -> Result<Vec<SeedSymbol>> {
@@ -36,16 +42,13 @@ fn parse_universe(path: &Path) -> Result<Vec<SeedSymbol>> { if ticker.is_empty() { continue; } let exchange = cell(3); let opt = |s: String| if s.is_empty() { None } else { Some(s) }; out.push(SeedSymbol { ticker, name: cell(1), kind: cell(2), exchange: if exchange.is_empty() { None } else { Some(exchange) }, exchange: opt(cell(3)), benchmark: opt(cell(4)), }); } Ok(out)
@@ -59,20 +62,26 @@ pub async fn run(pool: &SqlitePool, config: &Config, history: &dyn HistoryProvid let symbols = parse_universe(&path)?; tracing::info!("seed: {} symbols in {}", symbols.len(), path.display()); // Upsert every symbol. Local only, no network. // Upsert every symbol. Local only, no network. Phase 28: the curated // benchmark column is set from the CSV on each seed pass, so a re-run // picks up any newly-curated mapping. A `NULL` benchmark stays `NULL` // (don't overwrite a user-edited one — they all live in the CSV). for s in &symbols { let now = now_ms(); sqlx::query( "INSERT INTO symbols (ticker, name, kind, exchange, is_seeded, created_at, updated_at) \ VALUES (?, ?, ?, ?, 1, ?, ?) \ "INSERT INTO symbols \ (ticker, name, kind, exchange, benchmark, is_seeded, created_at, updated_at) \ VALUES (?, ?, ?, ?, ?, 1, ?, ?) \ ON CONFLICT(ticker) DO UPDATE SET \ name = excluded.name, kind = excluded.kind, \ exchange = excluded.exchange, is_seeded = 1, updated_at = excluded.updated_at", exchange = excluded.exchange, benchmark = excluded.benchmark, \ is_seeded = 1, updated_at = excluded.updated_at", ) .bind(&s.ticker) .bind(&s.name) .bind(&s.kind) .bind(&s.exchange) .bind(&s.benchmark) .bind(now) .bind(now) .execute(pool)
modified
templates/pages/symbol.html
@@ -77,14 +77,16 @@ it and paints each swatch from its own ink palette. #} <div class="ind-bar" role="group" aria-label="Chart indicators"> {% for ind in [ {"key": "sma50", "label": "SMA 50", "on": true, "dot": true}, {"key": "sma200", "label": "SMA 200", "on": true, "dot": true}, {"key": "ema21", "label": "EMA 21", "on": false, "dot": true}, {"key": "volume", "label": "Volume", "on": true, "dot": false}, {"key": "rsi", "label": "RSI", "on": false, "dot": true} {"key": "sma50", "label": "SMA 50", "on": true, "dot": true, "hide": false}, {"key": "sma200", "label": "SMA 200", "on": true, "dot": true, "hide": false}, {"key": "ema21", "label": "EMA 21", "on": false, "dot": true, "hide": false}, {"key": "volume", "label": "Volume", "on": true, "dot": false, "hide": false}, {"key": "rsi", "label": "RSI", "on": false, "dot": true, "hide": false}, {"key": "benchmark", "label": (symbol.benchmark or "Benchmark"), "on": true, "dot": true, "hide": (not symbol.benchmark)} ] %} <button type="button" class="ind-btn{% if ind.on %} is-active{% endif %}" data-ind="{{ ind.key }}" aria-pressed="{{ 'true' if ind.on else 'false' }}"> data-ind="{{ ind.key }}" aria-pressed="{{ 'true' if ind.on else 'false' }}" {% if ind.hide %}hidden{% endif %}> {% if ind.dot %}<span class="ind-btn__dot" aria-hidden="true"></span>{% endif %}{{ ind.label }} </button> {% endfor %}
@@ -247,59 +249,6 @@ </section> {% endif %} {# --- dividends: payout history + on-track pace (Phase 26) --- #} {% if dividends %} <h2 class="section-title">Dividends{% if symbol.dividends_synced_at %}<span class="section-title__asof">synced from Yahoo {{ symbol.dividends_synced_at|ago }}</span>{% endif %}</h2> {% if dividends.synced and dividends.history %} <section class="panel div-panel"> {# headline: inferred cadence + the on-track verdict pill #} <div class="div-head"> <span class="div-cadence">{{ dividends.pace.cadence_caption }}</span> {% if dividends.projection_display %} <span class="vbadge vbadge--{{ dividends.pace.grade }}">{{ dividends.pace.verdict }}</span> {% endif %} </div> {# pace: prior calendar year vs YTD-projected current year #} <dl class="div-pace"> <div class="div-pace__item"> <dt class="div-pace__cap">{{ dividends.current_year - 1 }} total</dt> <dd class="div-pace__val num">{% if dividends.pace.prior_year_total > 0 %}{{ dividends.prior_year_display }}{% else %}—{% endif %}</dd> </div> <div class="div-pace__item"> <dt class="div-pace__cap">{{ dividends.current_year }} so far</dt> <dd class="div-pace__val num">{{ dividends.ytd_display }}<span class="div-pace__sub">{{ dividends.pace.ytd_count }} payment{% if dividends.pace.ytd_count != 1 %}s{% endif %}</span></dd> </div> {% if dividends.projection_display %} <div class="div-pace__item"> <dt class="div-pace__cap">{{ dividends.current_year }} projected</dt> <dd class="div-pace__val num">{{ dividends.projection_display }}<span class="div-pace__sub div-pace__sub--{{ dividends.pace.grade }}">{{ dividends.pct_change_display }} vs {{ dividends.current_year - 1 }}</span></dd> </div> {% endif %} </dl> <p class="div-pace__src">Projection scales {{ symbol.ticker }}’s declared payments so far this year up to the inferred cadence, then compares to last year’s total. Between payouts it is an estimate, not a guarantee.</p> {# per-event history, newest first #} <h3 class="div-hist__title">Payment history</h3> <ul class="div-hist"> {% for d in dividends.history %} <li class="div-row"> <span class="div-row__date">{{ d.ex_date|shortdate }}</span> <span class="div-row__amt num">{{ d.amount }}</span> </li> {% endfor %} </ul> </section> {% elif dividends.synced %} <div class="fund-pending">{{ symbol.ticker }} has not paid a dividend in the past five years.</div> {% else %} <div class="fund-pending"> Dividend history for {{ symbol.ticker }} has not synced yet. It lands on the next data refresh; see <a href="/health">data health</a>. </div> {% endif %} {% endif %} {# --- leadership: officers, board & recent changes (Phase 14) --- #} <h2 class="section-title">Leadership{% if symbol.leadership_synced_at %}<span class="section-title__asof">synced from SEC {{ symbol.leadership_synced_at|ago }}</span>{% endif %}</h2> {% if leadership and leadership.roster %}
@@ -345,8 +294,90 @@ {% endif %} {# --- fund profile: ETFs only (Phase 18) --- #} {# --- ETFs only: Phase 18 (fund profile) + Phase 28 (about / sector / --- #} {# --- geography / trailing returns / growth of $10k / benchmark) --- #} {% if symbol.kind == 'etf' %} {# About this fund: expense ratio, yield, NAV + premium/discount, family, #} {# category, inception, and the issuer's strategy paragraph (Phase 28). #} <h2 class="section-title">About this fund{% if symbol.fund_metadata_synced_at %}<span class="section-title__asof">synced from Yahoo {{ symbol.fund_metadata_synced_at|ago }}</span>{% endif %}</h2> {% if fund_meta %} <section class="panel fund-about"> <dl class="fund-about__stats"> <div class="fund-about__stat"> <dt class="fund-about__cap">Expense ratio</dt> <dd class="fund-about__val num">{{ fund_meta.expense_ratio }}</dd> </div> <div class="fund-about__stat"> <dt class="fund-about__cap">Distribution yield</dt> <dd class="fund-about__val num">{{ fund_meta.yield_pct }}</dd> </div> {% if fund_meta.nav_price is not none %} <div class="fund-about__stat"> <dt class="fund-about__cap">NAV</dt> <dd class="fund-about__val num">{{ fund_meta.nav_price|money }}{% if fund_meta.premium %}<span class="fund-about__sub fund-about__sub--{{ fund_meta.premium.grade }}">{{ fund_meta.premium.text }} to price</span>{% endif %}</dd> </div> {% endif %} {% if fund_meta.inception_date %} <div class="fund-about__stat"> <dt class="fund-about__cap">Inception</dt> <dd class="fund-about__val num">{{ fund_meta.inception_date|shortdate }}</dd> </div> {% endif %} {% if fund_meta.category %} <div class="fund-about__stat"> <dt class="fund-about__cap">Category</dt> <dd class="fund-about__val">{{ fund_meta.category }}</dd> </div> {% endif %} {% if fund_meta.fund_family %} <div class="fund-about__stat"> <dt class="fund-about__cap">Family</dt> <dd class="fund-about__val">{{ fund_meta.fund_family }}</dd> </div> {% endif %} </dl> {% if fund_meta.strategy_summary %} <p class="fund-about__summary">{{ fund_meta.strategy_summary }}</p> {% endif %} </section> {% else %} <div class="fund-pending"> Fund metadata for {{ symbol.ticker }} has not synced yet. It lands on the next refresh; see <a href="/health">data health</a>. </div> {% endif %} {# Trailing returns (Phase 28). Eight periods, annualised past 1 year. #} {% if returns %} <h2 class="section-title">Trailing returns{% if symbol.history_synced_at %}<span class="section-title__asof">from price history</span>{% endif %}</h2> <section class="panel ret-panel"> <table class="ret-table"> <thead> <tr><th scope="col">Window</th><th scope="col" class="num">Return</th><th scope="col" class="num">Annualised</th></tr> </thead> <tbody> {% for r in returns %} <tr> <th scope="row">{{ r.label }}</th> <td class="num{% if r.dir > 0 %} ret-cell--up{% elif r.dir < 0 %} ret-cell--down{% endif %}">{{ r.pct }}</td> <td class="num ret-cell--ann">{{ r.annualised }}</td> </tr> {% endfor %} </tbody> </table> <p class="ret-panel__src">Cumulative price returns from {{ symbol.ticker }}’s stored daily closes. Distributions are not folded in; see Distributions below.</p> </section> {# Growth of $10,000 (Phase 28). Drawn client-side from the longest #} {# available range, alongside the curated benchmark when one is set. #} <h2 class="section-title">Growth of $10,000{% if symbol.benchmark %}<span class="section-title__asof">vs {{ symbol.benchmark }}</span>{% endif %}</h2> <section class="panel growth-panel"> <div id="growth-chart" data-ticker="{{ symbol.ticker }}"{% if symbol.benchmark %} data-benchmark="{{ symbol.benchmark }}"{% endif %}></div> <p class="growth-panel__src">A $10,000 investment scaled by the fund’s daily closes since inception.{% if symbol.benchmark %} The dashed line tracks the same amount in {{ symbol.benchmark }} from the fund’s first trading day forward.{% endif %}</p> </section> {% endif %} <h2 class="section-title">Fund profile{% if symbol.fund_synced_at %}<span class="section-title__asof">synced from SEC {{ symbol.fund_synced_at|ago }}</span>{% endif %}</h2> {% if fund %} <section class="panel fund">
@@ -372,6 +403,7 @@ <p class="fund__note">{{ symbol.ticker }} is a grantor trust: it holds the physical commodity directly rather than a portfolio of securities, so it reports no holdings.</p> {% elif fund.asset_mix %} <div class="fund__mix"> <h3 class="fund__mix-label">Asset mix</h3> <div class="mixbar"> {% for s in fund.asset_mix %} <span class="mixbar__seg" style="width:{{ s.width }}%" title="{{ s.label }} {{ s.pct }}"></span>
@@ -387,6 +419,45 @@ </dl> </div> {% endif %} {# Phase 28 mixes hide when degenerate: an equity ETF whose issuer #} {# category rolls up entirely to "Corporate" gets a single-bucket #} {# sector_mix, which carries no information — skip it then. #} {% if fund.sector_mix and fund.sector_mix|length > 1 %} <div class="fund__mix"> <h3 class="fund__mix-label">Issuer sectors</h3> <div class="mixbar"> {% for s in fund.sector_mix %} <span class="mixbar__seg" style="width:{{ s.width }}%" title="{{ s.label }} {{ s.pct }}"></span> {% endfor %} </div> <dl class="mixlegend"> {% for s in fund.sector_mix %} <div class="mixlegend__item"> <dt><i class="mixlegend__dot"></i>{{ s.label }}</dt> <dd class="num">{{ s.pct }}</dd> </div> {% endfor %} </dl> </div> {% endif %} {% if fund.geography_mix and fund.geography_mix|length > 1 %} <div class="fund__mix"> <h3 class="fund__mix-label">Geography</h3> <div class="mixbar"> {% for s in fund.geography_mix %} <span class="mixbar__seg" style="width:{{ s.width }}%" title="{{ s.label }} {{ s.pct }}"></span> {% endfor %} </div> <dl class="mixlegend"> {% for s in fund.geography_mix %} <div class="mixlegend__item"> <dt><i class="mixlegend__dot"></i>{{ s.label }}</dt> <dd class="num">{{ s.pct }}</dd> </div> {% endfor %} </dl> </div> {% endif %} </section> {% if fund.holdings %}
@@ -413,6 +484,60 @@ {% endif %} {% endif %} {# --- dividends / distributions: stocks AND ETFs (Phase 26 + 28) --- #} {% if dividends %} {% set div_label = 'Distributions' if symbol.kind == 'etf' else 'Dividends' %} <h2 class="section-title">{{ div_label }}{% if symbol.dividends_synced_at %}<span class="section-title__asof">synced from Yahoo {{ symbol.dividends_synced_at|ago }}</span>{% endif %}</h2> {% if dividends.synced and dividends.history %} <section class="panel div-panel"> {# headline: inferred cadence + the on-track verdict pill #} <div class="div-head"> <span class="div-cadence">{{ dividends.pace.cadence_caption }}</span> {% if dividends.projection_display %} <span class="vbadge vbadge--{{ dividends.pace.grade }}">{{ dividends.pace.verdict }}</span> {% endif %} </div> {# pace: prior calendar year vs YTD-projected current year #} <dl class="div-pace"> <div class="div-pace__item"> <dt class="div-pace__cap">{{ dividends.current_year - 1 }} total</dt> <dd class="div-pace__val num">{% if dividends.pace.prior_year_total > 0 %}{{ dividends.prior_year_display }}{% else %}—{% endif %}</dd> </div> <div class="div-pace__item"> <dt class="div-pace__cap">{{ dividends.current_year }} so far</dt> <dd class="div-pace__val num">{{ dividends.ytd_display }}<span class="div-pace__sub">{{ dividends.pace.ytd_count }} payment{% if dividends.pace.ytd_count != 1 %}s{% endif %}</span></dd> </div> {% if dividends.projection_display %} <div class="div-pace__item"> <dt class="div-pace__cap">{{ dividends.current_year }} projected</dt> <dd class="div-pace__val num">{{ dividends.projection_display }}<span class="div-pace__sub div-pace__sub--{{ dividends.pace.grade }}">{{ dividends.pct_change_display }} vs {{ dividends.current_year - 1 }}</span></dd> </div> {% endif %} </dl> <p class="div-pace__src">Projection scales {{ symbol.ticker }}’s declared payments so far this year up to the inferred cadence, then compares to last year’s total. Between payouts it is an estimate, not a guarantee.</p> {# per-event history, newest first #} <h3 class="div-hist__title">Payment history</h3> <ul class="div-hist"> {% for d in dividends.history %} <li class="div-row"> <span class="div-row__date">{{ d.ex_date|shortdate }}</span> <span class="div-row__amt num">{{ d.amount }}</span> </li> {% endfor %} </ul> </section> {% elif dividends.synced %} <div class="fund-pending">{{ symbol.ticker }} has not {% if symbol.kind == 'etf' %}made a distribution{% else %}paid a dividend{% endif %} in the past five years.</div> {% else %} <div class="fund-pending"> {{ div_label }} history for {{ symbol.ticker }} has not synced yet. It lands on the next data refresh; see <a href="/health">data health</a>. </div> {% endif %} {% endif %} {# --- SEC filings: stocks and ETFs --- #} {% if filings %} {# A stock's filings sync stamps `filings_synced_at`; an ETF's ride along
modified
universe/starter.csv
@@ -1,155 +1,155 @@ticker,name,kind,exchange^SPX,S&P 500,index,^DJI,Dow Jones Industrial Average,index,^NDX,Nasdaq 100,index,^NDQ,Nasdaq Composite,index,^RUT,Russell 2000,index,^VIX,CBOE Volatility Index,index,SPY,SPDR S&P 500 ETF Trust,etf,NYSEARCAVOO,Vanguard S&P 500 ETF,etf,NYSEARCAIVV,iShares Core S&P 500 ETF,etf,NYSEARCAVTI,Vanguard Total Stock Market ETF,etf,NYSEARCAQQQ,Invesco QQQ Trust,etf,NASDAQDIA,SPDR Dow Jones Industrial Average ETF,etf,NYSEARCAIWM,iShares Russell 2000 ETF,etf,NYSEARCAVEA,Vanguard FTSE Developed Markets ETF,etf,NYSEARCAVWO,Vanguard FTSE Emerging Markets ETF,etf,NYSEARCAVUG,Vanguard Growth ETF,etf,NYSEARCAVTV,Vanguard Value ETF,etf,NYSEARCASCHD,Schwab US Dividend Equity ETF,etf,NYSEARCAVIG,Vanguard Dividend Appreciation ETF,etf,NYSEARCAVYM,Vanguard High Dividend Yield ETF,etf,NYSEARCAVXUS,Vanguard Total International Stock ETF,etf,NASDAQBND,Vanguard Total Bond Market ETF,etf,NASDAQAGG,iShares Core US Aggregate Bond ETF,etf,NYSEARCATLT,iShares 20+ Year Treasury Bond ETF,etf,NASDAQGLD,SPDR Gold Shares,etf,NYSEARCASLV,iShares Silver Trust,etf,NYSEARCAXLK,Technology Select Sector SPDR Fund,etf,NYSEARCAXLF,Financial Select Sector SPDR Fund,etf,NYSEARCAXLE,Energy Select Sector SPDR Fund,etf,NYSEARCAXLV,Health Care Select Sector SPDR Fund,etf,NYSEARCASMH,VanEck Semiconductor ETF,etf,NASDAQARKK,ARK Innovation ETF,etf,NYSEARCAEFA,iShares MSCI EAFE ETF,etf,NYSEARCAEEM,iShares MSCI Emerging Markets ETF,etf,NYSEARCAAAPL,Apple Inc.,stock,NASDAQMSFT,Microsoft Corporation,stock,NASDAQNVDA,NVIDIA Corporation,stock,NASDAQAMZN,Amazon.com Inc.,stock,NASDAQGOOGL,Alphabet Inc. Class A,stock,NASDAQGOOG,Alphabet Inc. Class C,stock,NASDAQMETA,Meta Platforms Inc.,stock,NASDAQTSLA,Tesla Inc.,stock,NASDAQBRK.B,Berkshire Hathaway Inc. Class B,stock,NYSEAVGO,Broadcom Inc.,stock,NASDAQORCL,Oracle Corporation,stock,NYSEJPM,JPMorgan Chase & Co.,stock,NYSEV,Visa Inc.,stock,NYSEMA,Mastercard Incorporated,stock,NYSELLY,Eli Lilly and Company,stock,NYSEUNH,UnitedHealth Group Incorporated,stock,NYSEJNJ,Johnson & Johnson,stock,NYSEXOM,Exxon Mobil Corporation,stock,NYSECVX,Chevron Corporation,stock,NYSEWMT,Walmart Inc.,stock,NYSEPG,Procter & Gamble Company,stock,NYSEHD,Home Depot Inc.,stock,NYSECOST,Costco Wholesale Corporation,stock,NASDAQABBV,AbbVie Inc.,stock,NYSEMRK,Merck & Co. Inc.,stock,NYSEPEP,PepsiCo Inc.,stock,NASDAQKO,Coca-Cola Company,stock,NYSEBAC,Bank of America Corporation,stock,NYSECRM,Salesforce Inc.,stock,NYSEADBE,Adobe Inc.,stock,NASDAQAMD,Advanced Micro Devices Inc.,stock,NASDAQNFLX,Netflix Inc.,stock,NASDAQDIS,Walt Disney Company,stock,NYSETMO,Thermo Fisher Scientific Inc.,stock,NYSEABT,Abbott Laboratories,stock,NYSEACN,Accenture plc,stock,NYSELIN,Linde plc,stock,NASDAQMCD,McDonald's Corporation,stock,NYSECSCO,Cisco Systems Inc.,stock,NASDAQINTC,Intel Corporation,stock,NASDAQQCOM,QUALCOMM Incorporated,stock,NASDAQTXN,Texas Instruments Incorporated,stock,NASDAQIBM,International Business Machines Corporation,stock,NYSENOW,ServiceNow Inc.,stock,NYSEINTU,Intuit Inc.,stock,NASDAQAMAT,Applied Materials Inc.,stock,NASDAQMU,Micron Technology Inc.,stock,NASDAQLRCX,Lam Research Corporation,stock,NASDAQADI,Analog Devices Inc.,stock,NASDAQPANW,Palo Alto Networks Inc.,stock,NASDAQCRWD,CrowdStrike Holdings Inc.,stock,NASDAQSNOW,Snowflake Inc.,stock,NYSEPLTR,Palantir Technologies Inc.,stock,NASDAQNET,Cloudflare Inc.,stock,NYSEDDOG,Datadog Inc.,stock,NASDAQUBER,Uber Technologies Inc.,stock,NYSEABNB,Airbnb Inc.,stock,NASDAQSHOP,Shopify Inc.,stock,NYSEWFC,Wells Fargo & Company,stock,NYSEGS,Goldman Sachs Group Inc.,stock,NYSEMS,Morgan Stanley,stock,NYSEC,Citigroup Inc.,stock,NYSEBLK,BlackRock Inc.,stock,NYSESCHW,Charles Schwab Corporation,stock,NYSEAXP,American Express Company,stock,NYSEBX,Blackstone Inc.,stock,NYSESPGI,S&P Global Inc.,stock,NYSEPFE,Pfizer Inc.,stock,NYSEDHR,Danaher Corporation,stock,NYSEBMY,Bristol-Myers Squibb Company,stock,NYSEAMGN,Amgen Inc.,stock,NASDAQGILD,Gilead Sciences Inc.,stock,NASDAQCVS,CVS Health Corporation,stock,NYSEMDT,Medtronic plc,stock,NYSEISRG,Intuitive Surgical Inc.,stock,NASDAQLOW,Lowe's Companies Inc.,stock,NYSENKE,NIKE Inc.,stock,NYSESBUX,Starbucks Corporation,stock,NASDAQTGT,Target Corporation,stock,NYSEPM,Philip Morris International Inc.,stock,NYSEMO,Altria Group Inc.,stock,NYSEMDLZ,Mondelez International Inc.,stock,NASDAQCL,Colgate-Palmolive Company,stock,NYSECOP,ConocoPhillips,stock,NYSESLB,Schlumberger Limited,stock,NYSEBA,Boeing Company,stock,NYSECAT,Caterpillar Inc.,stock,NYSEDE,Deere & Company,stock,NYSEGE,GE Aerospace,stock,NYSEHON,Honeywell International Inc.,stock,NASDAQUPS,United Parcel Service Inc.,stock,NYSEFDX,FedEx Corporation,stock,NYSELMT,Lockheed Martin Corporation,stock,NYSERTX,RTX Corporation,stock,NYSEMMM,3M Company,stock,NYSEUNP,Union Pacific Corporation,stock,NYSET,AT&T Inc.,stock,NYSEVZ,Verizon Communications Inc.,stock,NYSETMUS,T-Mobile US Inc.,stock,NASDAQCMCSA,Comcast Corporation,stock,NASDAQF,Ford Motor Company,stock,NYSEGM,General Motors Company,stock,NYSERIVN,Rivian Automotive Inc.,stock,NASDAQCOIN,Coinbase Global Inc.,stock,NASDAQHOOD,Robinhood Markets Inc.,stock,NASDAQSOFI,SoFi Technologies Inc.,stock,NASDAQMSTR,Strategy Incorporated,stock,NASDAQDELL,Dell Technologies Inc.,stock,NYSEMRVL,Marvell Technology Inc.,stock,NASDAQSMCI,Super Micro Computer Inc.,stock,NASDAQES=F,S&P 500 E-Mini Futures,future,CMENQ=F,Nasdaq 100 E-Mini Futures,future,CMEYM=F,Dow E-Mini Futures,future,CMERTY=F,Russell 2000 E-Mini Futures,future,CMECL=F,Crude Oil Futures (WTI),future,NYMEXBZ=F,Brent Crude Oil Futures,future,NYMEXGC=F,Gold Futures,future,COMEXSI=F,Silver Futures,future,COMEXHG=F,Copper Futures,future,COMEXNG=F,Natural Gas Futures,future,NYMEXticker,name,kind,exchange,benchmark^SPX,S&P 500,index,,^DJI,Dow Jones Industrial Average,index,,^NDX,Nasdaq 100,index,,^NDQ,Nasdaq Composite,index,,^RUT,Russell 2000,index,,^VIX,CBOE Volatility Index,index,,SPY,SPDR S&P 500 ETF Trust,etf,NYSEARCA,^SPXVOO,Vanguard S&P 500 ETF,etf,NYSEARCA,^SPXIVV,iShares Core S&P 500 ETF,etf,NYSEARCA,^SPXVTI,Vanguard Total Stock Market ETF,etf,NYSEARCA,^SPXQQQ,Invesco QQQ Trust,etf,NASDAQ,^NDXDIA,SPDR Dow Jones Industrial Average ETF,etf,NYSEARCA,^DJIIWM,iShares Russell 2000 ETF,etf,NYSEARCA,^RUTVEA,Vanguard FTSE Developed Markets ETF,etf,NYSEARCA,VWO,Vanguard FTSE Emerging Markets ETF,etf,NYSEARCA,VUG,Vanguard Growth ETF,etf,NYSEARCA,^SPXVTV,Vanguard Value ETF,etf,NYSEARCA,^SPXSCHD,Schwab US Dividend Equity ETF,etf,NYSEARCA,^SPXVIG,Vanguard Dividend Appreciation ETF,etf,NYSEARCA,^SPXVYM,Vanguard High Dividend Yield ETF,etf,NYSEARCA,^SPXVXUS,Vanguard Total International Stock ETF,etf,NASDAQ,BND,Vanguard Total Bond Market ETF,etf,NASDAQ,AGG,iShares Core US Aggregate Bond ETF,etf,NYSEARCA,TLT,iShares 20+ Year Treasury Bond ETF,etf,NASDAQ,GLD,SPDR Gold Shares,etf,NYSEARCA,SLV,iShares Silver Trust,etf,NYSEARCA,XLK,Technology Select Sector SPDR Fund,etf,NYSEARCA,^SPXXLF,Financial Select Sector SPDR Fund,etf,NYSEARCA,^SPXXLE,Energy Select Sector SPDR Fund,etf,NYSEARCA,^SPXXLV,Health Care Select Sector SPDR Fund,etf,NYSEARCA,^SPXSMH,VanEck Semiconductor ETF,etf,NASDAQ,^NDXARKK,ARK Innovation ETF,etf,NYSEARCA,^NDXEFA,iShares MSCI EAFE ETF,etf,NYSEARCA,EEM,iShares MSCI Emerging Markets ETF,etf,NYSEARCA,AAPL,Apple Inc.,stock,NASDAQ,MSFT,Microsoft Corporation,stock,NASDAQ,NVDA,NVIDIA Corporation,stock,NASDAQ,AMZN,Amazon.com Inc.,stock,NASDAQ,GOOGL,Alphabet Inc. Class A,stock,NASDAQ,GOOG,Alphabet Inc. Class C,stock,NASDAQ,META,Meta Platforms Inc.,stock,NASDAQ,TSLA,Tesla Inc.,stock,NASDAQ,BRK.B,Berkshire Hathaway Inc. Class B,stock,NYSE,AVGO,Broadcom Inc.,stock,NASDAQ,ORCL,Oracle Corporation,stock,NYSE,JPM,JPMorgan Chase & Co.,stock,NYSE,V,Visa Inc.,stock,NYSE,MA,Mastercard Incorporated,stock,NYSE,LLY,Eli Lilly and Company,stock,NYSE,UNH,UnitedHealth Group Incorporated,stock,NYSE,JNJ,Johnson & Johnson,stock,NYSE,XOM,Exxon Mobil Corporation,stock,NYSE,CVX,Chevron Corporation,stock,NYSE,WMT,Walmart Inc.,stock,NYSE,PG,Procter & Gamble Company,stock,NYSE,HD,Home Depot Inc.,stock,NYSE,COST,Costco Wholesale Corporation,stock,NASDAQ,ABBV,AbbVie Inc.,stock,NYSE,MRK,Merck & Co. Inc.,stock,NYSE,PEP,PepsiCo Inc.,stock,NASDAQ,KO,Coca-Cola Company,stock,NYSE,BAC,Bank of America Corporation,stock,NYSE,CRM,Salesforce Inc.,stock,NYSE,ADBE,Adobe Inc.,stock,NASDAQ,AMD,Advanced Micro Devices Inc.,stock,NASDAQ,NFLX,Netflix Inc.,stock,NASDAQ,DIS,Walt Disney Company,stock,NYSE,TMO,Thermo Fisher Scientific Inc.,stock,NYSE,ABT,Abbott Laboratories,stock,NYSE,ACN,Accenture plc,stock,NYSE,LIN,Linde plc,stock,NASDAQ,MCD,McDonald's Corporation,stock,NYSE,CSCO,Cisco Systems Inc.,stock,NASDAQ,INTC,Intel Corporation,stock,NASDAQ,QCOM,QUALCOMM Incorporated,stock,NASDAQ,TXN,Texas Instruments Incorporated,stock,NASDAQ,IBM,International Business Machines Corporation,stock,NYSE,NOW,ServiceNow Inc.,stock,NYSE,INTU,Intuit Inc.,stock,NASDAQ,AMAT,Applied Materials Inc.,stock,NASDAQ,MU,Micron Technology Inc.,stock,NASDAQ,LRCX,Lam Research Corporation,stock,NASDAQ,ADI,Analog Devices Inc.,stock,NASDAQ,PANW,Palo Alto Networks Inc.,stock,NASDAQ,CRWD,CrowdStrike Holdings Inc.,stock,NASDAQ,SNOW,Snowflake Inc.,stock,NYSE,PLTR,Palantir Technologies Inc.,stock,NASDAQ,NET,Cloudflare Inc.,stock,NYSE,DDOG,Datadog Inc.,stock,NASDAQ,UBER,Uber Technologies Inc.,stock,NYSE,ABNB,Airbnb Inc.,stock,NASDAQ,SHOP,Shopify Inc.,stock,NYSE,WFC,Wells Fargo & Company,stock,NYSE,GS,Goldman Sachs Group Inc.,stock,NYSE,MS,Morgan Stanley,stock,NYSE,C,Citigroup Inc.,stock,NYSE,BLK,BlackRock Inc.,stock,NYSE,SCHW,Charles Schwab Corporation,stock,NYSE,AXP,American Express Company,stock,NYSE,BX,Blackstone Inc.,stock,NYSE,SPGI,S&P Global Inc.,stock,NYSE,PFE,Pfizer Inc.,stock,NYSE,DHR,Danaher Corporation,stock,NYSE,BMY,Bristol-Myers Squibb Company,stock,NYSE,AMGN,Amgen Inc.,stock,NASDAQ,GILD,Gilead Sciences Inc.,stock,NASDAQ,CVS,CVS Health Corporation,stock,NYSE,MDT,Medtronic plc,stock,NYSE,ISRG,Intuitive Surgical Inc.,stock,NASDAQ,LOW,Lowe's Companies Inc.,stock,NYSE,NKE,NIKE Inc.,stock,NYSE,SBUX,Starbucks Corporation,stock,NASDAQ,TGT,Target Corporation,stock,NYSE,PM,Philip Morris International Inc.,stock,NYSE,MO,Altria Group Inc.,stock,NYSE,MDLZ,Mondelez International Inc.,stock,NASDAQ,CL,Colgate-Palmolive Company,stock,NYSE,COP,ConocoPhillips,stock,NYSE,SLB,Schlumberger Limited,stock,NYSE,BA,Boeing Company,stock,NYSE,CAT,Caterpillar Inc.,stock,NYSE,DE,Deere & Company,stock,NYSE,GE,GE Aerospace,stock,NYSE,HON,Honeywell International Inc.,stock,NASDAQ,UPS,United Parcel Service Inc.,stock,NYSE,FDX,FedEx Corporation,stock,NYSE,LMT,Lockheed Martin Corporation,stock,NYSE,RTX,RTX Corporation,stock,NYSE,MMM,3M Company,stock,NYSE,UNP,Union Pacific Corporation,stock,NYSE,T,AT&T Inc.,stock,NYSE,VZ,Verizon Communications Inc.,stock,NYSE,TMUS,T-Mobile US Inc.,stock,NASDAQ,CMCSA,Comcast Corporation,stock,NASDAQ,F,Ford Motor Company,stock,NYSE,GM,General Motors Company,stock,NYSE,RIVN,Rivian Automotive Inc.,stock,NASDAQ,COIN,Coinbase Global Inc.,stock,NASDAQ,HOOD,Robinhood Markets Inc.,stock,NASDAQ,SOFI,SoFi Technologies Inc.,stock,NASDAQ,MSTR,Strategy Incorporated,stock,NASDAQ,DELL,Dell Technologies Inc.,stock,NYSE,MRVL,Marvell Technology Inc.,stock,NASDAQ,SMCI,Super Micro Computer Inc.,stock,NASDAQ,ES=F,S&P 500 E-Mini Futures,future,CME,NQ=F,Nasdaq 100 E-Mini Futures,future,CME,YM=F,Dow E-Mini Futures,future,CME,RTY=F,Russell 2000 E-Mini Futures,future,CME,CL=F,Crude Oil Futures (WTI),future,NYMEX,BZ=F,Brent Crude Oil Futures,future,NYMEX,GC=F,Gold Futures,future,COMEX,SI=F,Silver Futures,future,COMEX,HG=F,Copper Futures,future,COMEX,NG=F,Natural Gas Futures,future,NYMEX,