Add SEO, agent-readable surfaces and a static browse page; fix date sort

The site had no sitemap, no robots.txt, no favicon, no meta description and
no social tags, and its default sort was noise. Vercel deploys docs/ on push
to main, so the committed build output is the deployed artefact.

Discoverability:

- astro.config.mjs now sets `site`, without which canonical URLs, Open Graph
  URLs and @astrojs/sitemap all silently produce nothing.
- BaseLayout emits canonical, Open Graph, Twitter card, favicon and optional
  JSON-LD; every page passes a real description. Repo pages carry
  SoftwareSourceCode schema, /browse/ carries an ItemList.
- Added sitemap (253 URLs), robots.txt, 1200x630 og-image, favicon set.

Agent-readable:

- /llms.txt, generated at build time from tagged_repos.json so it cannot drift
  from the index. Lists every repo by category plus the JSON endpoints and the
  marketplace install commands.
- robots.txt points agents at the JSON surfaces.

Navigation:

- /browse/ is a static listing of all 236 repos. The home page renders its list
  client-side, so a crawler without JS saw "Loading repositories..." and found
  no repo links at all. This puts them in the initial HTML.
- Hero now leads with the plugin marketplace: a CTA to the install guide and the
  `/plugin marketplace add` command inline.

Accessibility:

- Skip link, visible focus rings, prefers-reduced-motion, labelled search input,
  aria-live result count, aria-expanded on the nav toggle and dropdown, Escape
  closes menus and restores focus, rel="noopener" on external links.

Fixes:

- The "Newest" sort was meaningless. created_date fell back to added_date, which
  fell back to *today* on every build, because site_state.json history is capped
  at 50 entries and older repos fall off it. 125 of 236 repos claimed they were
  created 2026-08-10. Real GitHub creation dates are now fetched into
  data/repo_created_dates.json by the new scripts/fetch_created_dates.py, and
  added dates persist in data/repo_added_dates.json.
- 26 links on the ideas/ essay pages pointed at repo detail pages that are never
  generated. All 24 target repos are gone from GitHub entirely, confirmed against
  all 2444 repos on the account including private. Unlinked, prose kept.
- scripts/check_links.py now gates the build on broken internal links, since
  dropping a repo from the index silently 404s every essay-page mention of it.

notes/dead-repo-mentions.md records the deleted repos and the editorial cleanup
still outstanding. CLAUDE.md documents the deploy path, the SEO surfaces, why
/browse/ must stay static, and the date persistence rule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: danielrosehill

CLAUDE.md

@@ -310,6 +310,56 @@ either:
 Compare results against the category files before adding anything; the script
 does not know which repos were deliberately excluded.
 
+## The deployed site
+
+Live at **https://claude.danielrosehill.com**, deployed by **Vercel** on push to
+`main` (`vercel.json` sets `outputDirectory: docs`). `docs/` is the committed
+Astro build output — so **run `npm run build` and commit the result**; this repo
+is not one where the build can be left to CI.
+
+`npm run build` is three steps: `build:data` (the Python pipeline that regenerates
+`README.md`, the split markdown pages and `data/*.json`), `build:site` (Astro), and
+`check:links`, which fails the build on a broken internal link.
+
+### SEO and machine-readable surfaces
+
+`astro.config.mjs` sets `site:` — this is load-bearing. Without it canonical URLs,
+Open Graph URLs and `@astrojs/sitemap` all silently produce nothing.
+
+`BaseLayout.astro` takes `description`, `image`, `isHome` and `schema` props and
+emits the canonical link, Open Graph and Twitter tags, and JSON-LD. **Pass a
+`description` on every new page** — the fallback is the site-wide one, which is
+fine for a stray page but poor if it ends up on many.
+
+| Surface | Source | Notes |
+| --- | --- | --- |
+| `/sitemap-index.xml` | `@astrojs/sitemap` | every page, generated |
+| `/robots.txt` | `public/robots.txt` | static; points at the sitemap |
+| `/llms.txt` | `src/pages/llms.txt.ts` | generated from `tagged_repos.json`, cannot drift |
+| `/browse/` | `src/pages/browse/index.astro` | static listing of every repo |
+| `/og-image.png` | `public/` | 1200×630 |
+
+**Why `/browse/` exists:** the home page renders its repo list client-side from
+`tagged_repos.json`, so a crawler that does not run JS sees only "Loading
+repositories…" and finds no repo links at all. `/browse/` is the static
+counterpart that puts all 236 in the initial HTML. Keep it working; do not fold it
+into the client-rendered index.
+
+### Dates: both are persisted, deliberately
+
+`created_date` (real GitHub creation date, `data/repo_created_dates.json`, filled
+by `scripts/fetch_created_dates.py`) drives the site's default "Newest" sort.
+`added_date` (`data/repo_added_dates.json`) is when the repo entered this index.
+
+Both files exist because both values used to be recomputed as *today* on every
+build whenever a lookup missed — `site_state.json` history is capped at 50 entries,
+so older repos fell off and got restamped. On 2026-08-10 that had left 125 of 236
+repos claiming they were created that day, which made the default sort meaningless.
+Never reintroduce a `datetime.now()` fallback that is not written back to disk.
+
+After adding repos, run `python3 scripts/fetch_created_dates.py` before building,
+or the new entries sort to the top with a fabricated date.
+
 ## This repo's own name
 
 Canonical slug: **`danielrosehill/Claude-Code-Projects-Index`**. It has been

astro.config.mjs

@@ -1,8 +1,13 @@
 import { defineConfig } from 'astro/config';
+import sitemap from '@astrojs/sitemap';
 
+// `site` is required for canonical URLs, Open Graph tags and the sitemap —
+// without it Astro emits relative URLs only and @astrojs/sitemap is a no-op.
 export default defineConfig({
+  site: 'https://claude.danielrosehill.com',
   outDir: './docs',
   build: {
     format: 'directory',
   },
+  integrations: [sitemap()],
 });