Merge pull request #714 from vormadal/claude/notes-markdown-import-8kffgy

Add Markdown import to the notes importer

Author: inur93

CLAUDE.md

@@ -174,7 +174,16 @@ relationships resolved without registering OOXML namespaces — matched by their
 literal `w:`-prefixed tag/attribute names, since that's how the browser's
 `DOMParser` preserves them), then run through `generateJSON(html,
 noteExtensions)` — the same schema the editor itself uses — so an imported
-document can never contain something the editor can't open.
+document can never contain something the editor can't open. A `.md`/`.markdown`
+file takes the same route via `marked` (`breaks: true`, so a note's line breaks
+survive as `<br>` and every remaining newline sits strictly between block tags),
+followed by a `DOMParser` pass that degrades what the schema has no node for:
+tables flatten to one paragraph per cell (as in `docx.ts`), GFM task items become
+`☐`/`☑` text, and image references — markdown embeds no bytes — are kept only
+when absolute, with relative paths dropped and warned about. YAML front matter is
+stripped, its `title` preferred over the filename. The three-stage pipeline, and
+what adding a fourth format takes, are documented in
+[`anything-frontend/src/lib/agent.md`](anything-frontend/src/lib/agent.md).
 
 All endpoints are under `/api/somethings`:
 - `GET /` — List all (non-deleted)
@@ -219,6 +228,7 @@ Non-obvious gotchas:
 - **A shared helper under `Anything.Application.Features` must not be named `*Query`, `*Command` or `*Handler`.** `CqrsPatternTests` and `NamingConventionTests` assert that *every* type in that namespace with one of those suffixes implements `IRequest`/`IRequestHandler`, so a plain static helper called e.g. `InventoryThumbnailQuery` fails the architecture tests with nothing wrong in the code itself. Name it for what it does (`InventoryThumbnailLookup`, `*Mapping`).
 - **Filter a nullable FK with a `List<int?>`, not `.Value`.** `ids.Contains(a.ItemId)` where `ids` is `List<int?>` translates to a plain `= ANY(@ids)`; `ids.Contains(a.ItemId.Value)` leans on EF's nullable unwrapping and can throw at translation time. Nothing local catches this — the unit tests' `AsAsyncQueryable` provider is LINQ-to-objects and executes untranslatable expressions happily, and `dotnet build` has no opinion. Same class of trap as the raw-SQL translations `SearchEndpointTests` exists to cover.
 - **Adding a constructor parameter to a query handler breaks its unit tests at compile time**, since `InventoryHandlerTests` and friends construct handlers positionally. A repository substitute also needs `.Query()` stubbed (`Substitute.For<IRepository<T>>()` returns null otherwise, and `ToListAsync` NREs) — `InventoryTestDoubles` in `InventoryHandlerTests.cs` is the shared factory for that.
+- **An ESM-only npm package breaks Jest, and `transformIgnorePatterns` can't fix it — add it to `transpilePackages` in `next.config.ts`.** Packages shipping only ESM (`marked`, `@microsoft/kiota-abstractions`) fail with `SyntaxError: Unexpected token 'export'` in Jest, because next/jest **hardcodes** `/node_modules/` as the first ignore pattern and only *appends* a custom `transformIgnorePatterns` (see `next/dist/build/jest/jest.js`) — a `/node_modules/(?!(pkg)/)` override in `jest.config.mjs` is silently outranked. Listing the package in `transpilePackages` is what makes next/jest emit `/node_modules/(?!.pnpm)(?!(pkg)/)` instead. `npm run build` passes either way, so only the Jest run catches it.
 - **Adding a key to `HomeCardKeys.All` breaks `HomePreferenceEndpointTests`**, which asserts the exact default card list and sort orders. Update it in the same change; the frontend's `DEFAULT_HOME_CARD_ORDER` in `anything-frontend/src/app/HomeCards.tsx` mirrors the same list and needs the matching entry.
 - **Model-changing pushes break `update-api-client` on the first push — this is expected.** When a push adds/changes an entity, `update-ef-migrations` and `update-api-client` run in parallel on the *same* pre-migration commit. `update-api-client` boots the API, whose startup `MigrateAsync` escalates `PendingModelChangesWarning` to a fatal error because the migration doesn't exist yet, so Swagger never comes up and Kiota generation times out (exit 124). `update-ef-migrations` meanwhile commits the migration. To regenerate the client you must re-trigger `update-api-client` on a commit that *already contains* the migration — and it only triggers on `src/Anything.{API,Contracts,Application,Core}/**` (NOT `src/Anything.Database/**`, where migrations live). Fix: after the migration lands, make a small backend-path change (e.g. XML-doc the new contracts) in your next push, then pull/rebase the regenerated client before validating the frontend.
 - **Snapshot bot `[skip ci]` commits don't re-trigger PR checks.** The PR's `Visual Snapshot Tests` check does NOT rerun on the baseline-containing commit — the PR stays red with the pre-baseline failure as its latest check. After the bot commit lands, push a small non-`[skip ci]` commit (docs, comment) to sync the PR and rerun checks on a tip that includes the baselines.

anything-frontend/e2e/visual.spec.ts

@@ -1123,7 +1123,8 @@ test.describe("Visual Snapshots - Authenticated Pages", () => {
   test("note import - pick files", async ({ page }) => {
     await page.goto("/notes/import");
     await page.waitForLoadState("networkidle");
-    await expect(page.getByText("Choose exported files")).toBeVisible();
+    await expect(page.getByRole("heading", { name: "Exporting from Samsung Notes" })).toBeVisible();
+    await expect(page.getByText(".txt, .md or .docx — you can pick several at once")).toBeVisible();
     await expect(page).toHaveScreenshot("note-import-pick.png", screenshotOptions);
   });
 
@@ -1141,6 +1142,22 @@ test.describe("Visual Snapshots - Authenticated Pages", () => {
     await expect(page).toHaveScreenshot("note-import-review.png", screenshotOptions);
   });
 
+  test("note import - markdown review", async ({ page }) => {
+    const markdown = "# Weekend\n\n- [x] book flight\n- [ ] pack\n\n![map](./img/map.png)";
+    await page.goto("/notes/import");
+    await page.waitForLoadState("networkidle");
+    await page.locator('input[type="file"]').setInputFiles([
+      { name: "Weekend plans.md", mimeType: "text/markdown", buffer: Buffer.from(markdown) },
+      { name: "Reading list.markdown", mimeType: "text/markdown", buffer: Buffer.from("- Dune\n- Solaris") },
+    ]);
+    await expect(page.getByText("Weekend plans")).toBeVisible();
+    await expect(page.getByText("Reading list")).toBeVisible();
+    // The relative image path can't be resolved from an exported .md, so this
+    // row carries the dropped-image warning while the other one doesn't.
+    await expect(page.getByText("Images stored next to this file weren't imported.")).toBeVisible();
+    await expect(page).toHaveScreenshot("note-import-markdown-review.png", screenshotOptions);
+  });
+
   test("home page - notes card", async ({ page }) => {
     await page.route("**/api/home/card-preferences**", (route) => {
       if (route.request().method() === "GET") {

anything-frontend/e2e/visual.spec.ts-snapshots/note-import-markdown-review-visual-linux.png

@@ -4,7 +4,10 @@ const nextConfig: NextConfig = {
   output: "standalone",
   // Kiota ships its abstractions as ESM; transpiling it lets Jest (via next/jest)
   // transform the package when hooks import runtime helpers like DateOnly directly.
-  transpilePackages: ["@microsoft/kiota-abstractions"],
+  // `marked` (the notes markdown importer) is ESM-only for the same reason —
+  // next/jest hardcodes `/node_modules/` in transformIgnorePatterns unless a
+  // package is listed here, so a jest.config override alone can't reach it.
+  transpilePackages: ["@microsoft/kiota-abstractions", "marked"],
   async redirects() {
     return [
       { source: '/shopping-lists', destination: '/lists', permanent: true },

anything-frontend/e2e/visual.spec.ts-snapshots/note-import-pick-visual-linux.png

@@ -47,6 +47,7 @@
     "fflate": "^0.8.3",
     "idb-keyval": "^6.3.0",
     "lucide-react": "^1.27.0",
+    "marked": "^18.0.9",
     "next": "16.2.12",
     "react": "19.2.8",
     "react-dom": "19.2.8",

anything-frontend/next.config.ts

@@ -56,6 +56,32 @@ describe("ImportNotesPage", () => {
     );
   });
 
+  it("imports a markdown file as a formatted note", async () => {
+    const user = userEvent.setup();
+    renderWithClient(<ImportNotesPage />);
+
+    const recipe = new File(["# Pancakes\n\n- flour\n- eggs"], "Pancakes.md", { type: "text/markdown" });
+    await user.upload(pickFilesInput(), [recipe]);
+
+    await user.click(await screen.findByRole("button", { name: "Import 1 note" }));
+
+    await waitFor(() => expect(screen.getByText("1 note imported.")).toBeVisible());
+    expect(mockNotesPost).toHaveBeenCalledWith(
+      expect.objectContaining({ title: "Pancakes", contentJson: expect.stringContaining('"heading"') })
+    );
+  });
+
+  it("warns that a markdown file's local images can't come along", async () => {
+    const user = userEvent.setup();
+    renderWithClient(<ImportNotesPage />);
+
+    const withImage = new File(["Look:\n\n![kitchen](./img/kitchen.png)"], "House.md", { type: "text/markdown" });
+    await user.upload(pickFilesInput(), [withImage]);
+
+    expect(await screen.findByText("Images stored next to this file weren't imported.")).toBeVisible();
+    expect(screen.getByRole("button", { name: "Import 1 note" })).toBeEnabled();
+  });
+
   it("disables a file it can't parse and excludes it from the import count", async () => {
     renderWithClient(<ImportNotesPage />);
     const unsupported = new File(["%PDF"], "note.pdf", { type: "application/pdf" });
@@ -67,7 +93,7 @@ describe("ImportNotesPage", () => {
     Object.defineProperty(pickFilesInput(), "files", { value: [unsupported] });
     fireEvent.change(pickFilesInput());
 
-    expect(await screen.findByText("Only .txt and .docx files can be imported.")).toBeVisible();
+    expect(await screen.findByText("Only .txt, .md and .docx files can be imported.")).toBeVisible();
     expect(screen.getByText("Nothing selected")).toBeVisible();
   });
 

anything-frontend/package-lock.json

@@ -82,21 +82,24 @@ export default function ImportNotesPage() {
   if (phase === "pick") {
     return (
       <div className="container mx-auto max-w-lg space-y-4 px-4 py-4">
-        <PageTitle>Import from Samsung Notes</PageTitle>
+        <PageTitle>Import notes</PageTitle>
         <div className="space-y-4 rounded-lg border border-gray-200 bg-white p-4 shadow-sm sm:p-6 dark:border-gray-700 dark:bg-gray-800">
-          <ol className="list-inside list-decimal space-y-1 text-sm text-gray-600 dark:text-gray-400">
-            {INSTRUCTIONS.map((step) => (
-              <li key={step}>{step}</li>
-            ))}
-          </ol>
+          <div className="space-y-1">
+            <h2 className="text-sm font-medium text-gray-900 dark:text-white">Exporting from Samsung Notes</h2>
+            <ol className="list-inside list-decimal space-y-1 text-sm text-gray-600 dark:text-gray-400">
+              {INSTRUCTIONS.map((step) => (
+                <li key={step}>{step}</li>
+              ))}
+            </ol>
+          </div>
           <label className="flex flex-col items-center justify-center gap-2 rounded-lg border-2 border-dashed border-gray-300 p-8 text-gray-500 transition-colors hover:border-blue-500 dark:border-gray-600 dark:text-gray-400 dark:hover:border-blue-400">
             <Upload className="h-8 w-8" />
             <span className="text-sm font-medium">Choose exported files</span>
-            <span className="text-xs">.txt or .docx — you can pick several at once</span>
+            <span className="text-xs">.txt, .md or .docx — you can pick several at once</span>
             <input
               type="file"
               multiple
-              accept=".txt,.docx"
+              accept=".txt,.md,.markdown,.docx"
               className="sr-only"
               onChange={(e) => handleFilesSelected(e.target.files)}
             />

anything-frontend/package.json

@@ -10,6 +10,8 @@ import { buttonVariants } from "@/components/ui/button";
 import { cn } from "@/lib/utils";
 import { ChevronRight, Plus, Upload } from "lucide-react";
 
+const IMPORT_LABEL = "Import notes";
+
 export default function NotesPage() {
   const router = useRouter();
   const { setHeaderActions } = useHeaderActions();
@@ -20,8 +22,8 @@ export default function NotesPage() {
       <div className="ml-auto flex items-center gap-1">
         <Link
           href="/notes/import"
-          aria-label="Import from Samsung Notes"
-          title="Import from Samsung Notes"
+          aria-label={IMPORT_LABEL}
+          title={IMPORT_LABEL}
           className={cn(buttonVariants({ variant: "ghost", size: "icon" }))}
         >
           <Upload className="h-5 w-5" />