Skip to content

UPLOAD_BEHAVIOR.md

Latest commit

 

History

History
374 lines (343 loc) · 24.6 KB

UPLOAD_BEHAVIOR.md

File metadata and controls

374 lines (343 loc) · 24.6 KB
Raw

Upload Behavior

Date: 2026-08-21

This document records the current upload truth exposed by the package. It does not claim that every clone board is fully verified.

Native USB group

These boards currently declare USB-backed upload support in package metadata:

  • promicro_nrf52840
  • nicenano_v2
  • supermini_nrf52840
  • nrfmicro_nrf52840
  • mini_nrf52840
  • xiao_nrf52840
  • devboard_nrf52840
  • pitaya_go_nrf52840
  • usb_dongle_nrf52840

SWD upload choices

Boards with SWD upload support expose explicit Arduino IDE Upload Method entries for the probe type:

  • SWD programmer (CMSIS-DAP) uses OpenOCD with tools/openocd/nrf52-cmsis-dap.cfg.
  • SWD programmer (SEGGER J-Link) uses SEGGER JLink.exe through upload.ps1.

These entries are for the normal Upload button. They do not depend on the IDE Tools -> Programmer selection, because Arduino upload recipes use the board's selected Upload Method properties. Use Tools -> Programmer for Sketch -> Upload Using Programmer and Tools -> Burn Bootloader. SEGGER J-Link (SWD) uses SEGGER's command-line tools; CMSIS-DAP (SWD) uses OpenOCD for locked-target recover. For locked nRF52 parts, the recover/erase step is what clears APPROTECT; the bootloader HEX is flashed afterward.

J-Link transactions run non-interactively with automatic connection, GUI dialogs disabled, and exit-on-error enabled. On a host with more than one probe, set NIUS_JLINK_SERIAL to the exact SEGGER serial number or nickname. For CMSIS-DAP, set NIUS_CMSIS_DAP_SERIAL to the exact adapter serial. These selectors are passed before target initialization; they are not inferred from a board COM port. Firmware paths embedded in OpenOCD commands are Tcl-escaped before any probe is opened.

Normal SWD upload, Upload Using Programmer, and Burn Bootloader all use the same validated wrappers on Windows, Linux, and macOS; the former raw OpenOCD mutation recipes are not exposed as an alternate bypass. Before a bootloader recover/erase, the complete Intel HEX must pass checksum/overlap/EOF checks, fit the selected physical flash and UICR ranges, provide valid MBR and bootloader vectors for the selected SRAM, contain a valid page-aligned UICR bootloader address, and leave UICR.APPROTECT erased. Only then may nrf52_recover run.

Arduino IDE 2 SWD Debug follows the selected Upload Method: CMSIS-DAP keeps the OpenOCD server, while the J-Link upload method switches the debug server metadata to Arduino IDE's jlink server type.

The devboard_nrf52833 target is SWD-first and exposes only these SWD upload choices because it has no native USB upload path in this package.

Current Windows wrapper behavior

  • tools/niusrobotlab/upload.ps1 owns the touch/reset sequence on Windows instead of relying on the Arduino CLI default touch path.
  • Arduino board recipes keep upload.use_1200bps_touch=false; this prevents Arduino CLI from issuing a second, unscoped touch before the identity-aware wrapper runs.
  • Bootloader / DFU -> Auto-detect prefers a matching UF2 mass-storage volume when the selected board is already in bootloader mode. Explicit serial-DFU menu entries still use the packaged serial-DFU tool.
  • Auto-detection never treats a known application PID as a bootloader. Bootloader IDs shared by more than one flash layout require scoped INFO_UF2.TXT evidence; when MSC metadata is unavailable, select the explicit serial-DFU layout instead of asking the uploader to infer an application start.
  • UF2 drives are matched to the selected serial port by stable USB identity. If two boards expose the same volume label, the wrapper refuses ambiguous matches instead of choosing the first drive.
  • Selecting a USER CDC never permits a VID/PID-only jump to another board's SERVICE CDC. The uploader and USB GDB bridge require the service interface to share the selected interface's composite parent identity; a missing, unscoped, or ambiguous sibling fails before reset or write.
  • For bootloaders whose USB PID differs from the application PID, the wrapper accepts the selected board's scoped UF2 volume as transition proof. It never waits for the old runtime COM to return before copying firmware; that COM can only return after the copy and reboot have completed.
  • On Windows, runtime verification reads only the expected VID/PID branch and confirms that its COM endpoint is currently present. Stale registry records are rejected. Any required full PnP snapshot uses a two-second CIM operation bound and returns no candidates on provider failure; it never falls into the unbounded Get-PnpDevice walk. The UF2 evidence observed during touch is reused instead of rediscovering the same volume before transfer.
  • Windows interface-parent attribution joins the MI child's embedded parent prefix to the composite instance's Enum\USB ParentIdPrefix; it does not call the unbounded PnP property provider. A temporarily absent mapping is not cached as a negative identity, so later bounded discovery can observe the completed composite; only a nonempty unique mapping is cached.
  • When a UF2 volume is visible, upload.ps1 reads INFO_UF2.TXT and uses the SoftDevice field to infer the mounted layout (0x1000, 0x26000, or 0x27000). If that layout conflicts with the app start used by the selected Arduino IDE Bootloader / DFU option, upload fails before copying firmware. This applies to UF2 deploy, serial DFU, and the UF2-to-serial fallback path. Serial-only bootloaders do not expose mass storage, so an explicit UF2 menu choice may use the selected board's identity-scoped maintenance CDC when the matching UF2 volume is not mounted. That fallback is admitted only after a confirmed reset transition and uses the recipe's exact app start and SoftDevice requirement; it does not guess either value. If a matching INFO_UF2.TXT is available, it is still checked and any mismatch remains terminal before firmware is written. Auto-detection cannot use this fallback for a shared bootloader ID without scoped UF2 layout evidence. Arduino compiles before upload, so the wrapper cannot safely relocate an already-linked image.
  • Before any Windows USB discovery or touch, the uploader independently validates every Intel HEX record checksum and length, rejects overlaps or records after EOF, and requires the actual vector table, Thumb reset entry, exact target stack pointer, start address, and highest programmed byte to fit the selected target's exact flash and SRAM limits. The same validated image is used for serial DFU, UF2, OpenOCD, and J-Link application upload; recipe metadata alone is not accepted as proof that the compiled image is safe to transfer.
  • Layout guard (layout failure): compares the IDE Bootloader / DFU app start (0x1000 / 0x26000 / 0x27000) against INFO_UF2.TXT on the selected board's UF2 drive (matched by stable USB identity, not drive letter). This check is mandatory.
  • Misflash guard (misflash failure): after serial DFU or direct UF2, waits for the selected board to return in application mode. The fast path checks the expected VID/PID and preserved physical-device identity without opening the newly enumerated application COM. If USB never comes back (typical when app start was wrong), the wrapper fails without issuing another touch or transfer and reports explicit recovery choices. This check is mandatory. A reused COM number is not sufficient evidence: the endpoint must match the board recipe's runtime VID/PID. This prevents the departing bootloader CDC from being mistaken for a successfully started application.
  • Manual UF2 drag in DFU mode is not guarded. Copying a .uf2 from Explorer bypasses upload.ps1. You must match bootloader layout, not just the version string in INFO_UF2.TXT (for example 0.6.0 exists in both S140 @0x26000 and no-SoftDevice @0x1000 variants). Sketch UF2 must match the mounted layout. Adafruit update-* bootloader packages (family 0xd663823c) rewrite the bootloader and reboot into application mode; they are not sketch images. After a layout switch, also flash a matching app or USB may disappear. See ../bootloaders/README.md.
  • USB silent / COM missing after a bad or partial flash: the host cannot 1200-touch a port that is gone. Recovery is manual: double-tap RESET to re-enter UF2 (on boards without a reset button, short RST to GND twice quickly, like a double-tap), then fix the Bootloader / DFU menu and upload again; or recover over SWD (Tools → Programmer → SEGGER J-Link (SWD) or CMSIS-DAP, then Burn Bootloader / sketch upload). See ../bootloaders/README.md.
  • Upload Method → Enter UF2 drive only (no upload) performs the touch/bootloader wait, reports the matched drive, and exits before copying firmware.
  • If the selected upload COM is stale after a mode change, the wrapper fails with a clear "re-select the current SERVICE/DFU port" message instead of falling through to another board.
  • ProMicro-class application firmware uses runtime PID 0x00B4; PID 0x00B3 belongs to the UF2 bootloader. Keep this split even for no-SoftDevice builds: the bootloader exposes interface 2 as mass storage, while TaichiUSB exposes it as the user CDC port. Reusing one PID and chip serial for both descriptor layouts makes Windows retain the wrong per-interface driver binding.
  • After serial DFU, the upload wrapper accepts the selected runtime interface when it returns with the expected runtime VID/PID and physical-device identity. Windows may assign a different number when the bootloader and application expose different composite interfaces; that remap is not an upload failure.
  • Serial DFU performs exactly one identity-bound mutating transfer per upload invocation. A timeout, cable loss, protocol failure, or interrupted write is terminal and keeps its original error; the wrapper never infers that a second single-bank transfer or an inferred SoftDevice requirement is safe.
  • Each serial-DFU board/layout recipe must declare its exact sd-req. A missing or malformed value is terminal; the uploader does not infer a SoftDevice generation from the MCU type. 0xFFFE is the declared compatibility value used by supported no-SoftDevice layouts, not an inferred retry value.
  • That sole transfer is launched only after the exact bootloader maintenance CDC is proven in the selected physical scope. An unconfirmed touch and an unresolved bootloader port are terminal; the old runtime COM is never used as a speculative DFU endpoint.
  • When application and bootloader share one VID/PID, a matching COM node alone cannot prove its role. The uploader additionally requires a confirmed reset transition or stronger identity-scoped bootloader evidence; there is no environment-variable override that converts an application COM into proof.
  • Linux and macOS do not delegate 1200-baud transition ownership to adafruit-nrfutil: upstream sleeps and reopens the same tty without checking physical identity. ArduinoNRF sends one DTR edge, resolves exactly one bootloader maintenance endpoint in the captured sysfs/IOUSB scope, re-proves it, and launches the transfer without --touch.
  • The runtime DFU interface is hidden by default. Hands-free upload uses the service CDC's 1200-bps touch, so the extra driverless DFU-runtime node is not needed for ordinary use. Enable it explicitly only for a workflow that sends USB DFU runtime requests directly.
  • A runtime that is powered and attached but not configured no longer reboots itself into the bootloader after an ambient 60-second timeout. Bootloader entry requires an explicit maintenance-CDC 1200-bps/DTR-drop transaction, an admitted DFU detach request, or an external reset/debug action. A sleeping, disconnected, suspended, driver-less, or slow host therefore cannot turn an otherwise healthy application into a different USB identity by elapsed time.
  • The previous service-port "boot token" fallback (~NIUSBL!42\r after arming with line coding 134/8/2/2 + DTR+RTS) has been removed — the standard 1200 bps touch path is now the single primary trigger.
  • Pre-existing adafruit-nrfutil processes are never killed or treated as owned merely because their parent exited. The wrapper fails before touching USB; timeout cleanup may terminate only the exact child process tree launched by this upload.
  • The Boards Manager tool is preferred. An automatically discovered Conda copy is rejected, avoiding an implicit protocol-implementation substitution. A separately verified tool can still be selected by an explicit executable path.
  • Every Windows child tool is launched with CRT-correct argument escaping, including empty arguments, embedded quotes, and paths whose final character is a backslash. Verbose stdout and stderr are drained concurrently so a full pipe cannot freeze an otherwise completed OpenOCD, J-Link, package-generation, or conversion process.
  • OpenOCD and J-Link have finite 120-second process deadlines; local image-preflight and UF2-conversion helpers have a finite 60-second deadline. Serial DFU has a 240-second process deadline and a 30-second no-progress deadline. Advanced sessions may tune these within finite ranges: serial DFU process 10-600 seconds, serial DFU idle 1-120 seconds, OpenOCD/J-Link 1-600 seconds, and local helpers 1-300 seconds. Zero, negative, malformed, and out-of-range values are ignored; no setting disables a deadline. Timeout cleanup remains limited to the exact child tree launched by the current upload. The same PID-scoped cleanup runs from the command's finally boundary on cancellation or an unexpected host exception, so those exits cannot strand a launched helper on a COM/probe endpoint. Process-tree discovery itself has a two-second CIM operation bound; if descendant enumeration is unavailable, cleanup still targets the exact launched root PID rather than blocking indefinitely or widening ownership.
  • UF2 disk/volume attribution and post-transition mass-storage problem inspection use the same two-second CIM operation bound. A stalled storage or PnP provider cannot hold upload indefinitely, and missing diagnostic evidence is never converted into target identity or upload success.
  • Windows always takes both a selected-target upload lock and a host-wide DFU serialization lock before any target-visible action. Their wait intervals may be tuned only within 0-10 seconds and 0-600 seconds respectively; lock creation or acquisition failure is terminal before USB is touched.
  • Open/transition/settle tuning is finite and range-checked. The uploader always uses identity-aware adaptive settlement; a caller cannot replace it with an unbounded or blind fixed sleep. A missing concrete endpoint or a port that never remains present and openable through the stability window is terminal before transfer.
  • Each upload invocation makes at most one 1200-bps/DTR touch attempt. A helper timeout, partial host result, or lost confirmation is terminal rather than a reason to pulse a board that may already be resetting.
  • Generated serial-DFU ZIP and UF2 files are unique, invocation-owned temporary artifacts and are removed on both success and failure. The wrapper never reuses a package left by an earlier incomplete upload.

Linux and macOS wrapper behavior

  • upload.py accepts the selected port only when it matches either the board recipe's runtime identity or its bootloader identity; the script then owns the single 1200-bps/DTR transition itself and disables the transport tool's fixed-name touch.
  • The Arduino recipe passes the selected bootloader transport explicitly. Adafruit serial-DFU and Adafruit-compatible UF2 layouts use the verified serial service. auto and Nordic USB DFU fail before image or USB access rather than silently invoking an incompatible protocol; use an explicit supported layout or SWD until those distinct transports have their own verified wrappers.
  • The script validates the actual Intel HEX framing, vector table, link address, target-specific SRAM ceiling, and maximum application range before resolving or touching a USB device.
  • A distinct bootloader VID/PID that is already selected is not sent another 1200-bps reset. Same-identity runtime/bootloader layouts retain the guarded touch because endpoint identity alone cannot prove their current role.
  • OpenOCD application upload uses the same preflight contract, a finite invocation-owned process group, and protocol verify before reporting success. Package generation and serial DFU use that same ownership boundary; timeout, Ctrl+C, or an unexpected wait failure boundedly terminates and reaps only the invocation's complete process group before preserving the original failure.
  • On a dual-CDC runtime, a selected user endpoint is remapped to interface zero only when Linux sysfs or the macOS IOUSB registry proves it is a sibling on the same USB composite. Ambiguous mappings fail closed.
  • A successful transfer is not sufficient by itself. When available, Linux sysfs or the macOS IOUSB registry must return the selected physical USB topology with the declared runtime VID/PID before upload succeeds. Exact USB serial is the fallback when topology is unavailable, so serial-less boards and bootloader/runtime serial changes remain scoped without allowing a peer board to satisfy the check.
  • The same runtime identity must remain continuously present for 300 ms by default; a transient enumeration followed by an early application/watchdog failure is not reported as upload success. --runtime-stable-ms may tune this bounded check only within 100..5000 ms; zero cannot disable it.
  • When the host exposes CDC interface numbers, post-upload verification requires exactly one maintenance interface zero. A surviving user CDC alone is not accepted as proof that the hands-free upload path was restored. Hosts that omit interface metadata fall back to the exact physical USB topology/serial identity.
  • When bootloader and application share one VID/PID, Linux/macOS additionally require a new host enumeration session after transfer. A surviving bootloader CDC from the old session is never accepted as application return; hosts that cannot expose a session token fail closed for this ambiguous layout.
  • Windows likewise requires one identity-scoped maintenance COM to remain present for at least 300 ms; a composite parent or USER CDC alone is insufficient. A shared bootloader/runtime VID/PID also requires the bootloader MSC interface to be gone. The optional UF2-to-serial fallback is unavailable without a selected- target composite identity.
  • One host-local advisory lock owns a physical topology/serial identity throughout package generation, transfer, and runtime verification. A concurrent uploader fails before touching that target, while the OS releases the lock after exit.
  • Linux and macOS re-prove the selected physical identity after acquiring that lock and re-resolve its maintenance endpoint before any touch or transfer. They prove it again after package generation, immediately before the first target-visible action. A detach, tty renumber, session replacement, peer replacement, or stale pre-existing bootloader endpoint fails closed.
  • Linux devnum and the macOS device-level registry session are used as enumeration tokens when available. A same-VID/PID bootloader may satisfy a fast transition without a sampled absence only after that token changes; the selected port and token must then remain stable through the bounded settle interval.
  • USB IDs, address ranges, process/runtime timeouts, DFU/touch baud rates, device type, and SoftDevice requirement are finite and range-checked before transfer; malformed touch input cannot silently disable the bootloader transition.
  • Ambiguous same-identity devices fail closed instead of allowing a peer board to satisfy target detection or post-upload verification.
  • Generic boards whose base recipe says auto must select an explicit Bootloader / DFU identity on Linux/macOS; the cross-platform uploader does not guess a bootloader VID/PID after the application port disappears.

Probe and debugger image safety

  • The link recipe constrains ELF load-segment alignment to the nRF flash page size. An application linked for 0x0, 0x1000, 0x26000, or 0x27000 therefore has its first file-backed PT_LOAD segment at that exact address; debugger tools cannot interpret alignment padding as an earlier flash load and erase a SoftDevice prefix.
  • A debugger should still validate ELF program headers, not only section addresses, and protect the MBR/SoftDevice plus bootloader ranges in its target profile. Arduino USB upload continues to use the independently validated HEX image.
  • Startup, SoftDevice, fault, and USB diagnostic words live in the linker's .noinit allocation. No diagnostic uses a guessed absolute RAM address.
  • No-SoftDevice profiles link data at 0x20000000 and expose the full 256 KiB RAM. SoftDevice profiles retain the required 0x20006000 RAM origin.

USB-safe idle and power-down behavior

  • Native USB remains interrupt-driven during ordinary System ON WFI idle; applications do not need to busy-wait merely to preserve enumeration.
  • Generic nrfSystemPowerDown() refuses SystemOFF whenever VBUS is present, including the pre-configuration and USB-suspend windows. Entering SystemOFF in either window can leave no new VBUS edge to wake the MCU.
  • NrfPower::enterSystemOff() applies the same safe default. A caller that intentionally accepts disconnecting an already powered USB session must say so explicitly with enterSystemOff(true).

Real Promicro-class board result

What is now working

  • first upload from manual bootloader mode works on the user's board
  • the board returns to user mode afterward
  • with usbcdc=disabled, the board keeps a single visible SERVICE CDC path
  • UF2 upload from the current DFU port works in both bootloader=auto and explicit UF2 menu modes
  • explicit Adafruit serial DFU from the current DFU port works and is not confused by a mounted UF2 volume
  • with two boards simultaneously mounted as NICENANO, the selected board maps to its own volume by stable USB identity
  • selecting a stale COM after the board re-enumerates is rejected before any upload can target another board
  • a second upload from user mode works — the 1200 bps touch triggers NVIC_SystemReset() into the bootloader, the selected transport streams the image, and the board re-boots into user mode.
  • The maintenance CDC accepts a DTR falling edge that follows a valid 1200-bps request even if the host has already restored its normal baud. Once that explicit edge starts the bounded confirmation window, a baud replay cannot cancel it. USB reset/disconnect clears the recent-1200 evidence so a later session cannot inherit upload authority.
  • Malformed CDC framing fields are stalled before either CDC line-coding state or the upload gesture state changes.

What previously failed (historical)

Before the V1 firmware fixes, adafruit-nrfutil would stall at Sending DFU start packet during the second upload because the firmware never re-entered bootloader in response to the 1200 bps touch. The host then saw Port never detached after touch (port stayed present) repeatedly across the four host-side trigger mechanisms upload.ps1 used to try.

Root cause: three cooperating firmware bugs in NrfUsbd.cpp — EP0 OUT EasyDMA never triggered, EP0 OUT direction routed by stale BMREQUESTTYPE, and a subsequent DTR=true cancelling serviceTouchPending_ inside the 40 ms confirm window.

usbcdc=disabled in-app upload (host-side fix)

For a while, an in-app upload to a usbcdc=disabled board stalled at adafruit-dfu even though the firmware touch path was correct. Root cause was host-side: with no user CDC, the runtime DFU "Bootloader Control" vendor interface lands on MI_02 (where usbcdc=enabled puts the user CDC). upload.ps1 counted any non-Ports MI_02 interface as MSC/bootloader evidence, decided the board was already in the bootloader, skipped the 1200-touch entirely, and ran adafruit-nrfutil against the still-running app. The fix excludes the Bootloader Control interface from that evidence (the real bootloader is identified by its UF2 mass-storage volume, never by this control interface), so the touch runs and the board reboots normally. Verified 3× back-to-back plus usbcdc transitions in both directions.

What still needs validation

  • Boards beyond the user's ProMicro clone that share the firmware path
  • Linux/macOS physical serial-DFU and UF2 parity; the Python wrapper has identity-scoped host tests but no target-backed parity result in this revision