How to reproduce the Needle benchmark numbers on Cactus?

[32kilometre]

Hi Cactus team,

I am trying to reproduce the Needle performance numbers mentioned on the Hugging Face page:

https://huggingface.co/Cactus-Compute/needle

It says:

In production, Needle runs on Cactus at 6000 toks/sec prefill and 1200 decode speed.

I tested Needle on the align_needle_inference_logic branch with the Cactus old-style INT4 weights format.

My setup:

• Device: NVIDIA Jetson Thor / ARM Linux
• Branch: align_needle_inference_logic
• Commit: 71fc11bc
• Model format: config.txt + *.weights
• Quantization: INT4 weights, FP16 compute

My measured speed is around:

prefill: ~1200 tok/s
decode:  ~100-140 tok/s

Could you share the exact setup used to get the reported numbers?
Specifically:
Which device/hardware was used?
Which Cactus branch/commit was used?
Which model weights were used?
What benchmark command or script was used?
What input length and output length were measured?
Was this CPU-only, GPU, Apple NPU/ANE, Android NPU, or another backend?
Thanks，I look forward to hearing from you!