Skip to content

Latest commit

 

History

History
71 lines (58 loc) · 3.42 KB

File metadata and controls

71 lines (58 loc) · 3.42 KB

Baseline Results

  • Release: v0.3.0
  • Model: alibayram/smollm3
  • Mode: End-to-end AIgentOS
  • Completed: 4/8/2026, 10:05:09 PM
  • Duration: 559.13s

Simple Q/A

Case User Input Est Time to First Token Time to Response Completion Full Prompt Tokens Completion Tokens Total Tokens
Simple Q/A (100 user tokens) 109 6.52s 6.52s 0 9 9
Simple Q/A (250 user tokens) 254 5.77s 5.77s 0 11 11
Simple Q/A (500 user tokens) 508 8.23s 8.23s 0 10 10

Summarization Tasks

Case User Input Est Time to First Token Time to Response Completion Full Prompt Tokens Completion Tokens Total Tokens
Summarization (200 user tokens) 222 7.01s 8.48s 449 86 535
Summarization (500 user tokens) 518 11.13s 12.64s 689 86 775
Summarization (1000 user tokens) 1006 10.54s 14.86s 1106 247 1353
Summarization (2000 user tokens) 2012 15.09s 19.07s 1943 220 2163

20-Turn Conversation

Case User Input Est Time to First Token Time to Response Completion Full Prompt Tokens Completion Tokens Total Tokens
20-turn conversation (50-200 user tokens/turn) 2719 10.46s 28.52s 82437 4563 87000
Turn In Tok Out Tok TTFT Completion Throughput
1 352 232 13648ms 17622ms 13.2 tok/s
2 687 265 10462ms 15067ms 17.6 tok/s
3 1092 258 12603ms 17349ms 14.9 tok/s
4 1490 271 13177ms 18026ms 15.0 tok/s
5 1901 290 13904ms 19226ms 15.1 tok/s
6 2368 304 14525ms 20146ms 15.1 tok/s
7 2849 274 15377ms 20458ms 13.4 tok/s
8 3337 283 16975ms 22261ms 12.7 tok/s
9 3834 236 17775ms 22236ms 10.6 tok/s
10 4173 156 17030ms 20103ms 7.8 tok/s
11 4432 204 16510ms 20440ms 10.0 tok/s
12 4776 182 17198ms 20766ms 8.8 tok/s
13 5098 201 18607ms 22596ms 8.9 tok/s
14 5439 201 17638ms 21608ms 9.3 tok/s
15 5817 201 19206ms 23229ms 8.7 tok/s
16 6195 201 21470ms 25505ms 7.9 tok/s
17 6610 201 22299ms 26415ms 7.6 tok/s
18 7025 201 23148ms 27305ms 7.4 tok/s
19 7329 201 23617ms 27755ms 7.2 tok/s
20 7633 201 24290ms 28522ms 7.0 tok/s

Structured Extraction (Extra)

Case User Input Est Time to First Token Time to Response Completion Full Prompt Tokens Completion Tokens Total Tokens
Structured Extraction (400 user tokens) 429 8.76s 11.36s 605 154 759

System Prompt Pressure

Case User Input Est Time to First Token Time to Response Completion Full Prompt Tokens Completion Tokens Total Tokens
System Prompt Pressure (200 system tokens) 123 3.47s 4.27s 441 52 493
System Prompt Pressure (500 system tokens) 164 881ms 3.10s 658 132 790
System Prompt Pressure (1000 system tokens) 207 1.46s 2.68s 1025 75 1100
System Prompt Pressure (2000 system tokens) 248 2.58s 4.22s 1729 96 1825
System Prompt Pressure (5000 system tokens) 248 5.91s 7.63s 3745 96 3841
System Prompt Pressure (10000 system tokens) 291 12.23s 13.31s 7073 56 7129