ROW warm C4 T=default | e2e=650.36 tok/s | decode(C/meanTPOT)=739.4 | tok/step=3.34 | meanTTFT=144.15 ms
ROW en1 C1 T=0 | e2e=229.95 tok/s | decode(C/meanTPOT)=237.0 | tok/step=3.35 | meanTTFT=81.18 ms
ROW tr1 C1 T=0 | e2e=232.89 tok/s | decode(C/meanTPOT)=237.5 | tok/step=3.49 | meanTTFT=85.65 ms
ROW en1 C4 T=0 | e2e=624.73 tok/s | decode(C/meanTPOT)=764.8 | tok/step=3.34 | meanTTFT=133.85 ms
ROW tr1 C4 T=0 | e2e=680.77 tok/s | decode(C/meanTPOT)=757.6 | tok/step=3.42 | meanTTFT=145.45 ms
ROW en1 C1 T=default | e2e=222.45 tok/s | decode(C/meanTPOT)=226.8 | tok/step=3.29 | meanTTFT=84.59 ms
ROW tr1 C1 T=default | e2e=193.40 tok/s | decode(C/meanTPOT)=196.5 | tok/step=2.86 | meanTTFT=83.89 ms
ROW en1 C4 T=default | e2e=639.73 tok/s | decode(C/meanTPOT)=733.9 | tok/step=3.30 | meanTTFT=137.31 ms
ROW tr1 C4 T=default | e2e=590.59 tok/s | decode(C/meanTPOT)=670.0 | tok/step=3.05 | meanTTFT=144.98 ms
ROW en2 C1 T=0 | e2e=230.75 tok/s | decode(C/meanTPOT)=238.1 | tok/step=3.35 | meanTTFT=82.68 ms
ROW tr2 C1 T=0 | e2e=239.69 tok/s | decode(C/meanTPOT)=244.5 | tok/step=3.49 | meanTTFT=82.91 ms
ROW en2 C4 T=0 | e2e=622.98 tok/s | decode(C/meanTPOT)=763.4 | tok/step=3.34 | meanTTFT=133.43 ms
ROW tr2 C4 T=0 | e2e=658.98 tok/s | decode(C/meanTPOT)=733.9 | tok/step=3.42 | meanTTFT=144.83 ms
ROW en2 C1 T=default | e2e=227.50 tok/s | decode(C/meanTPOT)=234.7 | tok/step=3.14 | meanTTFT=78.53 ms
ROW tr2 C1 T=default | e2e=201.18 tok/s | decode(C/meanTPOT)=204.5 | tok/step=2.84 | meanTTFT=89.30 ms
ROW en2 C4 T=default | e2e=643.86 tok/s | decode(C/meanTPOT)=763.4 | tok/step=3.19 | meanTTFT=127.91 ms
ROW tr2 C4 T=default | e2e=668.39 tok/s | decode(C/meanTPOT)=749.1 | tok/step=3.17 | meanTTFT=135.28 ms
