Kernels: each pipeline step on real data
The C code composes a pipeline at run time (a switch per step per decode block; templates only over the pixel type), so each step is timed on its own: starting from the automatic pipeline, each value of one step with the others held there. Machines: AMD EPYC 9454, Intel Xeon Gold 6248; each has its own section after the description of the step values.
The pipeline steps
What each value timed below does, from the comments in the C sources (linked), and what it needs, from src/_pipeline.py.
decode
| value |
what it does |
needs |
source |
lz4-band |
the zero-aware decoder (lz4zero.h) for blocks compressed more than BSLZ4_LZ4ZERO_RATIO or less than BSLZ4_LZ4ZERO_DENSE, stock lz4 in between. Zero-aware wins on sparse blocks (skipped zero runs) and on dense ones (literal heavy; it never writes the empty high byte-planes), but blocks of ~8-24x are many short sequences (~90-130 per 8 kB) where stock's loop is faster. Measured by tools/bench_kernels.py, see the Every pipeline page of the documentation. |
lz4: zero-aware decoder below 8x and above 24x, stock lz4 in between |
decode.h:12 |
lz4-zero |
the zero-aware decoder for every block: POWER9, where stock lz4 is slower in every band, and Cascade Lake. |
lz4: zero-aware decoder for every block |
decode.h:22 |
lz4-stock |
LZ4_decompress_safe. |
lz4: LZ4_decompress_safe |
decode.h:25 |
zstd |
ZSTD_decompress. |
zstd |
decode.h:26 |
pixels |
no decode: the input is uncompressed pixel frames (from_frames), so the block loop reads each frame's blocks where they lie and runs only the mask, collect and dot steps. |
no decode: uncompressed pixel frames (from_frames); not for chunks |
decode.h:27 |
mask
| value |
what it does |
needs |
source |
pixel |
tested per pixel by the collect (the default: free on data whose masked pixels hold 0, as our Eigers write). |
tested per pixel when collecting |
mask.h:7 |
planes |
packed once per block per batch into the bit-plane layout (bit e % 8 of byte e / 8) and ANDed into every bit-plane straight after decode, so masked pixels are 0 before any untranspose and the block needs no mask tests. Blocks with nothing masked skip it. Wins only where masked pixels hold the dtype maximum: then they no longer stop the u16 byte skip; pure overhead on zero-filled data. |
ANDed into the bit-planes after decode (wins only if masked pixels hold the maximum) |
mask.h:9 |
none |
every pixel valid: no tests at all. |
every pixel valid: no mask tests |
mask.h:17 |
untranspose
| value |
what it does |
needs |
source |
lowplanes-vbmi |
needs AVX-512 F/BW/VL/VBMI/VBMI2, GFNI and popcnt. Bit transpose by gf2p8affineqb, 64 pixels at a time, stored straight into u16 for the dense route; the fused collect transposes only the 64-pixel groups holding data in well-compressed blocks. |
u16 pixels; AVX-512 VBMI, VBMI2 and GFNI |
untranspose.h:27 |
lowplanes-avx2 |
needs AVX2 and popcnt. Fused collect: planes ORed into a bitmap of non-empty 64-pixel groups, only those transposed (kcb's AVX2 kernel) and compacted; ne % 256 == 0, any block size (worked in tiles of BSLZ4_LOWPLANES_TILE pixels). Dense route: kcb's u8 transpose, widened to u16. |
u16 pixels; AVX2 |
untranspose.h:32 |
lowplanes-vsx |
needs POWER8+ (vgbbd). Fused collect: per 64-pixel group two vec_perm rounds and vec_gb give the pixel bytes; all-zero groups and words are skipped. The dense route takes kcb. |
u16 pixels; POWER8 VSX |
untranspose.h:38 |
lowplanes-c |
portable C, always available: kcb's u8 transpose of the low planes, then a SWAR collect of the pixels > cut. The dense route takes kcb. |
u16 pixels; portable C |
untranspose.h:42 |
kcb |
kcb, any dtype and block: dispatches SSE2, AVX2 or AVX-512 itself (no NEON code); the full bit and byte transpose into the pixel block, collect separate. |
kcb's bit transpose (SSE2/AVX2/AVX-512 dispatched inside) |
untranspose.h:45 |
neon |
bitshuffle's NEON kernels (bshuf_trans_byte_bitrow_NEON, then bshuf_shuffle_bit_eightelem_NEON); AArch64 only, any dtype, into the pixel block. |
bitshuffle's NEON kernels; aarch64 |
untranspose.h:48 |
scalar |
bitshuffle's scalar kernels (bshuf_trans_byte_bitrow_scal, then bshuf_shuffle_bit_eightelem_scal); any CPU, any dtype, into the pixel block. |
bitshuffle's scalar kernels |
untranspose.h:51 |
collect
| value |
what it does |
needs |
source |
avx512cs |
AVX-512 F+BW+VL+VBMI2: compress-store of the selected pixels and their addresses; u16/u32. |
u16/u32 pixels; AVX-512 BW/VL and VBMI2 |
collect.hpp:19 |
avx2cs |
AVX2: u16 compacts the selected lanes with a table, u32 walks the set bits; u16/u32. |
u16/u32 pixels; AVX2 |
collect.hpp:21 |
vsx |
POWER8+ (needs -maltivec -mvsx); u16/u32. |
u16/u32 pixels; POWER VSX |
collect.hpp:23 |
neon |
AArch64 ASIMD (ARMv8-A baseline); u16/u32. |
u16/u32 pixels; aarch64 NEON |
collect.hpp:24 |
sse2 |
x86-64 baseline, always present; u16/u32. |
u16/u32 pixels; SSE2 |
collect.hpp:25 |
scalar |
the plain scalar loop, any dtype; every dtype other than u16/u32 takes it whatever the value. |
any pixel type |
collect.hpp:26 |
route
| value |
what it does |
needs |
source |
ratio |
per block: the sparse route when the block is compressed more than BSLZ4_ROUTE_RATIO (8) times (blocksize > 8 * compressed bytes), the dense route otherwise. Well-compressed blocks are mostly zero, so a list of the few non-zero pixels is cheaper than a pass over the block. |
per block: sparse when compressed more than 8x |
driver.c:18 |
dense |
every block untransposed to the pixel block and the dot taken over every pixel of it. |
dot over every pixel of every block |
driver.c:27 |
sparse |
every block reduced to the list of its non-zero pixels (mask applied), the dot taken over the list; for u16 blocks < 256 the untranspose is fused with the list's collect. |
dot over the non-zero pixels of every block |
driver.c:29 |
bpp |
as the ratio rule, in bits per pixel: the sparse route when the block is compressed below BSLZ4_ROUTE_BPP (2) bits per pixel, whatever the pixel width (the ratio rule's 8x is 2 bits per pixel for u16 only). |
per block: sparse when compressed below 2 bits per pixel |
driver.c:24 |
dot
| value |
what it does |
needs |
source |
csc |
general CSC: any matrix, one (bin, weight) entry per pixel/bin pair. |
any matrix |
registry.c:50 |
padded |
every pixel one ascending run of consecutive bins, at most 64 wide: per row a first bin and a fixed width of weights, zero padded; no index read per entry. Rows are per pixel, or only the pixels with entries when under half have any (listed). |
each pixel's bins one consecutive run (at most 64) |
registry.c:52 |
padded-avx2 |
the padded layout with AVX2+FMA row kernels on the dense route, for widths <= 8 (wider: scalar). |
as padded; AVX2 + FMA on the dense route |
registry.c:57 |
csc-run |
CSC where each pixel's bins are one ascending run of consecutive bins: stores only the first bin per pixel beside the weights; any width. |
each pixel's bins one consecutive run |
registry.c:59 |
csc-nosplit |
a histogram: at most one bin per pixel, weight exactly 1; stores that bin per pixel, no weights. |
at most one bin per pixel, weight 1 (a histogram) |
registry.c:62 |
bsb-csr |
any matrix, for ones where most pixels have no entry (rings): per decode block a CSR of the bins it touches, so the dense route walks only those bins with register accumulators; the sparse route uses the nested CSC. |
any matrix; for few occupied pixels (rings) |
registry.c:64 |
bsb-csr-nosplit |
bsb-csr for a histogram (one bin per pixel, weight 1). |
as csc-nosplit |
registry.c:69 |
csc-permute |
a permutation: each pixel its own bin, weight 1; the pixel value is stored (not added) at its bin, so the output is the re-ordered image, in the pixel dtype rather than a float64 powder. |
a permutation: one bin per pixel, weight 1, each bin used once; output in the pixel dtype |
registry.c:71 |
csc-int |
general CSC with u32 fixed-point weights round(w * 2b) (unsigned, so no negative weights), integer pixel dtypes only; exact int64 sums of w * v, rescaled by 2-b afterwards. |
integer pixels; fixed-point u32 weights, exact int64 sums |
registry.c:75 |
csc-run-int |
csc-run's layout (first bin per pixel) with u32 fixed-point weights and an int64 powder, as csc-int. |
integer pixels; as csc-run with fixed-point u32 weights |
registry.c:79 |
csc-run-int16 |
as csc-run-int with u16 weights. |
integer pixels; as csc-run with fixed-point u16 weights |
registry.c:81 |
csc-nosplit-int |
csc-nosplit's layout (one bin per pixel, weight 1) summed in int64: no weights, no multiply, no float conversion; the dense route walks tiles of a block in strided passes. Pixels of at most 32 bits, so the sums cannot overflow. The sums are int64 in the powder buffer; the driver converts each frame in place to float64 once its last block is in (a sum above 2**53 is rounded there), so the result is a float64 powder. |
integer pixels of at most 32 bits; csc-nosplit summed in int64, returned as float64 (one cast at the end) |
registry.c:82 |
AMD EPYC 9454
AMD EPYC 9454: measured on older code
Measured at git v0.0.21a1-83-ga1a55ac (C sources sha256 5a0108f1b0c2, kernel-choice modules ce1cdafcbf56); this site was built from 6929bd75ab7a / 99941e0ecc68. The code changed since, so these numbers may no longer describe it. Re-run tools/bench_docs.py on that machine and commit its JSON.
One core of AMD EPYC 9454, 2026-10-14 17:28 UTC, bslz4_to_sparse 0.0.22a1 (git v0.0.21a1-83-ga1a55ac); the whole run took 247 s. 25 frames spread over each file, re-encoded in memory with each codec and block size listed; one pass over all frames checks the result against the automatic pipeline and warms up, then the best of 2 timed passes (batches of 25, no file I/O). Sparsify is also timed no mask: the same frames with the masked pixels 0 and an all-valid mask. Bold marks the automatic choice; vs auto: the sparse output must be the same, the powder is compared as max |difference| / max |powder| (the fixed-point dots round). Values the library refuses for the data, block size or matrix are listed with the reason.
The matrices (the approximate calib.poni, 2162 x 2068 pixels, and every object folds the mask of the example data):
| matrix |
what |
bins |
entries |
pixels with an entry |
bins per pixel (mean, max) |
| 1D |
chunk2sparseCSC, pyFAI 1D bbox matrix (r_mm, one bin per pixel width) |
1542 |
9655790 |
100 % |
2.30, 3 |
| 1D fine |
chunk2sparseCSC, pyFAI 1D bbox matrix with 6000 bins (r_mm): a fine split, several bins per pixel |
6000 |
25427825 |
100 % |
6.05, 7 |
| 1D no-split |
chunk2sparseCSC, pyFAI 1D no-split matrix: a histogram, every pixel in one bin with weight 1 (r_mm, one bin per pixel width) |
1542 |
4202182 |
100 % |
1.00, 1 |
| 2D |
chunk2sparseCSC, pyFAI 2D bbox matrix (q_nm^-1, one radial bin per pixel width x 72 azimuthal bins) |
1542 x 72 |
9835464 |
100 % |
2.34, 88 |
| 2D+rings |
chunk2sparseCSC, 8 LaB6 ring windows x 72 azimuthal bins summed from the 2D matrix, the windows sized to hold ~8 % of the unmasked pixels |
8 x 72 |
375415 |
8.8 % |
1.02, 2 |
silicon: sparse photon counting, uint16
silicon_lz4mask.h5 (/entry_0000/ESRF-ID11/eiger/data, 2162 x 2068 uint16, 268834 masked pixels, cut 0). Encodings:
| encoding |
codec |
level |
block |
compression |
tables |
| lz4 8k |
lz4 |
- |
8 kB |
39.2 |
sparsify, matrices |
| lz4 256k |
lz4 |
- |
256 kB |
57.6 |
sparsify |
| zstd-2 8k |
zstd |
2 |
8 kB |
43.4 |
sparsify |
| zstd-2 256k |
zstd |
2 |
256 kB |
131.0 |
sparsify, matrices |
silicon, lz4 8k, sparsify
Automatic pipeline: decode lz4-band, mask planes, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-zero |
961.6 |
0.219 |
8.60 |
same |
lz4-band |
960.5 |
0.219 |
8.59 |
auto; same |
lz4-stock |
951.4 |
0.217 |
8.51 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
planes |
958.3 |
0.218 |
8.57 |
auto; same |
pixel |
747.6 |
0.170 |
6.69 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-vbmi |
958.6 |
0.218 |
8.57 |
auto; same |
lowplanes-avx2 |
867.8 |
0.198 |
7.76 |
same |
kcb |
723.1 |
0.165 |
6.47 |
same |
lowplanes-c |
662.2 |
0.151 |
5.92 |
same |
scalar |
128.9 |
0.029 |
1.15 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
722.6 |
0.165 |
6.46 |
same |
avx2cs |
639.5 |
0.146 |
5.72 |
same |
sse2 |
553.4 |
0.126 |
4.95 |
same |
scalar |
282.1 |
0.064 |
2.52 |
same |
silicon, lz4 8k, sparsify, no mask
Automatic pipeline: decode lz4-band, mask none, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-zero |
1823.7 |
0.272 |
16.31 |
same |
lz4-band |
1815.6 |
0.271 |
16.24 |
auto; same |
lz4-stock |
1203.5 |
0.180 |
10.76 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
1820.4 |
0.272 |
16.28 |
auto; same |
pixel |
1795.9 |
0.268 |
16.06 |
same |
planes |
1790.8 |
0.268 |
16.01 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-vbmi |
1818.3 |
0.272 |
16.26 |
auto; same |
lowplanes-avx2 |
1620.9 |
0.242 |
14.49 |
same |
lowplanes-c |
994.7 |
0.149 |
8.89 |
same |
kcb |
973.0 |
0.145 |
8.70 |
same |
scalar |
136.0 |
0.020 |
1.22 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
971.0 |
0.145 |
8.68 |
same |
avx2cs |
827.9 |
0.124 |
7.40 |
same |
sse2 |
688.6 |
0.103 |
6.16 |
same |
scalar |
312.9 |
0.047 |
2.80 |
same |
silicon, lz4 8k, 1D matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
703.5 |
0.160 |
6.29 |
same; powder exact |
sparse |
699.7 |
0.159 |
6.26 |
same; powder exact |
bpp |
698.0 |
0.159 |
6.24 |
auto; same; powder exact |
dense |
113.1 |
0.026 |
1.01 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded-avx2 |
706.7 |
0.161 |
6.32 |
auto; same; powder exact |
padded |
700.9 |
0.160 |
6.27 |
same; powder exact |
csc-run |
673.4 |
0.153 |
6.02 |
same; powder exact |
bsb-csr |
644.9 |
0.147 |
5.77 |
same; powder exact |
csc |
629.3 |
0.143 |
5.63 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, lz4 8k, 1D no-split matrix
Automatic pipeline: route bpp, dot csc-nosplit-int. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
867.2 |
0.198 |
7.75 |
auto; same; powder exact |
sparse |
864.4 |
0.197 |
7.73 |
same; powder exact |
ratio |
861.9 |
0.196 |
7.71 |
same; powder exact |
dense |
291.1 |
0.066 |
2.60 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bsb-csr-nosplit |
871.3 |
0.199 |
7.79 |
same; powder exact |
csc-nosplit |
844.9 |
0.192 |
7.55 |
same; powder exact |
csc-nosplit-int |
844.7 |
0.192 |
7.55 |
auto; same; powder exact |
padded |
780.3 |
0.178 |
6.98 |
same; powder exact |
padded-avx2 |
778.7 |
0.177 |
6.96 |
same; powder exact |
bsb-csr |
739.5 |
0.168 |
6.61 |
same; powder exact |
csc |
734.6 |
0.167 |
6.57 |
same; powder exact |
csc-run |
729.4 |
0.166 |
6.52 |
same; powder exact |
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4200640 bins are reached by more than one pixel (auto picks 'csc-nosplit-int') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
silicon, lz4 8k, 1D fine matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
590.0 |
0.134 |
5.28 |
same; powder exact |
bpp |
587.3 |
0.134 |
5.25 |
auto; same; powder exact |
sparse |
577.5 |
0.132 |
5.16 |
same; powder exact |
dense |
86.4 |
0.020 |
0.77 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded |
592.9 |
0.135 |
5.30 |
same; powder exact |
padded-avx2 |
591.2 |
0.135 |
5.29 |
auto; same; powder exact |
csc-run |
555.1 |
0.126 |
4.96 |
same; powder exact |
bsb-csr |
542.7 |
0.124 |
4.85 |
same; powder exact |
csc |
533.6 |
0.122 |
4.77 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, lz4 8k, 2D matrix
Automatic pipeline: route bpp, dot csc. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
585.2 |
0.133 |
5.23 |
same; powder exact |
bpp |
583.7 |
0.133 |
5.22 |
auto; same; powder exact |
sparse |
583.2 |
0.133 |
5.21 |
same; powder exact |
dense |
85.7 |
0.020 |
0.77 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bsb-csr |
596.3 |
0.136 |
5.33 |
same; powder exact |
csc |
583.9 |
0.133 |
5.22 |
auto; same; powder exact |
padded |
refused: pipeline['dot']='padded' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
padded-avx2 |
refused: pipeline['dot']='padded-avx2' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
csc-run |
refused: pipeline['dot']='csc-run' cannot represent this matrix: 4197949 of 4198000 pixels with several bins reach bins that are not one ascending run of consecutive bins (auto picks 'csc') |
|
|
|
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
silicon, lz4 8k, 2D+rings matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
822.9 |
0.187 |
7.36 |
same; powder exact |
bpp |
821.8 |
0.187 |
7.35 |
auto; same; powder exact |
sparse |
819.0 |
0.187 |
7.32 |
same; powder exact |
dense |
518.9 |
0.118 |
4.64 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bsb-csr |
842.7 |
0.192 |
7.54 |
same; powder exact |
csc |
842.0 |
0.192 |
7.53 |
same; powder exact |
padded-avx2 |
819.7 |
0.187 |
7.33 |
auto; same; powder exact |
padded |
817.5 |
0.186 |
7.31 |
same; powder exact |
csc-run |
808.8 |
0.184 |
7.23 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, lz4 256k, sparsify
Automatic pipeline: decode lz4-band, mask pixel, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-band |
736.7 |
0.114 |
6.59 |
auto; same |
lz4-zero |
736.6 |
0.114 |
6.59 |
same |
lz4-stock |
735.4 |
0.114 |
6.58 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
planes |
736.9 |
0.114 |
6.59 |
same |
pixel |
736.6 |
0.114 |
6.59 |
auto; same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-c |
737.3 |
0.114 |
6.59 |
same |
lowplanes-avx2 |
736.8 |
0.114 |
6.59 |
same |
kcb |
736.6 |
0.114 |
6.59 |
same |
lowplanes-vbmi |
736.6 |
0.114 |
6.59 |
auto; same |
scalar |
132.2 |
0.021 |
1.18 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
737.7 |
0.115 |
6.60 |
same |
avx2cs |
654.4 |
0.102 |
5.85 |
same |
sse2 |
569.7 |
0.088 |
5.09 |
same |
scalar |
282.0 |
0.044 |
2.52 |
same |
silicon, lz4 256k, sparsify, no mask
Automatic pipeline: decode lz4-band, mask none, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-zero |
2007.6 |
0.244 |
17.95 |
same |
lz4-band |
2001.1 |
0.244 |
17.89 |
auto; same |
lz4-stock |
1216.0 |
0.148 |
10.87 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
2009.5 |
0.245 |
17.97 |
auto; same |
planes |
1959.9 |
0.239 |
17.53 |
same |
pixel |
1942.9 |
0.237 |
17.37 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-vbmi |
2005.8 |
0.244 |
17.94 |
auto; same |
lowplanes-avx2 |
1753.2 |
0.213 |
15.68 |
same |
lowplanes-c |
1037.9 |
0.126 |
9.28 |
same |
kcb |
891.9 |
0.109 |
7.98 |
same |
scalar |
136.1 |
0.017 |
1.22 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
889.9 |
0.108 |
7.96 |
same |
avx2cs |
790.1 |
0.096 |
7.06 |
same |
sse2 |
660.0 |
0.080 |
5.90 |
same |
scalar |
307.0 |
0.037 |
2.75 |
same |
silicon, zstd-2 8k, sparsify
Automatic pipeline: decode zstd, mask planes, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
326.1 |
0.067 |
2.92 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
planes |
326.3 |
0.067 |
2.92 |
auto; same |
pixel |
295.0 |
0.061 |
2.64 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-vbmi |
326.3 |
0.067 |
2.92 |
auto; same |
lowplanes-avx2 |
311.7 |
0.064 |
2.79 |
same |
kcb |
290.7 |
0.060 |
2.60 |
same |
lowplanes-c |
281.9 |
0.058 |
2.52 |
same |
scalar |
101.6 |
0.021 |
0.91 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
290.3 |
0.060 |
2.60 |
same |
avx2cs |
275.8 |
0.057 |
2.47 |
same |
sse2 |
258.7 |
0.053 |
2.31 |
same |
scalar |
178.5 |
0.037 |
1.60 |
same |
silicon, zstd-2 8k, sparsify, no mask
Automatic pipeline: decode zstd, mask none, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
472.0 |
0.047 |
4.22 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
472.4 |
0.047 |
4.22 |
auto; same |
planes |
469.5 |
0.046 |
4.20 |
same |
pixel |
466.0 |
0.046 |
4.17 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-vbmi |
472.1 |
0.047 |
4.22 |
auto; same |
lowplanes-avx2 |
444.9 |
0.044 |
3.98 |
same |
kcb |
392.3 |
0.039 |
3.51 |
same |
lowplanes-c |
385.7 |
0.038 |
3.45 |
same |
scalar |
112.5 |
0.011 |
1.01 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
392.3 |
0.039 |
3.51 |
same |
avx2cs |
368.0 |
0.036 |
3.29 |
same |
sse2 |
338.7 |
0.033 |
3.03 |
same |
scalar |
213.3 |
0.021 |
1.91 |
same |
silicon, zstd-2 256k, sparsify
Automatic pipeline: decode zstd, mask pixel, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
650.5 |
0.044 |
5.82 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
pixel |
650.5 |
0.044 |
5.82 |
auto; same |
planes |
649.2 |
0.044 |
5.80 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-vbmi |
651.7 |
0.044 |
5.83 |
auto; same |
lowplanes-c |
651.4 |
0.044 |
5.82 |
same |
kcb |
651.1 |
0.044 |
5.82 |
same |
lowplanes-avx2 |
650.2 |
0.044 |
5.81 |
same |
scalar |
128.4 |
0.009 |
1.15 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
651.3 |
0.044 |
5.82 |
same |
avx2cs |
579.6 |
0.040 |
5.18 |
same |
sse2 |
509.6 |
0.035 |
4.56 |
same |
scalar |
266.1 |
0.018 |
2.38 |
same |
silicon, zstd-2 256k, sparsify, no mask
Automatic pipeline: decode zstd, mask none, untranspose lowplanes-vbmi, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
833.5 |
0.039 |
7.45 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
833.9 |
0.039 |
7.46 |
auto; same |
planes |
817.2 |
0.039 |
7.31 |
same |
pixel |
815.1 |
0.038 |
7.29 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-vbmi |
832.7 |
0.039 |
7.45 |
auto; same |
lowplanes-avx2 |
744.7 |
0.035 |
6.66 |
same |
kcb |
600.6 |
0.028 |
5.37 |
same |
lowplanes-c |
595.3 |
0.028 |
5.32 |
same |
scalar |
125.4 |
0.006 |
1.12 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
600.9 |
0.028 |
5.37 |
same |
avx2cs |
543.4 |
0.026 |
4.86 |
same |
sse2 |
479.4 |
0.023 |
4.29 |
same |
scalar |
260.8 |
0.012 |
2.33 |
same |
silicon, zstd-2 256k, 1D matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
530.2 |
0.036 |
4.74 |
auto; same; powder exact |
sparse |
530.2 |
0.036 |
4.74 |
same; powder exact |
ratio |
529.4 |
0.036 |
4.73 |
same; powder exact |
dense |
110.6 |
0.008 |
0.99 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded-avx2 |
531.0 |
0.036 |
4.75 |
auto; same; powder exact |
padded |
530.0 |
0.036 |
4.74 |
same; powder exact |
csc-run |
500.3 |
0.034 |
4.47 |
same; powder exact |
csc |
486.5 |
0.033 |
4.35 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, zstd-2 256k, 1D no-split matrix
Automatic pipeline: route bpp, dot csc-nosplit-int. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
597.0 |
0.041 |
5.34 |
same; powder exact |
sparse |
596.9 |
0.041 |
5.34 |
same; powder exact |
bpp |
594.4 |
0.041 |
5.32 |
auto; same; powder exact |
dense |
273.3 |
0.019 |
2.44 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc-nosplit |
594.9 |
0.041 |
5.32 |
same; powder exact |
csc-nosplit-int |
581.2 |
0.040 |
5.20 |
auto; same; powder exact |
padded |
571.0 |
0.039 |
5.11 |
same; powder exact |
padded-avx2 |
570.4 |
0.039 |
5.10 |
same; powder exact |
csc |
546.7 |
0.037 |
4.89 |
same; powder exact |
csc-run |
532.2 |
0.036 |
4.76 |
same; powder exact |
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'csc-nosplit-int') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'csc-nosplit-int') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4200640 bins are reached by more than one pixel (auto picks 'csc-nosplit-int') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
silicon, zstd-2 256k, 1D fine matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
452.5 |
0.031 |
4.05 |
auto; same; powder exact |
sparse |
452.2 |
0.031 |
4.04 |
same; powder exact |
ratio |
451.0 |
0.031 |
4.03 |
same; powder exact |
dense |
80.5 |
0.005 |
0.72 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
458.2 |
0.031 |
4.10 |
same; powder exact |
padded |
452.3 |
0.031 |
4.04 |
same; powder exact |
padded-avx2 |
451.7 |
0.031 |
4.04 |
auto; same; powder exact |
csc-run |
422.5 |
0.029 |
3.78 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, zstd-2 256k, 2D matrix
Automatic pipeline: route bpp, dot csc. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
sparse |
456.9 |
0.031 |
4.09 |
same; powder exact |
bpp |
456.5 |
0.031 |
4.08 |
auto; same; powder exact |
ratio |
455.6 |
0.031 |
4.07 |
same; powder exact |
dense |
75.2 |
0.005 |
0.67 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
453.2 |
0.031 |
4.05 |
auto; same; powder exact |
padded |
refused: pipeline['dot']='padded' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
padded-avx2 |
refused: pipeline['dot']='padded-avx2' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
csc-run |
refused: pipeline['dot']='csc-run' cannot represent this matrix: 4197949 of 4198000 pixels with several bins reach bins that are not one ascending run of consecutive bins (auto picks 'csc') |
|
|
|
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'csc') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
silicon, zstd-2 256k, 2D+rings matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
sparse |
593.2 |
0.040 |
5.30 |
same; powder exact |
ratio |
591.3 |
0.040 |
5.29 |
same; powder exact |
bpp |
589.4 |
0.040 |
5.27 |
auto; same; powder exact |
dense |
459.7 |
0.031 |
4.11 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded |
573.7 |
0.039 |
5.13 |
same; powder exact |
padded-avx2 |
572.5 |
0.039 |
5.12 |
auto; same; powder exact |
csc-run |
563.8 |
0.038 |
5.04 |
same; powder exact |
csc |
563.1 |
0.038 |
5.04 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2: dense powder rings, uint32
ceo2_lz4mask.h5 (/entry_0000/ESRF-ID11/eiger/data, 2162 x 2068 uint32, 268834 masked pixels, cut 150). Encodings:
| encoding |
codec |
level |
block |
compression |
tables |
| lz4 8k |
lz4 |
- |
8 kB |
4.5 |
sparsify, matrices |
| zstd-2 256k |
zstd |
2 |
256 kB |
4.9 |
sparsify |
ceo2, lz4 8k, sparsify
Automatic pipeline: decode lz4-band, mask pixel, untranspose kcb, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-stock |
444.4 |
1.785 |
7.95 |
same |
lz4-zero |
419.5 |
1.685 |
7.50 |
same |
lz4-band |
418.5 |
1.681 |
7.49 |
auto; same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
pixel |
416.9 |
1.675 |
7.46 |
auto; same |
planes |
369.9 |
1.486 |
6.62 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
419.6 |
1.685 |
7.50 |
auto; same |
scalar |
65.8 |
0.264 |
1.18 |
same |
lowplanes-vbmi |
refused: pipeline['untranspose']='lowplanes-vbmi' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
417.5 |
1.677 |
7.47 |
auto; same |
avx2cs |
348.9 |
1.402 |
6.24 |
same |
sse2 |
312.3 |
1.254 |
5.58 |
same |
scalar |
216.2 |
0.868 |
3.87 |
same |
ceo2, lz4 8k, sparsify, no mask
Automatic pipeline: decode lz4-band, mask none, untranspose kcb, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-zero |
463.8 |
1.823 |
8.30 |
same |
lz4-band |
463.5 |
1.822 |
8.29 |
auto; same |
lz4-stock |
439.2 |
1.726 |
7.86 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
pixel |
462.9 |
1.819 |
8.28 |
same |
none |
452.1 |
1.777 |
8.09 |
auto; same |
planes |
447.1 |
1.757 |
8.00 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
466.6 |
1.834 |
8.34 |
auto; same |
scalar |
67.0 |
0.263 |
1.20 |
same |
lowplanes-vbmi |
refused: pipeline['untranspose']='lowplanes-vbmi' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
464.1 |
1.824 |
8.30 |
auto; same |
avx2cs |
409.1 |
1.608 |
7.32 |
same |
sse2 |
313.2 |
1.231 |
5.60 |
same |
scalar |
229.4 |
0.901 |
4.10 |
same |
ceo2, lz4 8k, 1D matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
106.2 |
0.427 |
1.90 |
same; powder exact |
bpp |
106.1 |
0.426 |
1.90 |
auto; same; powder exact |
dense |
97.8 |
0.393 |
1.75 |
same; powder exact |
sparse |
69.4 |
0.279 |
1.24 |
same; powder 1.2e-16 |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded-avx2 |
105.8 |
0.425 |
1.89 |
auto; same; powder exact |
csc-run |
103.5 |
0.416 |
1.85 |
same; powder 1.2e-16 |
bsb-csr |
98.1 |
0.394 |
1.75 |
same; powder 1.2e-16 |
csc |
92.1 |
0.370 |
1.65 |
same; powder 1.2e-16 |
padded |
88.7 |
0.356 |
1.59 |
same; powder 1.2e-16 |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2, lz4 8k, 1D no-split matrix
Automatic pipeline: route bpp, dot csc-nosplit-int. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
219.7 |
0.883 |
3.93 |
same; powder exact |
bpp |
219.1 |
0.880 |
3.92 |
auto; same; powder exact |
dense |
213.5 |
0.858 |
3.82 |
same; powder exact |
sparse |
132.2 |
0.531 |
2.36 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc-nosplit-int |
219.9 |
0.883 |
3.93 |
auto; same; powder exact |
csc-nosplit |
177.4 |
0.713 |
3.17 |
same; powder exact |
bsb-csr-nosplit |
170.3 |
0.684 |
3.05 |
same; powder exact |
padded-avx2 |
158.9 |
0.638 |
2.84 |
same; powder exact |
bsb-csr |
152.5 |
0.613 |
2.73 |
same; powder exact |
csc-run |
134.3 |
0.539 |
2.40 |
same; powder exact |
padded |
128.2 |
0.515 |
2.29 |
same; powder exact |
csc |
126.5 |
0.508 |
2.26 |
same; powder exact |
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4200640 bins are reached by more than one pixel (auto picks 'csc-nosplit-int') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
ceo2, lz4 8k, 1D fine matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
82.5 |
0.331 |
1.48 |
same; powder exact |
bpp |
82.4 |
0.331 |
1.47 |
auto; same; powder exact |
dense |
76.9 |
0.309 |
1.37 |
same; powder exact |
sparse |
48.4 |
0.194 |
0.87 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded-avx2 |
82.6 |
0.332 |
1.48 |
auto; same; powder exact |
csc-run |
60.3 |
0.242 |
1.08 |
same; powder exact |
csc |
54.3 |
0.218 |
0.97 |
same; powder exact |
padded |
51.2 |
0.206 |
0.92 |
same; powder exact |
bsb-csr |
41.7 |
0.168 |
0.75 |
same; powder 5.3e-17 |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2, lz4 8k, 2D matrix
Automatic pipeline: route bpp, dot csc. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
74.2 |
0.298 |
1.33 |
auto; same; powder exact |
ratio |
74.0 |
0.297 |
1.32 |
same; powder exact |
dense |
73.4 |
0.295 |
1.31 |
same; powder exact |
sparse |
38.6 |
0.155 |
0.69 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
74.0 |
0.297 |
1.32 |
auto; same; powder exact |
bsb-csr |
71.4 |
0.287 |
1.28 |
same; powder 3.4e-17 |
padded |
refused: pipeline['dot']='padded' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
padded-avx2 |
refused: pipeline['dot']='padded-avx2' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
csc-run |
refused: pipeline['dot']='csc-run' cannot represent this matrix: 4197949 of 4198000 pixels with several bins reach bins that are not one ascending run of consecutive bins (auto picks 'csc') |
|
|
|
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
ceo2, lz4 8k, 2D+rings matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
306.9 |
1.233 |
5.49 |
same; powder exact |
bpp |
306.8 |
1.232 |
5.49 |
auto; same; powder exact |
dense |
306.5 |
1.231 |
5.48 |
same; powder exact |
sparse |
117.8 |
0.473 |
2.11 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bsb-csr |
348.9 |
1.402 |
6.24 |
same; powder exact |
padded-avx2 |
307.6 |
1.236 |
5.50 |
auto; same; powder exact |
padded |
287.3 |
1.154 |
5.14 |
same; powder exact |
csc |
179.4 |
0.720 |
3.21 |
same; powder exact |
csc-run |
156.1 |
0.627 |
2.79 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2, zstd-2 256k, sparsify
Automatic pipeline: decode zstd, mask pixel, untranspose kcb, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
235.5 |
0.868 |
4.21 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
pixel |
235.5 |
0.868 |
4.21 |
auto; same |
planes |
235.4 |
0.868 |
4.21 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
235.3 |
0.868 |
4.21 |
auto; same |
scalar |
59.5 |
0.219 |
1.06 |
same |
lowplanes-vbmi |
refused: pipeline['untranspose']='lowplanes-vbmi' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
235.4 |
0.868 |
4.21 |
auto; same |
avx2cs |
212.5 |
0.783 |
3.80 |
same |
sse2 |
198.2 |
0.731 |
3.55 |
same |
scalar |
154.8 |
0.571 |
2.77 |
same |
ceo2, zstd-2 256k, sparsify, no mask
Automatic pipeline: decode zstd, mask none, untranspose kcb, collect avx512cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
229.5 |
0.839 |
4.10 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
229.3 |
0.839 |
4.10 |
auto; same |
planes |
228.1 |
0.834 |
4.08 |
same |
pixel |
228.0 |
0.834 |
4.08 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
229.5 |
0.839 |
4.11 |
auto; same |
scalar |
59.0 |
0.216 |
1.05 |
same |
lowplanes-vbmi |
refused: pipeline['untranspose']='lowplanes-vbmi' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx512cs |
229.5 |
0.839 |
4.10 |
auto; same |
avx2cs |
215.2 |
0.787 |
3.85 |
same |
sse2 |
185.3 |
0.677 |
3.31 |
same |
scalar |
153.5 |
0.561 |
2.75 |
same |
Source: docs/bench/kernels_epyc-9454.json, written by tools/bench_kernels.py.
Intel Xeon Gold 6248
Intel Xeon Gold 6248: measured on older code
Measured at git v0.0.21a1-83-ga1a55ac (C sources sha256 5a0108f1b0c2, kernel-choice modules ce1cdafcbf56); this site was built from 6929bd75ab7a / 99941e0ecc68. The code changed since, so these numbers may no longer describe it. Re-run tools/bench_docs.py on that machine and commit its JSON.
One core of Intel Xeon Gold 6248, 2026-10-14 17:08 UTC, bslz4_to_sparse 0.0.22a1 (git v0.0.21a1-83-ga1a55ac); the whole run took 424 s. 25 frames spread over each file, re-encoded in memory with each codec and block size listed; one pass over all frames checks the result against the automatic pipeline and warms up, then the best of 2 timed passes (batches of 25, no file I/O). Sparsify is also timed no mask: the same frames with the masked pixels 0 and an all-valid mask. Bold marks the automatic choice; vs auto: the sparse output must be the same, the powder is compared as max |difference| / max |powder| (the fixed-point dots round). Values the library refuses for the data, block size or matrix are listed with the reason.
The matrices (the approximate calib.poni, 2162 x 2068 pixels, and every object folds the mask of the example data):
| matrix |
what |
bins |
entries |
pixels with an entry |
bins per pixel (mean, max) |
| 1D |
chunk2sparseCSC, pyFAI 1D bbox matrix (r_mm, one bin per pixel width) |
1542 |
9655790 |
100 % |
2.30, 3 |
| 1D fine |
chunk2sparseCSC, pyFAI 1D bbox matrix with 6000 bins (r_mm): a fine split, several bins per pixel |
6000 |
25427825 |
100 % |
6.05, 7 |
| 1D no-split |
chunk2sparseCSC, pyFAI 1D no-split matrix: a histogram, every pixel in one bin with weight 1 (r_mm, one bin per pixel width) |
1542 |
4202182 |
100 % |
1.00, 1 |
| 2D |
chunk2sparseCSC, pyFAI 2D bbox matrix (q_nm^-1, one radial bin per pixel width x 72 azimuthal bins) |
1542 x 72 |
9835464 |
100 % |
2.34, 88 |
| 2D+rings |
chunk2sparseCSC, 8 LaB6 ring windows x 72 azimuthal bins summed from the 2D matrix, the windows sized to hold ~8 % of the unmasked pixels |
8 x 72 |
375415 |
8.8 % |
1.02, 2 |
silicon: sparse photon counting, uint16
silicon_lz4mask.h5 (/entry_0000/ESRF-ID11/eiger/data, 2162 x 2068 uint16, 268834 masked pixels, cut 0). Encodings:
| encoding |
codec |
level |
block |
compression |
tables |
| lz4 8k |
lz4 |
- |
8 kB |
39.2 |
sparsify, matrices |
| lz4 256k |
lz4 |
- |
256 kB |
57.6 |
sparsify |
| zstd-2 8k |
zstd |
2 |
8 kB |
43.4 |
sparsify |
| zstd-2 256k |
zstd |
2 |
256 kB |
131.0 |
sparsify, matrices |
silicon, lz4 8k, sparsify
Automatic pipeline: decode lz4-band, mask planes, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-stock |
467.7 |
0.107 |
4.18 |
same |
lz4-band |
459.0 |
0.105 |
4.10 |
auto; same |
lz4-zero |
442.0 |
0.101 |
3.95 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
planes |
458.9 |
0.105 |
4.10 |
auto; same |
pixel |
329.8 |
0.075 |
2.95 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-avx2 |
457.1 |
0.104 |
4.09 |
auto; same |
lowplanes-c |
333.9 |
0.076 |
2.99 |
same |
kcb |
324.8 |
0.074 |
2.90 |
same |
scalar |
62.1 |
0.014 |
0.55 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
325.4 |
0.074 |
2.91 |
same |
sse2 |
298.5 |
0.068 |
2.67 |
same |
scalar |
136.6 |
0.031 |
1.22 |
same |
silicon, lz4 8k, sparsify, no mask
Automatic pipeline: decode lz4-band, mask none, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-zero |
843.3 |
0.126 |
7.54 |
same |
lz4-band |
840.0 |
0.126 |
7.51 |
auto; same |
lz4-stock |
377.2 |
0.056 |
3.37 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
833.3 |
0.125 |
7.45 |
auto; same |
pixel |
812.7 |
0.121 |
7.27 |
same |
planes |
773.4 |
0.116 |
6.92 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-avx2 |
843.9 |
0.126 |
7.55 |
auto; same |
lowplanes-c |
494.6 |
0.074 |
4.42 |
same |
kcb |
417.0 |
0.062 |
3.73 |
same |
scalar |
74.2 |
0.011 |
0.66 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
417.3 |
0.062 |
3.73 |
same |
sse2 |
373.1 |
0.056 |
3.34 |
same |
scalar |
150.6 |
0.022 |
1.35 |
same |
silicon, lz4 8k, 1D matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
377.1 |
0.086 |
3.37 |
auto; same; powder exact |
sparse |
377.0 |
0.086 |
3.37 |
same; powder exact |
ratio |
374.5 |
0.085 |
3.35 |
same; powder exact |
dense |
63.1 |
0.014 |
0.56 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded |
372.5 |
0.085 |
3.33 |
same; powder exact |
padded-avx2 |
370.8 |
0.084 |
3.32 |
auto; same; powder exact |
csc-run |
361.8 |
0.082 |
3.24 |
same; powder exact |
bsb-csr |
351.6 |
0.080 |
3.14 |
same; powder exact |
csc |
350.9 |
0.080 |
3.14 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, lz4 8k, 1D no-split matrix
Automatic pipeline: route bpp, dot csc-nosplit-int. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
426.2 |
0.097 |
3.81 |
auto; same; powder exact |
sparse |
425.1 |
0.097 |
3.80 |
same; powder exact |
ratio |
422.6 |
0.096 |
3.78 |
same; powder exact |
dense |
137.0 |
0.031 |
1.23 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc-nosplit-int |
417.7 |
0.095 |
3.74 |
auto; same; powder exact |
bsb-csr-nosplit |
415.5 |
0.095 |
3.71 |
same; powder exact |
csc-nosplit |
415.2 |
0.095 |
3.71 |
same; powder exact |
padded |
391.8 |
0.089 |
3.50 |
same; powder exact |
padded-avx2 |
391.3 |
0.089 |
3.50 |
same; powder exact |
csc |
379.1 |
0.086 |
3.39 |
same; powder exact |
bsb-csr |
379.0 |
0.086 |
3.39 |
same; powder exact |
csc-run |
378.6 |
0.086 |
3.39 |
same; powder exact |
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4200640 bins are reached by more than one pixel (auto picks 'csc-nosplit-int') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
silicon, lz4 8k, 1D fine matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
sparse |
331.1 |
0.075 |
2.96 |
same; powder exact |
bpp |
329.8 |
0.075 |
2.95 |
auto; same; powder exact |
ratio |
328.0 |
0.075 |
2.93 |
same; powder exact |
dense |
50.7 |
0.012 |
0.45 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded |
326.4 |
0.074 |
2.92 |
same; powder exact |
padded-avx2 |
326.3 |
0.074 |
2.92 |
auto; same; powder exact |
csc-run |
306.5 |
0.070 |
2.74 |
same; powder exact |
csc |
303.4 |
0.069 |
2.71 |
same; powder exact |
bsb-csr |
303.3 |
0.069 |
2.71 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, lz4 8k, 2D matrix
Automatic pipeline: route bpp, dot csc. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
sparse |
309.6 |
0.071 |
2.77 |
same; powder exact |
ratio |
309.5 |
0.071 |
2.77 |
same; powder exact |
bpp |
309.4 |
0.070 |
2.77 |
auto; same; powder exact |
dense |
34.5 |
0.008 |
0.31 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
316.4 |
0.072 |
2.83 |
auto; same; powder exact |
bsb-csr |
302.8 |
0.069 |
2.71 |
same; powder exact |
padded |
refused: pipeline['dot']='padded' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
padded-avx2 |
refused: pipeline['dot']='padded-avx2' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
csc-run |
refused: pipeline['dot']='csc-run' cannot represent this matrix: 4197949 of 4198000 pixels with several bins reach bins that are not one ascending run of consecutive bins (auto picks 'csc') |
|
|
|
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
silicon, lz4 8k, 2D+rings matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
sparse |
416.3 |
0.095 |
3.72 |
same; powder exact |
bpp |
415.7 |
0.095 |
3.72 |
auto; same; powder exact |
ratio |
410.5 |
0.094 |
3.67 |
same; powder exact |
dense |
247.4 |
0.056 |
2.21 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded |
406.2 |
0.093 |
3.63 |
same; powder exact |
csc |
405.3 |
0.092 |
3.62 |
same; powder exact |
bsb-csr |
404.9 |
0.092 |
3.62 |
same; powder exact |
padded-avx2 |
404.3 |
0.092 |
3.62 |
auto; same; powder exact |
csc-run |
401.1 |
0.091 |
3.59 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, lz4 256k, sparsify
Automatic pipeline: decode lz4-band, mask pixel, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-stock |
380.4 |
0.059 |
3.40 |
same |
lz4-zero |
360.6 |
0.056 |
3.22 |
same |
lz4-band |
360.1 |
0.056 |
3.22 |
auto; same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
pixel |
360.3 |
0.056 |
3.22 |
auto; same |
planes |
360.0 |
0.056 |
3.22 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
360.7 |
0.056 |
3.23 |
same |
lowplanes-c |
360.1 |
0.056 |
3.22 |
same |
lowplanes-avx2 |
359.9 |
0.056 |
3.22 |
auto; same |
scalar |
73.2 |
0.011 |
0.65 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
360.4 |
0.056 |
3.22 |
same |
sse2 |
345.5 |
0.054 |
3.09 |
same |
scalar |
118.8 |
0.018 |
1.06 |
same |
silicon, lz4 256k, sparsify, no mask
Automatic pipeline: decode lz4-band, mask none, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-band |
918.7 |
0.112 |
8.21 |
auto; same |
lz4-zero |
917.6 |
0.112 |
8.20 |
same |
lz4-stock |
395.9 |
0.048 |
3.54 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
918.1 |
0.112 |
8.21 |
auto; same |
planes |
874.3 |
0.106 |
7.82 |
same |
pixel |
874.2 |
0.106 |
7.82 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-avx2 |
918.6 |
0.112 |
8.21 |
auto; same |
lowplanes-c |
520.0 |
0.063 |
4.65 |
same |
kcb |
426.0 |
0.052 |
3.81 |
same |
scalar |
75.9 |
0.009 |
0.68 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
425.7 |
0.052 |
3.81 |
same |
sse2 |
376.1 |
0.046 |
3.36 |
same |
scalar |
151.8 |
0.018 |
1.36 |
same |
silicon, zstd-2 8k, sparsify
Automatic pipeline: decode zstd, mask planes, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
183.3 |
0.038 |
1.64 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
planes |
183.2 |
0.038 |
1.64 |
auto; same |
pixel |
155.7 |
0.032 |
1.39 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-avx2 |
183.2 |
0.038 |
1.64 |
auto; same |
lowplanes-c |
160.5 |
0.033 |
1.44 |
same |
kcb |
156.4 |
0.032 |
1.40 |
same |
scalar |
51.4 |
0.011 |
0.46 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
156.4 |
0.032 |
1.40 |
same |
sse2 |
150.4 |
0.031 |
1.34 |
same |
scalar |
93.9 |
0.019 |
0.84 |
same |
silicon, zstd-2 8k, sparsify, no mask
Automatic pipeline: decode zstd, mask none, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
282.2 |
0.028 |
2.52 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
282.3 |
0.028 |
2.52 |
auto; same |
pixel |
279.1 |
0.028 |
2.50 |
same |
planes |
247.5 |
0.024 |
2.21 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-avx2 |
282.1 |
0.028 |
2.52 |
auto; same |
lowplanes-c |
213.2 |
0.021 |
1.91 |
same |
kcb |
207.1 |
0.020 |
1.85 |
same |
scalar |
63.7 |
0.006 |
0.57 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
207.3 |
0.020 |
1.85 |
same |
sse2 |
194.8 |
0.019 |
1.74 |
same |
scalar |
109.8 |
0.011 |
0.98 |
same |
silicon, zstd-2 256k, sparsify
Automatic pipeline: decode zstd, mask pixel, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
314.7 |
0.021 |
2.81 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
planes |
315.4 |
0.022 |
2.82 |
same |
pixel |
314.7 |
0.021 |
2.81 |
auto; same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-avx2 |
315.5 |
0.022 |
2.82 |
auto; same |
lowplanes-c |
315.5 |
0.022 |
2.82 |
same |
kcb |
315.4 |
0.022 |
2.82 |
same |
scalar |
71.9 |
0.005 |
0.64 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
315.5 |
0.022 |
2.82 |
same |
sse2 |
308.3 |
0.021 |
2.76 |
same |
scalar |
113.9 |
0.008 |
1.02 |
same |
silicon, zstd-2 256k, sparsify, no mask
Automatic pipeline: decode zstd, mask none, untranspose lowplanes-avx2, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
496.1 |
0.023 |
4.44 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
495.2 |
0.023 |
4.43 |
auto; same |
pixel |
478.7 |
0.023 |
4.28 |
same |
planes |
477.9 |
0.023 |
4.27 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lowplanes-avx2 |
496.0 |
0.023 |
4.43 |
auto; same |
lowplanes-c |
347.8 |
0.016 |
3.11 |
same |
kcb |
316.4 |
0.015 |
2.83 |
same |
scalar |
72.3 |
0.003 |
0.65 |
same |
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
315.9 |
0.015 |
2.82 |
same |
sse2 |
294.2 |
0.014 |
2.63 |
same |
scalar |
136.3 |
0.006 |
1.22 |
same |
silicon, zstd-2 256k, 1D matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
267.9 |
0.018 |
2.40 |
same; powder exact |
bpp |
267.7 |
0.018 |
2.39 |
auto; same; powder exact |
sparse |
267.4 |
0.018 |
2.39 |
same; powder exact |
dense |
60.7 |
0.004 |
0.54 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded |
267.7 |
0.018 |
2.39 |
same; powder exact |
padded-avx2 |
267.5 |
0.018 |
2.39 |
auto; same; powder exact |
csc |
260.9 |
0.018 |
2.33 |
same; powder exact |
csc-run |
254.1 |
0.017 |
2.27 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, zstd-2 256k, 1D no-split matrix
Automatic pipeline: route bpp, dot csc-nosplit-int. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
303.5 |
0.021 |
2.71 |
auto; same; powder exact |
sparse |
303.3 |
0.021 |
2.71 |
same; powder exact |
ratio |
303.1 |
0.021 |
2.71 |
same; powder exact |
dense |
135.5 |
0.009 |
1.21 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc-nosplit-int |
303.1 |
0.021 |
2.71 |
auto; same; powder exact |
csc-nosplit |
302.1 |
0.021 |
2.70 |
same; powder exact |
padded-avx2 |
286.6 |
0.020 |
2.56 |
same; powder exact |
padded |
285.8 |
0.020 |
2.56 |
same; powder exact |
csc |
277.3 |
0.019 |
2.48 |
same; powder exact |
csc-run |
269.2 |
0.018 |
2.41 |
same; powder exact |
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'csc-nosplit-int') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'csc-nosplit-int') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4200640 bins are reached by more than one pixel (auto picks 'csc-nosplit-int') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
silicon, zstd-2 256k, 1D fine matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
244.6 |
0.017 |
2.19 |
auto; same; powder exact |
sparse |
244.4 |
0.017 |
2.19 |
same; powder exact |
ratio |
244.0 |
0.017 |
2.18 |
same; powder exact |
dense |
43.9 |
0.003 |
0.39 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
246.5 |
0.017 |
2.20 |
same; powder exact |
padded |
244.2 |
0.017 |
2.18 |
same; powder exact |
padded-avx2 |
244.0 |
0.017 |
2.18 |
auto; same; powder exact |
csc-run |
224.3 |
0.015 |
2.01 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
silicon, zstd-2 256k, 2D matrix
Automatic pipeline: route bpp, dot csc. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
234.9 |
0.016 |
2.10 |
auto; same; powder exact |
sparse |
234.7 |
0.016 |
2.10 |
same; powder exact |
ratio |
233.8 |
0.016 |
2.09 |
same; powder exact |
dense |
35.9 |
0.002 |
0.32 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
235.7 |
0.016 |
2.11 |
auto; same; powder exact |
padded |
refused: pipeline['dot']='padded' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
padded-avx2 |
refused: pipeline['dot']='padded-avx2' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
csc-run |
refused: pipeline['dot']='csc-run' cannot represent this matrix: 4197949 of 4198000 pixels with several bins reach bins that are not one ascending run of consecutive bins (auto picks 'csc') |
|
|
|
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'csc') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
silicon, zstd-2 256k, 2D+rings matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
296.7 |
0.020 |
2.65 |
same; powder exact |
sparse |
296.4 |
0.020 |
2.65 |
same; powder exact |
bpp |
294.2 |
0.020 |
2.63 |
auto; same; powder exact |
dense |
241.2 |
0.016 |
2.16 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
293.7 |
0.020 |
2.63 |
same; powder exact |
padded-avx2 |
289.5 |
0.020 |
2.59 |
auto; same; powder exact |
padded |
289.1 |
0.020 |
2.59 |
same; powder exact |
csc-run |
286.4 |
0.020 |
2.56 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr |
refused: pipeline['dot']='bsb-csr' cannot represent this matrix: the decode blocks have 131072 pixels and bsb-csr takes at most 65536 (its in-block index is uint16) (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2: dense powder rings, uint32
ceo2_lz4mask.h5 (/entry_0000/ESRF-ID11/eiger/data, 2162 x 2068 uint32, 268834 masked pixels, cut 150). Encodings:
| encoding |
codec |
level |
block |
compression |
tables |
| lz4 8k |
lz4 |
- |
8 kB |
4.5 |
sparsify, matrices |
| zstd-2 256k |
zstd |
2 |
256 kB |
4.9 |
sparsify |
ceo2, lz4 8k, sparsify
Automatic pipeline: decode lz4-band, mask pixel, untranspose kcb, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-stock |
178.1 |
0.716 |
3.19 |
same |
lz4-zero |
166.9 |
0.670 |
2.98 |
same |
lz4-band |
165.5 |
0.665 |
2.96 |
auto; same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
pixel |
166.7 |
0.670 |
2.98 |
auto; same |
planes |
160.0 |
0.643 |
2.86 |
same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
166.8 |
0.670 |
2.98 |
auto; same |
scalar |
35.9 |
0.144 |
0.64 |
same |
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
166.8 |
0.670 |
2.98 |
auto; same |
sse2 |
153.2 |
0.615 |
2.74 |
same |
scalar |
89.3 |
0.359 |
1.60 |
same |
ceo2, lz4 8k, sparsify, no mask
Automatic pipeline: decode lz4-band, mask none, untranspose kcb, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
lz4-zero |
195.4 |
0.768 |
3.50 |
same |
lz4-band |
194.9 |
0.766 |
3.49 |
auto; same |
lz4-stock |
141.4 |
0.556 |
2.53 |
same |
pixels |
refused: pipeline['decode']='pixels' is for uncompressed frames: call from_frames() instead |
|
|
|
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
195.1 |
0.767 |
3.49 |
auto; same |
planes |
194.0 |
0.763 |
3.47 |
same |
pixel |
188.3 |
0.740 |
3.37 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
195.3 |
0.767 |
3.49 |
auto; same |
scalar |
36.9 |
0.145 |
0.66 |
same |
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
195.0 |
0.767 |
3.49 |
auto; same |
sse2 |
168.8 |
0.663 |
3.02 |
same |
scalar |
110.1 |
0.433 |
1.97 |
same |
ceo2, lz4 8k, 1D matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
46.8 |
0.188 |
0.84 |
auto; same; powder exact |
ratio |
46.8 |
0.188 |
0.84 |
same; powder exact |
dense |
45.1 |
0.181 |
0.81 |
same; powder exact |
sparse |
26.5 |
0.106 |
0.47 |
same; powder 1.2e-16 |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded-avx2 |
46.8 |
0.188 |
0.84 |
auto; same; powder exact |
bsb-csr |
44.7 |
0.180 |
0.80 |
same; powder 1.2e-16 |
csc |
43.2 |
0.173 |
0.77 |
same; powder 1.2e-16 |
csc-run |
41.9 |
0.168 |
0.75 |
same; powder 1.2e-16 |
padded |
39.6 |
0.159 |
0.71 |
same; powder 1.2e-16 |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202181 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2, lz4 8k, 1D no-split matrix
Automatic pipeline: route bpp, dot csc-nosplit-int. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
100.0 |
0.402 |
1.79 |
auto; same; powder exact |
ratio |
99.7 |
0.401 |
1.78 |
same; powder exact |
dense |
97.7 |
0.392 |
1.75 |
same; powder exact |
sparse |
49.2 |
0.197 |
0.88 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc-nosplit-int |
99.7 |
0.401 |
1.78 |
auto; same; powder exact |
csc-nosplit |
83.8 |
0.337 |
1.50 |
same; powder exact |
padded-avx2 |
75.6 |
0.304 |
1.35 |
same; powder exact |
bsb-csr-nosplit |
75.6 |
0.304 |
1.35 |
same; powder exact |
csc-run |
67.7 |
0.272 |
1.21 |
same; powder exact |
csc |
64.2 |
0.258 |
1.15 |
same; powder exact |
bsb-csr |
62.3 |
0.250 |
1.11 |
same; powder exact |
padded |
59.2 |
0.238 |
1.06 |
same; powder exact |
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4200640 bins are reached by more than one pixel (auto picks 'csc-nosplit-int') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc-nosplit-int') |
|
|
|
ceo2, lz4 8k, 1D fine matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
ratio |
44.9 |
0.180 |
0.80 |
same; powder exact |
bpp |
44.8 |
0.180 |
0.80 |
auto; same; powder exact |
dense |
43.0 |
0.173 |
0.77 |
same; powder exact |
sparse |
19.9 |
0.080 |
0.36 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded-avx2 |
44.2 |
0.177 |
0.79 |
auto; same; powder exact |
padded |
23.9 |
0.096 |
0.43 |
same; powder exact |
csc-run |
23.6 |
0.095 |
0.42 |
same; powder exact |
csc |
23.0 |
0.092 |
0.41 |
same; powder exact |
bsb-csr |
19.0 |
0.076 |
0.34 |
same; powder 5.3e-17 |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4202182 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2, lz4 8k, 2D matrix
Automatic pipeline: route bpp, dot csc. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
32.3 |
0.130 |
0.58 |
auto; same; powder exact |
ratio |
32.3 |
0.130 |
0.58 |
same; powder exact |
dense |
32.1 |
0.129 |
0.57 |
same; powder exact |
sparse |
16.3 |
0.066 |
0.29 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
csc |
32.3 |
0.130 |
0.58 |
auto; same; powder exact |
bsb-csr |
29.5 |
0.119 |
0.53 |
same; powder 3.4e-17 |
padded |
refused: pipeline['dot']='padded' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
padded-avx2 |
refused: pipeline['dot']='padded-avx2' cannot represent this matrix: a pixel reaches 88 bins (padded takes at most 64) (auto picks 'csc') |
|
|
|
csc-run |
refused: pipeline['dot']='csc-run' cannot represent this matrix: 4197949 of 4198000 pixels with several bins reach bins that are not one ascending run of consecutive bins (auto picks 'csc') |
|
|
|
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'csc') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 4198000 pixels reach more than one bin (auto picks 'csc') |
|
|
|
ceo2, lz4 8k, 2D+rings matrix
Automatic pipeline: route bpp, dot padded-avx2. Each table varies one step, the others held there.
| route |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
bpp |
141.2 |
0.567 |
2.53 |
auto; same; powder exact |
dense |
141.1 |
0.567 |
2.52 |
same; powder exact |
ratio |
140.6 |
0.565 |
2.51 |
same; powder exact |
sparse |
41.5 |
0.167 |
0.74 |
same; powder exact |
| dot |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
padded-avx2 |
141.2 |
0.567 |
2.53 |
auto; same; powder exact |
bsb-csr |
138.3 |
0.556 |
2.47 |
same; powder exact |
padded |
133.5 |
0.536 |
2.39 |
same; powder exact |
csc |
98.1 |
0.394 |
1.75 |
same; powder exact |
csc-run |
86.9 |
0.349 |
1.55 |
same; powder exact |
csc-nosplit |
refused: pipeline['dot']='csc-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
bsb-csr-nosplit |
refused: pipeline['dot']='bsb-csr-nosplit' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-permute |
refused: pipeline['dot']='csc-permute' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
csc-int |
refused: pipeline['dot']='csc-int' cannot represent this matrix: csc-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int |
refused: pipeline['dot']='csc-run-int' cannot represent this matrix: csc-run-int needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-run-int16 |
refused: pipeline['dot']='csc-run-int16' cannot represent this matrix: csc-run-int16 needs the matrix prepared by fixed_point_matrix (auto picks 'padded') |
|
|
|
csc-nosplit-int |
refused: pipeline['dot']='csc-nosplit-int' cannot represent this matrix: 7278 pixels reach more than one bin (auto picks 'padded') |
|
|
|
ceo2, zstd-2 256k, sparsify
Automatic pipeline: decode zstd, mask pixel, untranspose kcb, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
119.4 |
0.440 |
2.14 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
planes |
119.5 |
0.441 |
2.14 |
same |
pixel |
119.3 |
0.440 |
2.13 |
auto; same |
none |
refused: pipeline['mask']='none': the mask has 268834 masked pixels (use 'pixel' or 'planes') |
|
|
|
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
119.6 |
0.441 |
2.14 |
auto; same |
scalar |
33.7 |
0.124 |
0.60 |
same |
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
119.5 |
0.441 |
2.14 |
auto; same |
sse2 |
112.5 |
0.415 |
2.01 |
same |
scalar |
73.8 |
0.272 |
1.32 |
same |
ceo2, zstd-2 256k, sparsify, no mask
Automatic pipeline: decode zstd, mask none, untranspose kcb, collect avx2cs. Each table varies one step, the others held there (the collect table holds untranspose kcb: the low-planes values fuse the collect).
| decode |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
zstd |
120.4 |
0.440 |
2.15 |
auto; same |
| mask |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
none |
120.4 |
0.440 |
2.15 |
auto; same |
pixel |
117.6 |
0.430 |
2.10 |
same |
planes |
117.5 |
0.430 |
2.10 |
same |
| untranspose |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
kcb |
120.4 |
0.440 |
2.15 |
auto; same |
scalar |
33.9 |
0.124 |
0.61 |
same |
lowplanes-avx2 |
refused: pipeline['untranspose']='lowplanes-avx2' is for u16 pixels, not uint32 (use kcb) |
|
|
|
lowplanes-c |
refused: pipeline['untranspose']='lowplanes-c' is for u16 pixels, not uint32 (use kcb) |
|
|
|
| collect |
frames/s |
compressed GB/s |
pixels GB/s |
vs auto |
avx2cs |
120.4 |
0.440 |
2.15 |
auto; same |
sse2 |
109.8 |
0.402 |
1.96 |
same |
scalar |
82.1 |
0.300 |
1.47 |
same |
Source: docs/bench/kernels_xeon-gold-6248.json, written by tools/bench_kernels.py.
Generated by tools/generate_docs.py from bslz4_to_sparse 0.0.22a2 (git v0.0.22-13-g610ed27, sources sha256 6929bd75ab7a).