[igt-dev] [PATCH igt] RFC tools: capture execution pathways

[Chris Wilson]

Quoting Chris Wilson (2018-04-20 22:50:33)
> igt_kcov edges -o edges.yaml command-to-trace args
> ...
> igt_kcov sort -o sorted.yaml edges.yaml
> 
> Requires CONFIG_KCOV
> 
> Use LD_PRELOAD to wrap calls to DRM ioctls and capture the execution
> trace of all basic blocks invoked directly from the syscall. (Key
> limitation!) From the chain of basic blocks, the "edges" are computed
> and the number of times each edge is hit is stored in a vector (modulo a
> large hash). The goal then is to select a series of tests that execute
> the most unique pathways in the sortest amount of time. To do this we
> compare the similarity of the coverage vectors between all commands, and
> try to preferentially pick those that are dissimilar from the rest.
> 
> * Direct execution pathways from syscall is a huge limitation for
> evaluating general testing of driver pathways, but is not so huge when
> myopically focusing on the ABI as say used by mesa.
> 
> ** Caveat lector. Hastily thrown together.

So after some more hacking around, here's the top 30s of "piglit run -p
gdm gpu" on a bxt (try not to overdose on the pinch of salt required):

'bin/draw-sync' '-auto' '-fbo' # 9723.773ms, total 9723.8ms
'bin/fbo-depthstencil' 'drawpixels' 'default_fb' '32F_24_8_REV' '-samples=32' '-auto' # 38.638ms, total 9762.4ms
'bin/gl30basic' '-auto' # 49.098ms, total 9811.5ms
'bin/tex3d-maxsize' '-auto' # 8921.198ms, total 18732.7ms
'bin/copyteximage' 'RECT' '-samples=32' '-auto' # 43.987ms, total 18776.7ms
'bin/copypixels-sync' '-auto' '-fbo' # 2843.007ms, total 21619.7ms
'bin/fbo-sys-sub-blit' '-auto' # 58.708ms, total 21678.4ms
'bin/draw-copypixels-sync' '-auto' '-fbo' # 6022.916ms, total 27701.3ms
'bin/fbo-depthstencil' 'readpixels' 'default_fb' 'FLOAT-and-USHORT' '-samples=32' '-auto' # 32.175ms, total 27733.5ms
'bin/getteximage-depth' '-auto' '-fbo' # 181.520ms, total 27915.0ms
'bin/recursion' '-rlmit' '268435456' 'indirect-complex-separate' '-auto' # 47.082ms, total 27962.1ms
'bin/read-front' 'clear-front-first' '-samples=16' '-auto' # 50.148ms, total 28012.3ms
'bin/texrect-many' '-auto' # 123.456ms, total 28135.7ms
'bin/read-front' '-samples=32' '-auto' # 35.239ms, total 28170.9ms
'bin/fbo-depthstencil' 'clear' 'default_fb' '-samples=32' '-auto' # 39.926ms, total 28210.9ms
'bin/teximage-colors' 'GL_RGBA32F' '-auto' '-fbo' # 164.443ms, total 28375.3ms
'bin/copy_buffer_coherency' '-auto' # 109.981ms, total 28485.3ms
'bin/glsl-explicit-location-04' '-auto' # 66.258ms, total 28551.6ms
'bin/gl-3.2-layered-rendering-clear-color-all-types' 'cube_map_array' 'mipmapped' '-auto' '-fbo' # 121.422ms, total 28673.0ms
'bin/arb_transform_feedback3-ext_interleaved_two_bufs' 'vs' '-auto' # 67.861ms, total 28740.8ms
'bin/copyteximage' 'CUBE' '-samples=32' '-auto' # 30.807ms, total 28771.6ms
'bin/activeprogram-get' '-auto' '-fbo' # 44.429ms, total 28816.1ms
'bin/clear-accum' '-auto' '-fbo' # 35.903ms, total 28852.0ms
'bin/gl-1.3-alpha_to_coverage_nop' '-auto' '-fbo' # 713.298ms, total 29565.3ms
'bin/arb_program_interface_query-getprogramresourceiv' '-auto' # 115.116ms, total 29680.4ms
'bin/max-samplers' 'border' '-auto' '-fbo' # 208.097ms, total 29888.5ms
'bin/masked-clear' '-auto' '-fbo' # 45.091ms, total 29933.6ms
'bin/copyteximage' '2D_ARRAY' '-samples=32' '-auto' # 34.492ms, total 29968.1ms
'bin/fbo-depthstencil' 'drawpixels' 'default_fb' '24_8' '-samples=32' '-auto' # 27.307ms, total 29995.4ms

These are ranked by uniqueness of the basic blocks they execute from
drmIoctls. If we exclude any test that runs longer than 0.5s:

'bin/fbo-depthstencil' 'drawpixels' 'default_fb' '32F_24_8_REV' '-samples=32' '-auto' # 38.638ms, total 38.6ms
'bin/gl30basic' '-auto' # 49.098ms, total 87.7ms
'bin/copyteximage' 'RECT' '-samples=32' '-auto' # 43.987ms, total 131.7ms
'bin/fbo-sys-sub-blit' '-auto' # 58.708ms, total 190.4ms
'bin/fbo-depthstencil' 'readpixels' 'default_fb' 'FLOAT-and-USHORT' '-samples=32' '-auto' # 32.175ms, total 222.6ms
'bin/getteximage-depth' '-auto' '-fbo' # 181.520ms, total 404.1ms
'bin/recursion' '-rlmit' '268435456' 'indirect-complex-separate' '-auto' # 47.082ms, total 451.2ms
'bin/read-front' 'clear-front-first' '-samples=16' '-auto' # 50.148ms, total 501.4ms
'bin/texrect-many' '-auto' # 123.456ms, total 624.8ms
'bin/read-front' '-samples=32' '-auto' # 35.239ms, total 660.1ms
'bin/fbo-depthstencil' 'clear' 'default_fb' '-samples=32' '-auto' # 39.926ms, total 700.0ms
'bin/teximage-colors' 'GL_RGBA32F' '-auto' '-fbo' # 164.443ms, total 864.4ms
'bin/copy_buffer_coherency' '-auto' # 109.981ms, total 974.4ms
'bin/glsl-explicit-location-04' '-auto' # 66.258ms, total 1040.7ms
'bin/gl-3.2-layered-rendering-clear-color-all-types' 'cube_map_array' 'mipmapped' '-auto' '-fbo' # 121.422ms, total 1162.1ms
'bin/arb_transform_feedback3-ext_interleaved_two_bufs' 'vs' '-auto' # 67.861ms, total 1229.9ms
'bin/copyteximage' 'CUBE' '-samples=32' '-auto' # 30.807ms, total 1260.7ms
'bin/activeprogram-get' '-auto' '-fbo' # 44.429ms, total 1305.2ms
'bin/clear-accum' '-auto' '-fbo' # 35.903ms, total 1341.1ms
'bin/arb_program_interface_query-getprogramresourceiv' '-auto' # 115.116ms, total 1456.2ms
'bin/max-samplers' 'border' '-auto' '-fbo' # 208.097ms, total 1664.3ms
'bin/masked-clear' '-auto' '-fbo' # 45.091ms, total 1709.4ms
'bin/gen-nonzero-unit' '-auto' # 98.092ms, total 1807.5ms
'bin/copyteximage' '2D_ARRAY' '-samples=32' '-auto' # 34.492ms, total 1842.0ms
'bin/arb_transform_feedback2-change-objects-while-paused_gles3' '-auto' # 87.031ms, total 1929.0ms
'bin/gl-3.0-required-texture-attachment-formats' '33' '-auto' '-fbo' # 51.791ms, total 1980.8ms
'bin/ext_image_dma_buf_import-invalid_hints' '-auto' # 45.979ms, total 2026.8ms
'bin/read-front' 'clear-front-first' '-samples=8' '-auto' # 71.906ms, total 2098.7ms
'bin/copyteximage' 'CUBE' '-samples=2' '-auto' # 268.906ms, total 2367.6ms
'bin/fbo-depthstencil' 'drawpixels' 'default_fb' '24_8' '-samples=32' '-auto' # 27.307ms, total 2394.9ms
'bin/ext_texture_array-compressed_gles3' 'teximage' '-fbo' '-auto' # 73.392ms, total 2468.3ms
'bin/gl-3.0-required-sized-texture-formats' '31' '-auto' '-fbo' # 52.273ms, total 2520.6ms
'bin/amd_seamless_cubemap_per_texture' '-auto' # 62.070ms, total 2582.6ms
'bin/copyteximage' '1D_ARRAY' '-auto' # 375.446ms, total 2958.1ms
'bin/fcc-front-buffer-distraction' '-auto' # 59.025ms, total 3017.1ms
'bin/read-front' '-samples=16' '-auto' # 104.732ms, total 3121.8ms
'bin/read-front' 'clear-front-first' '-samples=32' '-auto' # 27.369ms, total 3149.2ms
'bin/arb_base_instance-baseinstance-doesnt-affect-gl-instance-id_gles3' '-auto' # 37.624ms, total 3186.8ms
'bin/gl-3.0-bound-resource-limits' '-auto' '-fbo' # 154.116ms, total 3340.9ms
'bin/tex3d' '-auto' '-fbo' # 427.575ms, total 3768.5ms
'bin/fbo-depthstencil' 'readpixels' 'default_fb' '32F_24_8_REV' '-auto' # 82.329ms, total 3850.8ms
'bin/arb_es2_compatibility-getshaderprecisionformat' '-auto' # 53.062ms, total 3903.9ms
'bin/copyteximage' '1D_ARRAY' '-samples=32' '-auto' # 34.307ms, total 3938.2ms
'bin/depthstencil-render-miplevels' '146' 'd=z32f_s8' '-auto' # 66.579ms, total 4004.8ms
'bin/arb_arrays_of_arrays-max-binding' '-auto' # 70.830ms, total 4075.6ms
'bin/copyteximage' 'RECT' '-samples=2' '-auto' # 215.228ms, total 4290.8ms
'bin/gl-1.0-front-invalidate-back' '-auto' # 41.917ms, total 4332.8ms
'bin/fbo-depthstencil' 'readpixels' 'default_fb' '32F_24_8_REV' '-samples=32' '-auto' # 36.760ms, total 4369.5ms
'bin/incomplete-cubemap' 'size' '-auto' '-fbo' # 79.044ms, total 4448.6ms
'bin/arb_sample_shading-builtin-gl-num-samples' '16' '-auto' # 105.381ms, total 4553.9ms
'bin/teximage-errors' '-auto' '-fbo' # 45.772ms, total 4599.7ms
'bin/max-samplers' '-auto' '-fbo' # 208.750ms, total 4808.5ms
'bin/vp-address-06' '-auto' # 49.582ms, total 4858.1ms
'bin/read-front' 'clear-front-first' '-samples=6' '-auto' # 50.643ms, total 4908.7ms
'bin/glsl-explicit-location-02' '-auto' # 41.985ms, total 4950.7ms
'bin/fbo-depthstencil' 'readpixels' 'default_fb' '24_8' '-samples=32' '-auto' # 32.637ms, total 4983.3ms
'bin/copyteximage' '1D_ARRAY' '-samples=4' '-auto' # 316.604ms, total 5299.9ms
'bin/gl-3.0-required-sized-texture-formats' '33' '-auto' '-fbo' # 47.754ms, total 5347.7ms
'bin/arb_transform_feedback2-change-objects-while-paused' '-auto' # 54.757ms, total 5402.4ms
'bin/texture-packed-formats' '-auto' '-fbo' # 395.531ms, total 5798.0ms
'bin/vertex-program-two-side' 'enabled' 'back' 'back2' '-auto' '-fbo' # 53.321ms, total 5851.3ms
'bin/read-front' 'clear-front-first' '-auto' # 51.246ms, total 5902.5ms
'bin/gl-1.0-scissor-many' '-auto' '-fbo' # 338.981ms, total 6241.5ms
'bin/fbo-depthstencil' 'drawpixels' 'default_fb' 'FLOAT-and-USHORT' '-samples=32' '-auto' # 47.502ms, total 6289.0ms
'bin/read-front' '-samples=2' '-auto' # 62.456ms, total 6351.5ms
'bin/arb_sample_shading-builtin-gl-sample-position' '32' '-auto' # 36.846ms, total 6388.3ms
'bin/rg-teximage-01' '-auto' # 80.979ms, total 6469.3ms
'bin/gl-1.0-dlist-bitmap' '-auto' '-fbo' # 195.049ms, total 6664.3ms
'bin/gl-3.0-required-sized-texture-formats' '30' '-auto' '-fbo' # 48.607ms, total 6713.0ms
'bin/cubemap-shader' 'lod' '-auto' # 113.548ms, total 6826.5ms
'bin/fbo-depthstencil' 'copypixels' 'default_fb' '-samples=32' '-auto' # 27.107ms, total 6853.6ms
'bin/fbo-depthstencil' 'readpixels' 'default_fb' 'FLOAT-and-USHORT' '-samples=6' '-auto' # 73.768ms, total 6927.4ms
'bin/read-front' 'clear-front-first' '-samples=4' '-auto' # 73.699ms, total 7001.1ms
'bin/gl-3.2-get-integer-64v' '-auto' '-fbo' # 46.387ms, total 7047.5ms
'bin/ext_image_dma_buf_import-transcode-nv12-as-r8-gr88' '-auto' # 59.923ms, total 7107.4ms
'bin/gl-1.0-drawbuffer-modes' '-auto' # 47.332ms, total 7154.7ms
'bin/ext_texture_array-compressed_gles3' 'texsubimage' '-fbo' '-auto' # 52.964ms, total 7207.7ms
'bin/fbo-depthstencil' 'blit' 'default_fb' '-samples=32' '-auto' # 26.703ms, total 7234.4ms
'bin/gl-1.0-logicop' '-auto' '-fbo' # 109.021ms, total 7343.4ms
'bin/gl-3.0-required-sized-texture-formats' '42' '-auto' '-fbo' # 46.672ms, total 7390.1ms
'bin/primitive-restart-draw-mode' 'lines' '-auto' # 55.387ms, total 7445.5ms
'bin/gl-3.0-required-texture-attachment-formats' '30' '-auto' '-fbo' # 50.314ms, total 7495.8ms
'bin/gen-texsubimage' '-auto' # 90.425ms, total 7586.2ms
'bin/gl-3.2-layered-rendering-clear-color-all-types' 'cube_map' 'mipmapped' '-auto' '-fbo' # 63.288ms, total 7649.5ms
'bin/max-texture-size-level' '-auto' '-fbo' # 168.147ms, total 7817.6ms
'bin/copyteximage' 'CUBE' '-auto' # 254.585ms, total 8072.2ms
'bin/hiz-depth-stencil-test-fbo-d24s8' '-auto' # 70.293ms, total 8142.5ms
'bin/read-front' 'clear-front-first' '-samples=2' '-auto' # 71.141ms, total 8213.7ms
'bin/glsl-vs-raytrace-bug26691' '-auto' '-fbo' # 470.930ms, total 8684.6ms
'bin/teximage-colors' 'GL_RGBA4' '-auto' '-fbo' # 153.488ms, total 8838.1ms
'bin/gl-1.0-long-dlist' '-auto' '-fbo' # 58.517ms, total 8896.6ms
'bin/glsl-fs-fragcoord-zw-ortho' '-auto' '-fbo' # 59.801ms, total 8956.4ms
'bin/copyteximage' '3D' '-auto' '-fbo' # 180.824ms, total 9137.2ms
'bin/gl-1.0-swapbuffers-behavior' '-auto' # 47.067ms, total 9184.3ms
'bin/gl-3.0-required-renderbuffer-attachment-formats' '42' '-auto' '-fbo' # 48.456ms, total 9232.7ms
'bin/cubemap' '-auto' # 272.528ms, total 9505.3ms
'bin/glsl-explicit-location-05' '-auto' # 64.614ms, total 9569.9ms
'bin/copyteximage' '1D_ARRAY' '-samples=8' '-auto' # 324.004ms, total 9893.9ms
'bin/read-front' '-samples=8' '-auto' # 61.458ms, total 9955.3ms
'bin/arb_program_interface_query-getprogramresourceindex' '-auto' # 146.634ms, total 10102.0ms
'bin/arb_enhanced_layouts-transform-feedback-layout-qualifiers' 'gs_max' '-auto' # 100.837ms, total 10202.8ms
'bin/gl-3.2-get-buffer-parameter-i64v' '-auto' '-fbo' # 50.338ms, total 10253.2ms
'bin/ext_transform_feedback-builtin-varyings' 'gl_ClipDistance[6]-no-subscript' '-auto' # 49.990ms, total 10303.1ms
'bin/arb_program_interface_query-resource-location' '-auto' # 75.369ms, total 10378.5ms
'bin/copyteximage' 'RECT' '-samples=4' '-auto' # 148.635ms, total 10527.1ms
'bin/fbo-sys-blit' '-auto' # 65.767ms, total 10592.9ms
'bin/gen-teximage' '-auto' # 77.274ms, total 10670.2ms
'bin/gl-3.2-layered-rendering-framebuffer-layer-count-mismatch' '-auto' '-fbo' # 48.021ms, total 10718.2ms
'bin/arb_sampler_objects-framebufferblit' '-auto' # 70.290ms, total 10788.5ms
'bin/teximage-colors' 'GL_R16F' '-auto' '-fbo' # 147.662ms, total 10936.2ms
'bin/gl-1.2-texture-base-level' '-auto' '-fbo' # 60.154ms, total 10996.3ms
'bin/sized-texture-format-channels' '-auto' '-fbo' # 68.128ms, total 11064.4ms
'bin/texredefine' '-auto' '-fbo' # 77.855ms, total 11142.3ms
'bin/arb_sample_shading-builtin-gl-sample-position' '16' '-auto' # 113.356ms, total 11255.6ms
'bin/copyteximage' '1D_ARRAY' '-samples=2' '-auto' # 366.872ms, total 11622.5ms
'bin/initialized-vbo' '-auto' # 121.142ms, total 11743.7ms
'bin/arb_explicit_uniform_location-use-of-unused-loc' '-auto' # 68.386ms, total 11812.1ms
'bin/gl-1.0-ortho-pos' '-auto' '-fbo' # 87.511ms, total 11899.6ms
'bin/arb_sample_shading-builtin-gl-sample-id' '32' '-auto' # 48.758ms, total 11948.3ms
'bin/teximage-colors' 'GL_ALPHA' '-auto' '-fbo' # 143.929ms, total 12092.2ms
'bin/glsl-vs-mov-after-deref' '-auto' '-fbo' # 52.318ms, total 12144.6ms
'bin/vp-clipdistance-01' '-auto' # 51.502ms, total 12196.1ms
'bin/read-front' '-samples=6' '-auto' # 64.680ms, total 12260.7ms
'bin/copyteximage' 'CUBE' '-samples=16' '-auto' # 249.863ms, total 12510.6ms
'bin/genmipmap-errors' '-auto' '-fbo' # 53.652ms, total 12564.3ms
'bin/fbo-depthstencil' 'blit' 'default_fb' '-auto' # 74.364ms, total 12638.6ms
'bin/arb_vertex_type_10f_11f_11f_rev-api-errors' '-auto' # 76.030ms, total 12714.7ms
'bin/depthstencil-render-miplevels' '585' 's=z24_s8_d=z32f_s8' '-auto' # 164.833ms, total 12879.5ms
'bin/gl-2.0-vertexattribpointer' '-auto' '-fbo' # 104.009ms, total 12983.5ms
'bin/gl-3.1-primitive-restart-xfb' 'generated' '-auto' '-fbo' # 55.630ms, total 13039.1ms
'bin/gl-3.0-required-texture-attachment-formats' '42' '-auto' '-fbo' # 50.500ms, total 13089.6ms
'bin/depthstencil-render-miplevels' '1024' 'd=z32f_s8' '-auto' # 197.145ms, total 13286.8ms
'bin/gl-3.0-texture-integer' '-auto' '-fbo' # 130.958ms, total 13417.7ms
'bin/depthstencil-render-miplevels' '1024' 's=z24_s8' '-auto' # 273.880ms, total 13691.6ms
'bin/gl-1.0-texgen' '-auto' '-fbo' # 76.620ms, total 13768.2ms
'bin/teximage-colors' 'GL_RG8_SNORM' '-auto' '-fbo' # 144.052ms, total 13912.3ms
'bin/ext_image_dma_buf_import-missing_attributes' '-auto' # 40.312ms, total 13952.6ms
'bin/recursion' '-rlmit' '268435456' 'indirect-separate' '-auto' # 69.803ms, total 14022.4ms
'bin/copyteximage' '2D_ARRAY' '-samples=6' '-auto' # 162.608ms, total 14185.0ms
'bin/rg-teximage-02' '-auto' # 115.089ms, total 14300.1ms
'bin/ext_transform_feedback-builtin-varyings' 'gl_ClipDistance[3]-no-subscript' '-auto' # 46.967ms, total 14347.1ms
'bin/gl-3.1-draw-buffers-errors' '-auto' '-fbo' # 45.662ms, total 14392.7ms
'bin/gl-3.2-layered-rendering-framebuffer-layer-complete' '-auto' '-fbo' # 44.793ms, total 14437.5ms
'bin/teximage-colors' 'GL_LUMINANCE12_ALPHA4' '-auto' '-fbo' # 161.042ms, total 14598.6ms
'bin/read-front' '-samples=4' '-auto' # 65.224ms, total 14663.8ms
'bin/copyteximage' '1D_ARRAY' '-samples=6' '-auto' # 364.844ms, total 15028.6ms
'bin/ext_fog_coord-modes' '-auto' # 58.006ms, total 15086.6ms
'bin/map_buffer_range_test' '-auto' # 67.780ms, total 15154.4ms
'bin/vp-clipdistance-03' '-auto' # 39.314ms, total 15193.7ms
'bin/glsl-preprocessor-comments' '-auto' '-fbo' # 53.134ms, total 15246.9ms
'bin/teximage-colors' 'GL_RGBA8_SNORM' '-auto' '-fbo' # 157.683ms, total 15404.5ms
'bin/geterror-invalid-enum' '-auto' '-fbo' # 187.664ms, total 15592.2ms
'bin/arb_enhanced_layouts-transform-feedback-layout-qualifiers' 'vs_struct' '-auto' # 67.596ms, total 15659.8ms
'bin/gl-3.2-layered-rendering-framebuffer-layered-attachments' '-auto' '-fbo' # 42.774ms, total 15702.6ms
'bin/depthstencil-render-miplevels' '292' 'd=s=z32f_s8' '-auto' # 87.815ms, total 15790.4ms
'bin/teximage-colors' 'GL_RED' '-auto' '-fbo' # 186.126ms, total 15976.5ms
'bin/gl-3.0-required-texture-attachment-formats' '31' '-auto' '-fbo' # 53.383ms, total 16029.9ms
'bin/incomplete-cubemap' 'format' '-auto' '-fbo' # 62.008ms, total 16091.9ms
'bin/vp-address-04' '-auto' # 45.814ms, total 16137.7ms
'bin/copyteximage' '2D_ARRAY' '-samples=16' '-auto' # 269.946ms, total 16407.7ms
'bin/glsl-dlist-getattriblocation' '-auto' '-fbo' # 47.843ms, total 16455.5ms
'bin/read-front' '-auto' # 53.942ms, total 16509.5ms
'bin/teximage-colors' 'GL_RGB8' '-auto' '-fbo' # 149.343ms, total 16658.8ms
'bin/fbo-depthstencil' 'drawpixels' 'default_fb' '24_8' '-samples=2' '-auto' # 114.971ms, total 16773.8ms
'bin/cubemap-mismatch' '-auto' # 51.126ms, total 16824.9ms
'bin/arb_transform_feedback3-ext_interleaved_two_bufs' 'gs' '-auto' # 56.760ms, total 16881.7ms
'bin/copyteximage' 'CUBE' '-samples=4' '-auto' # 240.430ms, total 17122.1ms
'bin/depthstencil-render-miplevels' '273' 'd=z32f_s8_s=z24_s8' '-auto' # 78.412ms, total 17200.5ms
'bin/proxy-texture' '-auto' '-fbo' # 46.361ms, total 17246.9ms
'bin/teximage-colors' 'GL_R8' '-auto' '-fbo' # 136.959ms, total 17383.8ms
'bin/depth-tex-modes-glsl' '-auto' '-fbo' # 74.382ms, total 17458.2ms
'bin/primitive-restart' 'VBO_SEPARATE_VERTEX_AND_INDEX' '-auto' # 104.232ms, total 17562.4ms
'bin/roundmode-pixelstore' '-auto' '-fbo' # 51.305ms, total 17613.7ms
'bin/recursion' '-rlmit' '268435456' 'unreachable' '-auto' # 49.374ms, total 17663.1ms
'bin/teximage-colors' 'GL_SRGB8_ALPHA8' '-auto' '-fbo' # 191.689ms, total 17854.8ms
'bin/depthstencil-render-miplevels' '1024' 'd=z32f_s8_s=z24_s8' '-auto' # 389.327ms, total 18244.1ms
'bin/ext_framebuffer_multisample-turn-on-off' '6' '-auto' # 179.773ms, total 18423.9ms
'bin/teximage-colors' 'GL_SLUMINANCE8' '-auto' '-fbo' # 189.052ms, total 18613.0ms
'bin/gl-3.0-required-renderbuffer-attachment-formats' '31' '-auto' '-fbo' # 45.732ms, total 18658.7ms
'bin/arb_sample_shading-api' '-auto' # 53.567ms, total 18712.3ms
'bin/gl-3.2-adj-prims' 'line' 'cull-front' 'pv-first' '-auto' '-fbo' # 63.143ms, total 18775.4ms
'bin/copyteximage' '2D_ARRAY' '-samples=8' '-auto' # 260.001ms, total 19035.4ms
'bin/ext_image_dma_buf_import-sample_yuv' '-fmt=YU12' '-alpha-one' '-auto' # 50.441ms, total 19085.8ms
'bin/arb_program_interface_query-resource-query' '-auto' # 301.738ms, total 19387.6ms
'bin/teximage-colors' 'GL_ALPHA8' '-auto' '-fbo' # 142.210ms, total 19529.8ms
'bin/gl-3.2-adj-prims' 'cull-back' 'pv-first' '-auto' '-fbo' # 54.782ms, total 19584.6ms
'bin/glsl-uniform-out-of-bounds' '-auto' '-fbo' # 77.130ms, total 19661.7ms
'bin/depthstencil-render-miplevels' '273' 'd=s=z32f_s8' '-auto' # 99.332ms, total 19761.0ms
'bin/cubemap' 'npot' '-auto' # 206.273ms, total 19967.3ms
'bin/depthstencil-render-miplevels' '585' 's=z24_s8' '-auto' # 113.819ms, total 20081.1ms
'bin/teximage-colors' 'GL_LUMINANCE12_ALPHA12' '-auto' '-fbo' # 159.678ms, total 20240.8ms
'bin/arb_debug_output-api_error' '-auto' # 55.500ms, total 20296.3ms
'bin/arb_enhanced_layouts-transform-feedback-layout-qualifiers' 'gs' '-auto' # 64.531ms, total 20360.8ms
'bin/arb_texture_cube_map_array-cubemap-lod' '-auto' # 137.990ms, total 20498.8ms
'bin/gl-3.0-required-renderbuffer-attachment-formats' '30' '-auto' '-fbo' # 59.196ms, total 20558.0ms
'bin/copyteximage' '2D_ARRAY' '-auto' # 168.361ms, total 20726.4ms
'bin/ext_image_dma_buf_import-ownership_transfer' '-auto' # 42.445ms, total 20768.8ms
'bin/pbo-teximage-tiling' '-auto' # 87.060ms, total 20855.9ms
'bin/depthstencil-render-miplevels' '585' 'd=s=z32f_s8' '-auto' # 206.428ms, total 21062.3ms
'bin/glsl-resource-not-bound' 'Cube' '-auto' '-fbo' # 67.890ms, total 21130.2ms
'bin/copyteximage' 'RECT' '-auto' # 123.651ms, total 21253.9ms
'bin/arb_transform_feedback3-draw_using_invalid_stream_index' '-auto' # 39.585ms, total 21293.4ms
'bin/glsl-mat-attribute' '-auto' '-fbo' # 63.779ms, total 21357.2ms
'bin/teximage-colors' 'GL_R11F_G11F_B10F' '-auto' '-fbo' # 137.933ms, total 21495.2ms
'bin/depthstencil-render-miplevels' '1024' 'd=z32f' '-auto' # 145.291ms, total 21640.4ms
'bin/sso-uniforms-01' '-auto' '-fbo' # 59.910ms, total 21700.4ms
'bin/rgtc-teximage-02' '-auto' # 350.363ms, total 22050.7ms
'bin/vertex-program-two-side' 'back' 'front2' 'back2' '-auto' '-fbo' # 51.341ms, total 22102.1ms
'bin/teximage-colors' 'GL_ALPHA16' '-auto' '-fbo' # 160.668ms, total 22262.7ms
'bin/integer-errors' '-auto' '-fbo' # 45.788ms, total 22308.5ms
'bin/copyteximage' '2D_ARRAY' '-samples=2' '-auto' # 172.109ms, total 22480.6ms
'bin/ext_texture_array-compressed' 'teximage' 'pbo' '-fbo' '-auto' # 65.308ms, total 22545.9ms
'bin/ext_framebuffer_multisample-turn-on-off' '32' '-auto' # 48.415ms, total 22594.3ms
'bin/gl-1.0-scissor-polygon' '-auto' '-fbo' # 163.852ms, total 22758.2ms
'bin/depthstencil-render-miplevels' '146' 's=d=z32f_s8' '-auto' # 70.495ms, total 22828.7ms
'bin/copyteximage' '2D' '-auto' '-fbo' # 147.559ms, total 22976.3ms
'bin/geterror-inside-begin' '-auto' '-fbo' # 48.789ms, total 23025.0ms
'bin/depthstencil-render-miplevels' '1024' 's=z24_s8_d=z32f_s8' '-auto' # 343.656ms, total 23368.7ms
'bin/fbo-depthstencil' 'clear' 'default_fb' '-samples=6' '-auto' # 79.205ms, total 23447.9ms
'bin/teximage-colors' 'GL_RGBA16_SNORM' '-auto' '-fbo' # 150.670ms, total 23598.6ms
'bin/arb-provoking-vertex-control' '-auto' # 76.680ms, total 23675.2ms
'bin/fbo-depthstencil' 'readpixels' 'default_fb' '24_8' '-samples=8' '-auto' # 76.620ms, total 23751.9ms
'bin/teximage-colors' 'GL_LUMINANCE' '-auto' '-fbo' # 161.753ms, total 23913.6ms
'bin/gl-3.2-layered-rendering-gl-layer-cube-map' '-auto' '-fbo' # 73.565ms, total 23987.2ms
'bin/gl-3.3-minmax' '-auto' '-fbo' # 46.301ms, total 24033.5ms
'bin/copyteximage' 'CUBE' '-samples=8' '-auto' # 271.276ms, total 24304.8ms
'bin/vao-02' '-auto' # 60.266ms, total 24365.0ms
'bin/ext_framebuffer_multisample-turn-on-off' '16' '-auto' # 234.869ms, total 24599.9ms
'bin/depthstencil-render-miplevels' '273' 'd=z16' '-auto' # 67.602ms, total 24667.5ms
'bin/gl-3.0-required-renderbuffer-attachment-formats' '33' '-auto' '-fbo' # 50.558ms, total 24718.1ms
'bin/ext_image_dma_buf_import-sample_yuv' '-fmt=YV12' '-alpha-one' '-auto' # 42.094ms, total 24760.2ms
'bin/teximage-colors' 'GL_R16' '-auto' '-fbo' # 152.071ms, total 24912.2ms
'bin/arb_sample_shading-builtin-gl-sample-mask' '0' '-auto' # 118.550ms, total 25030.8ms
'bin/bindfragdata-link-error' '-auto' '-fbo' # 56.134ms, total 25086.9ms
'bin/arb_robustness_client-mem-bounds' '-auto' # 81.317ms, total 25168.2ms
'bin/vp-clipdistance-02' '-auto' # 55.045ms, total 25223.3ms
'bin/fbo-depthstencil' 'drawpixels' 'default_fb' 'FLOAT-and-USHORT' '-samples=6' '-auto' # 115.834ms, total 25339.1ms
'bin/teximage-colors' 'GL_RGBA' '-auto' '-fbo' # 162.751ms, total 25501.9ms
'bin/copyteximage-clipping' '-auto' '-fbo' # 56.837ms, total 25558.7ms
'bin/quad-invariance' '-auto' '-fbo' # 147.417ms, total 25706.1ms
'bin/teximage-colors' 'GL_SLUMINANCE8_ALPHA8' '-auto' '-fbo' # 211.445ms, total 25917.6ms
'bin/rgtc-teximage-01' '-auto' # 215.997ms, total 26133.6ms
'bin/arb_map_buffer_alignment-sanity_test' '-auto' # 41.860ms, total 26175.4ms
'bin/teximage-colors' 'GL_ALPHA4' '-auto' '-fbo' # 152.335ms, total 26327.7ms
'bin/gl-3.2-layered-rendering-framebuffer-layer-attachment-mismatch' '-auto' '-fbo' # 49.223ms, total 26377.0ms
'bin/ext_texture_array-compressed' 'texsubimage' 'pbo' '-fbo' '-auto' # 74.147ms, total 26451.1ms
'bin/oes_draw_elements_base_vertex-drawrangeelements' '-auto' # 63.079ms, total 26514.2ms
'bin/copyteximage' 'CUBE' '-samples=6' '-auto' # 297.885ms, total 26812.1ms
'bin/arb_vertex_program-getlocal4d-with-error' '-auto' # 43.682ms, total 26855.8ms
'bin/gl-3.2-layered-rendering-clear-color-mismatched-layer-count' '-auto' '-fbo' # 89.791ms, total 26945.6ms
'bin/depthstencil-render-miplevels' '1024' 'd=z32f_s=z24_s8' '-auto' # 349.357ms, total 27294.9ms
'bin/fbo-blit-stretch' '-auto' # 292.928ms, total 27587.8ms
'bin/arb_transform_feedback3-set_varyings_with_invalid_args' '-auto' # 40.395ms, total 27628.2ms
'bin/fbo-depthstencil' 'drawpixels' 'default_fb' '32F_24_8_REV' '-samples=4' '-auto' # 100.551ms, total 27728.8ms
'bin/getuniform-02' '-auto' '-fbo' # 55.287ms, total 27784.1ms
'bin/crossbar' '-auto' # 99.245ms, total 27883.3ms
'bin/teximage-colors' 'GL_RGB16_SNORM' '-auto' '-fbo' # 147.525ms, total 28030.8ms
'bin/depthstencil-render-miplevels' '585' 'd=z24_s=z24_s8' '-auto' # 159.331ms, total 28190.2ms
'bin/ext_framebuffer_multisample-turn-on-off' '8' '-auto' # 174.701ms, total 28364.9ms
'bin/copyteximage' '2D_ARRAY' '-samples=4' '-auto' # 218.974ms, total 28583.8ms
'bin/link-unresolved-function' '-auto' '-fbo' # 48.695ms, total 28632.5ms
'bin/gl-3.0-texparameteri' '-auto' '-fbo' # 44.085ms, total 28676.6ms
'bin/generatemipmap-cubemap' '-auto' '-fbo' # 59.226ms, total 28735.9ms
'bin/teximage-colors' 'GL_RG' '-auto' '-fbo' # 165.208ms, total 28901.1ms
'bin/varying-packing-simple' 'vec2' 'arrays_of_arrays' '-auto' '-fbo' # 384.974ms, total 29286.0ms
'bin/depthstencil-render-miplevels' '585' 'd=z32f' '-auto' # 123.815ms, total 29409.9ms
'bin/gl-3.2-layered-rendering-framebuffertexture' '-auto' '-fbo' # 70.107ms, total 29480.0ms
'bin/gl-3.2-layered-rendering-clear-color-all-types' '2d_array' 'mipmapped' '-auto' '-fbo' # 59.439ms, total 29539.4ms
'bin/built-in-constants_gles2' '/home/ickle/piglit/tests/spec/glsl-es-1.00/minimum-maximums.txt' '-auto' '-fbo' # 90.612ms, total 29630.0ms
'bin/gl-3.0-forward-compatible-bit' 'no' '-auto' '-fbo' # 48.805ms, total 29678.8ms
'bin/copyteximage' 'RECT' '-samples=6' '-auto' # 160.115ms, total 29838.9ms
'bin/gl-3.1-primitive-restart-xfb' 'written' '-auto' '-fbo' # 57.620ms, total 29896.5ms
'bin/vao-01' '-auto' # 47.613ms, total 29944.2ms
'bin/gl-3.2-layered-rendering-clear-color-all-types' '3d' 'mipmapped' '-auto' '-fbo' # 51.769ms, total 29995.9ms

-Chris