[Piglit] Piglit 'changes' page no longer showing all tests that are 'Not Run'

Tom Stellard tom at stellard.net
Tue Jul 30 11:42:43 PDT 2013


Hi,

I recall this being discussed on the list a few weeks ago, but I can't
find the email now.  There was a recent change to piglit which caused
tests that were 'Not Run' to be excluded from the 'changes' page.

This is a problem for tests with subtests, because if the parent test
crashes, the subtests are listed as 'Not Run' and excluded from the
'changes' page.  This means that if you regress a test with subtests by
crashing it, it will likely go unnoticed.

I have attached two example files to demonstrate the problem,
just run:
./piglit-summary-html.py not-run-changes pass crash

Would it be possible to restore the old piglit behavior or at least come
up with another solution?

Thanks,
Tom
-------------- next part --------------
{
    "options": {
        "profile": "tests/all_cl.tests",
        "filter": ["scalar-store-char-global"],
"exclude_filter": []
},
    "name": "pass",
    "lspci": "00:00.0 Host bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Complex\n00:01.0 VGA compatible controller: ATI Technologies Inc Device 9640\n00:01.1 Audio device: ATI Technologies Inc Device 1714\n00:04.0 PCI bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Port\n00:10.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:10.1 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:11.0 SATA controller: Advanced Micro Devices [AMD] Hudson SATA Controller [IDE mode] (rev 40)\n00:12.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:12.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:13.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:13.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:14.0 SMBus: Advanced Micro Devices [AMD] Hudson SMBus Controller (rev 13)\n00:14.1 IDE interface: Advanced Micro Devices [AMD] Hudson IDE Controller\n00:14.2 Audio device: Advanced Micro Devices [AMD] Hudson Azalia Controller (rev 01)\n00:14.3 ISA bridge: Advanced Micro Devices [AMD] Hudson LPC Bridge (rev 11)\n00:14.4 PCI bridge: Advanced Micro Devices [AMD] Hudson PCI Bridge (rev 40)\n00:14.5 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 0 (rev 43)\n00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 1\n00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 2\n00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 3\n00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 4\n00:18.5 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 6\n00:18.6 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 5\n00:18.7 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 7\n01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)\n02:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)\n",
    "glxinfo": "Error: unable to open display \n",
    "tests": {
        "Program/Execute/scalar-store-char-global/positive": {
                "info": "Returncode: 0\n\nErrors:\n\n\nOutput:\n## Test:  (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n#   Platform: Default\n#   Device: AMD SUMO\n#   OpenCL version: 1.1\n#   OpenCL C version: 1.1\n#   Build options:  -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: positive\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: work item\nUsing kernel store_work_item\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n>> All of the tests PASSED\n# Result:\n", 
                "returncode": 0, 
                "subtest": {
                    "positive": "pass", 
                    "negative": "pass", 
                    "work item": "pass"
                }, 
                "command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl", 
                "result": "pass", 
                "time": 0.2675039768218994
            },
        "Program/Execute/scalar-store-char-global/negative": {
                "info": "Returncode: 0\n\nErrors:\n\n\nOutput:\n## Test:  (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n#   Platform: Default\n#   Device: AMD SUMO\n#   OpenCL version: 1.1\n#   OpenCL C version: 1.1\n#   Build options:  -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: positive\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: work item\nUsing kernel store_work_item\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n>> All of the tests PASSED\n# Result:\n", 
                "returncode": 0, 
                "subtest": {
                    "positive": "pass", 
                    "negative": "pass", 
                    "work item": "pass"
                }, 
                "command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl", 
                "result": "pass", 
                "time": 0.2675039768218994
            },
        "Program/Execute/scalar-store-char-global/work item": {
                "info": "Returncode: 0\n\nErrors:\n\n\nOutput:\n## Test:  (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n#   Platform: Default\n#   Device: AMD SUMO\n#   OpenCL version: 1.1\n#   OpenCL C version: 1.1\n#   Build options:  -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: positive\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: work item\nUsing kernel store_work_item\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n>> All of the tests PASSED\n# Result:\n", 
                "returncode": 0, 
                "subtest": {
                    "positive": "pass", 
                    "negative": "pass", 
                    "work item": "pass"
                }, 
                "command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl", 
                "result": "pass", 
                "time": 0.2675039768218994
            }
    },
    "time_elapsed": 0.2777831554412842
}
-------------- next part --------------
{
    "options": {
        "profile": "tests/all_cl.tests",
        "filter": ["scalar-store-char-global"],
"exclude_filter": []
},
    "name": "crash",
    "lspci": "00:00.0 Host bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Complex\n00:01.0 VGA compatible controller: ATI Technologies Inc Device 9640\n00:01.1 Audio device: ATI Technologies Inc Device 1714\n00:04.0 PCI bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Port\n00:10.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:10.1 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:11.0 SATA controller: Advanced Micro Devices [AMD] Hudson SATA Controller [IDE mode] (rev 40)\n00:12.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:12.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:13.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:13.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:14.0 SMBus: Advanced Micro Devices [AMD] Hudson SMBus Controller (rev 13)\n00:14.1 IDE interface: Advanced Micro Devices [AMD] Hudson IDE Controller\n00:14.2 Audio device: Advanced Micro Devices [AMD] Hudson Azalia Controller (rev 01)\n00:14.3 ISA bridge: Advanced Micro Devices [AMD] Hudson LPC Bridge (rev 11)\n00:14.4 PCI bridge: Advanced Micro Devices [AMD] Hudson PCI Bridge (rev 40)\n00:14.5 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 0 (rev 43)\n00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 1\n00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 2\n00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 3\n00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 4\n00:18.5 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 6\n00:18.6 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 5\n00:18.7 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 7\n01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)\n02:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)\n",
    "glxinfo": "Error: unable to open display \n",
    "tests": {
        "Program/Execute/scalar-store-char-global": {
                "info": "Returncode: 1\n\nErrors:\nLLVM ERROR: Cannot select: 0x2577530: ch = store 0x2577130:1, 0x2577130, 0x2577330, 0x2572f70<ST1[%arrayidx7], trunc to i8> [ORD=14] [ID=30]\n  0x2577130: i32,ch = load 0x2577030, 0x2575fb0, 0x2572f70<LD1[%arrayidx6], anyext from i8> [ORD=12] [ID=29]\n    0x2575fb0: i32 = add 0x2573970, 0x2575eb0 [ORD=11] [ID=10]\n      0x2573970: i32 = CONST_ADDRESS 0x2573570 [ID=7]\n        0x2573570: i32 = Constant<8232> [ID=5]\n      0x2575eb0: i32 = Constant<3> [ID=4]\n    0x2572f70: i32 = undef [ID=1]\n  0x2577330: i32 = DWORDADDR 0x2577230 [ORD=14] [ID=22]\n    0x2577230: i32 = srl 0x25761b0, 0x25758b0 [ORD=14] [ID=18]\n      0x25761b0: i32 = add 0x2573270, 0x2575eb0 [ORD=13] [ID=14]\n        0x2573270: i32 = CONST_ADDRESS 0x2573170 [ID=8]\n          0x2573170: i32 = Constant<8228> [ID=6]\n        0x2575eb0: i32 = Constant<3> [ID=4]\n      0x25758b0: i32 = Constant<2> [ID=3]\n  0x2572f70: i32 = undef [ID=1]\nIn function: store\n\n\nOutput:\n## Test:  (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n#   Platform: Default\n#   Device: AMD SUMO\n#   OpenCL version: 1.1\n#   OpenCL C version: 1.1\n#   Build options:  -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\n", 
                "errors": [
                    "0x2575fb0: i32 = add 0x2573970, 0x2575eb0 [ORD=11] [ID=10]", 
                    "0x2573970: i32 = CONST_ADDRESS 0x2573570 [ID=7]", 
                    "0x2573570: i32 = Constant<8232> [ID=5]", 
                    "0x2575eb0: i32 = Constant<3> [ID=4]", 
                    "0x2572f70: i32 = undef [ID=1]", 
                    "0x2577330: i32 = DWORDADDR 0x2577230 [ORD=14] [ID=22]", 
                    "0x2577230: i32 = srl 0x25761b0, 0x25758b0 [ORD=14] [ID=18]", 
                    "0x25761b0: i32 = add 0x2573270, 0x2575eb0 [ORD=13] [ID=14]", 
                    "0x2573270: i32 = CONST_ADDRESS 0x2573170 [ID=8]", 
                    "0x2573170: i32 = Constant<8228> [ID=6]", 
                    "0x2575eb0: i32 = Constant<3> [ID=4]", 
                    "0x25758b0: i32 = Constant<2> [ID=3]", 
                    "0x2572f70: i32 = undef [ID=1]", 
                    "In function: store"
                ], 
                "note": "Returncode was 1", 
                "returncode": 1, 
                "command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl", 
                "result": "fail", 
                "time": 0.24598097801208496, 
                "errors_ignored": [
                    "LLVM ERROR: Cannot select: 0x2577530: ch = store 0x2577130:1, 0x2577130, 0x2577330, 0x2572f70<ST1[%arrayidx7], trunc to i8> [ORD=14] [ID=30]", 
                    "0x2577130: i32,ch = load 0x2577030, 0x2575fb0, 0x2572f70<LD1[%arrayidx6], anyext from i8> [ORD=12] [ID=29]"
                ]
            }
    },
    "time_elapsed": 0.25454282760620117
}


More information about the Piglit mailing list