[Piglit] Piglit 'changes' page no longer showing all tests that are 'Not Run'
Tom Stellard
tom at stellard.net
Tue Jul 30 11:42:43 PDT 2013
Hi,
I recall this being discussed on the list a few weeks ago, but I can't
find the email now. There was a recent change to piglit which caused
tests that were 'Not Run' to be excluded from the 'changes' page.
This is a problem for tests with subtests, because if the parent test
crashes, the subtests are listed as 'Not Run' and excluded from the
'changes' page. This means that if you regress a test with subtests by
crashing it, it will likely go unnoticed.
I have attached two example files to demonstrate the problem,
just run:
./piglit-summary-html.py not-run-changes pass crash
Would it be possible to restore the old piglit behavior or at least come
up with another solution?
Thanks,
Tom
-------------- next part --------------
{
"options": {
"profile": "tests/all_cl.tests",
"filter": ["scalar-store-char-global"],
"exclude_filter": []
},
"name": "pass",
"lspci": "00:00.0 Host bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Complex\n00:01.0 VGA compatible controller: ATI Technologies Inc Device 9640\n00:01.1 Audio device: ATI Technologies Inc Device 1714\n00:04.0 PCI bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Port\n00:10.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:10.1 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:11.0 SATA controller: Advanced Micro Devices [AMD] Hudson SATA Controller [IDE mode] (rev 40)\n00:12.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:12.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:13.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:13.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:14.0 SMBus: Advanced Micro Devices [AMD] Hudson SMBus Controller (rev 13)\n00:14.1 IDE interface: Advanced Micro Devices [AMD] Hudson IDE Controller\n00:14.2 Audio device: Advanced Micro Devices [AMD] Hudson Azalia Controller (rev 01)\n00:14.3 ISA bridge: Advanced Micro Devices [AMD] Hudson LPC Bridge (rev 11)\n00:14.4 PCI bridge: Advanced Micro Devices [AMD] Hudson PCI Bridge (rev 40)\n00:14.5 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 0 (rev 43)\n00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 1\n00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 2\n00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 3\n00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 4\n00:18.5 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 6\n00:18.6 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 5\n00:18.7 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 7\n01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)\n02:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)\n",
"glxinfo": "Error: unable to open display \n",
"tests": {
"Program/Execute/scalar-store-char-global/positive": {
"info": "Returncode: 0\n\nErrors:\n\n\nOutput:\n## Test: (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n# Platform: Default\n# Device: AMD SUMO\n# OpenCL version: 1.1\n# OpenCL C version: 1.1\n# Build options: -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: positive\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: work item\nUsing kernel store_work_item\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n>> All of the tests PASSED\n# Result:\n",
"returncode": 0,
"subtest": {
"positive": "pass",
"negative": "pass",
"work item": "pass"
},
"command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl",
"result": "pass",
"time": 0.2675039768218994
},
"Program/Execute/scalar-store-char-global/negative": {
"info": "Returncode: 0\n\nErrors:\n\n\nOutput:\n## Test: (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n# Platform: Default\n# Device: AMD SUMO\n# OpenCL version: 1.1\n# OpenCL C version: 1.1\n# Build options: -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: positive\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: work item\nUsing kernel store_work_item\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n>> All of the tests PASSED\n# Result:\n",
"returncode": 0,
"subtest": {
"positive": "pass",
"negative": "pass",
"work item": "pass"
},
"command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl",
"result": "pass",
"time": 0.2675039768218994
},
"Program/Execute/scalar-store-char-global/work item": {
"info": "Returncode: 0\n\nErrors:\n\n\nOutput:\n## Test: (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n# Platform: Default\n# Device: AMD SUMO\n# OpenCL version: 1.1\n# OpenCL C version: 1.1\n# Build options: -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: positive\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n> Running kernel test: work item\nUsing kernel store_work_item\nSetting kernel arguments...\nRunning the kernel...\nValidating results...\n Argument 0: PASS\n>> All of the tests PASSED\n# Result:\n",
"returncode": 0,
"subtest": {
"positive": "pass",
"negative": "pass",
"work item": "pass"
},
"command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl",
"result": "pass",
"time": 0.2675039768218994
}
},
"time_elapsed": 0.2777831554412842
}
-------------- next part --------------
{
"options": {
"profile": "tests/all_cl.tests",
"filter": ["scalar-store-char-global"],
"exclude_filter": []
},
"name": "crash",
"lspci": "00:00.0 Host bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Complex\n00:01.0 VGA compatible controller: ATI Technologies Inc Device 9640\n00:01.1 Audio device: ATI Technologies Inc Device 1714\n00:04.0 PCI bridge: Advanced Micro Devices [AMD] Family 12h Processor Root Port\n00:10.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:10.1 USB Controller: Advanced Micro Devices [AMD] Hudson USB XHCI Controller (rev 03)\n00:11.0 SATA controller: Advanced Micro Devices [AMD] Hudson SATA Controller [IDE mode] (rev 40)\n00:12.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:12.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:13.0 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:13.2 USB Controller: Advanced Micro Devices [AMD] Hudson USB EHCI Controller (rev 11)\n00:14.0 SMBus: Advanced Micro Devices [AMD] Hudson SMBus Controller (rev 13)\n00:14.1 IDE interface: Advanced Micro Devices [AMD] Hudson IDE Controller\n00:14.2 Audio device: Advanced Micro Devices [AMD] Hudson Azalia Controller (rev 01)\n00:14.3 ISA bridge: Advanced Micro Devices [AMD] Hudson LPC Bridge (rev 11)\n00:14.4 PCI bridge: Advanced Micro Devices [AMD] Hudson PCI Bridge (rev 40)\n00:14.5 USB Controller: Advanced Micro Devices [AMD] Hudson USB OHCI Controller (rev 11)\n00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 0 (rev 43)\n00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 1\n00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 2\n00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 3\n00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 4\n00:18.5 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 6\n00:18.6 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 5\n00:18.7 Host bridge: Advanced Micro Devices [AMD] Family 12h/14h Processor Function 7\n01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)\n02:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)\n",
"glxinfo": "Error: unable to open display \n",
"tests": {
"Program/Execute/scalar-store-char-global": {
"info": "Returncode: 1\n\nErrors:\nLLVM ERROR: Cannot select: 0x2577530: ch = store 0x2577130:1, 0x2577130, 0x2577330, 0x2572f70<ST1[%arrayidx7], trunc to i8> [ORD=14] [ID=30]\n 0x2577130: i32,ch = load 0x2577030, 0x2575fb0, 0x2572f70<LD1[%arrayidx6], anyext from i8> [ORD=12] [ID=29]\n 0x2575fb0: i32 = add 0x2573970, 0x2575eb0 [ORD=11] [ID=10]\n 0x2573970: i32 = CONST_ADDRESS 0x2573570 [ID=7]\n 0x2573570: i32 = Constant<8232> [ID=5]\n 0x2575eb0: i32 = Constant<3> [ID=4]\n 0x2572f70: i32 = undef [ID=1]\n 0x2577330: i32 = DWORDADDR 0x2577230 [ORD=14] [ID=22]\n 0x2577230: i32 = srl 0x25761b0, 0x25758b0 [ORD=14] [ID=18]\n 0x25761b0: i32 = add 0x2573270, 0x2575eb0 [ORD=13] [ID=14]\n 0x2573270: i32 = CONST_ADDRESS 0x2573170 [ID=8]\n 0x2573170: i32 = Constant<8228> [ID=6]\n 0x2575eb0: i32 = Constant<3> [ID=4]\n 0x25758b0: i32 = Constant<2> [ID=3]\n 0x2572f70: i32 = undef [ID=1]\nIn function: store\n\n\nOutput:\n## Test: (/home/tstellar/piglit/tests/cl/program/program-tester.c) ##\n\n# Running on:\n# Platform: Default\n# Device: AMD SUMO\n# OpenCL version: 1.1\n# OpenCL C version: 1.1\n# Build options: -cl-std=CL1.1\nProgram has been built successfully\n> Running kernel test: negative\nUsing kernel store\nSetting kernel arguments...\nRunning the kernel...\n",
"errors": [
"0x2575fb0: i32 = add 0x2573970, 0x2575eb0 [ORD=11] [ID=10]",
"0x2573970: i32 = CONST_ADDRESS 0x2573570 [ID=7]",
"0x2573570: i32 = Constant<8232> [ID=5]",
"0x2575eb0: i32 = Constant<3> [ID=4]",
"0x2572f70: i32 = undef [ID=1]",
"0x2577330: i32 = DWORDADDR 0x2577230 [ORD=14] [ID=22]",
"0x2577230: i32 = srl 0x25761b0, 0x25758b0 [ORD=14] [ID=18]",
"0x25761b0: i32 = add 0x2573270, 0x2575eb0 [ORD=13] [ID=14]",
"0x2573270: i32 = CONST_ADDRESS 0x2573170 [ID=8]",
"0x2573170: i32 = Constant<8228> [ID=6]",
"0x2575eb0: i32 = Constant<3> [ID=4]",
"0x25758b0: i32 = Constant<2> [ID=3]",
"0x2572f70: i32 = undef [ID=1]",
"In function: store"
],
"note": "Returncode was 1",
"returncode": 1,
"command": "/home/tstellar/piglit/framework/../bin/cl-program-tester tests/cl/program/execute/scalar-store-char-global.cl",
"result": "fail",
"time": 0.24598097801208496,
"errors_ignored": [
"LLVM ERROR: Cannot select: 0x2577530: ch = store 0x2577130:1, 0x2577130, 0x2577330, 0x2572f70<ST1[%arrayidx7], trunc to i8> [ORD=14] [ID=30]",
"0x2577130: i32,ch = load 0x2577030, 0x2575fb0, 0x2572f70<LD1[%arrayidx6], anyext from i8> [ORD=12] [ID=29]"
]
}
},
"time_elapsed": 0.25454282760620117
}
More information about the Piglit
mailing list