[Mesa-dev] [RFC shader-db] Add support for shadertoy tests
Rob Clark
robdclark at gmail.com
Tue Jun 23 17:18:45 PDT 2015
On Tue, Jun 23, 2015 at 7:27 PM, Dylan Baker <baker.dylan.c at gmail.com> wrote:
> I have a couple of python pointers for you, feel free to take them or
> leave them.
cool, thanks..
What do others think about including shadertoy in shader-db? If it is
a useful thing, I'll clean up my script and re-submit..
And if we include it, should we commit the scripts we pull down for
repeatable results and just keep the script as something to resync and
pull in new shaders? (Esp. given that a small percentage need some
hand massaging.. I haven't figured out a good way to
reliably/programatically figure out if the samplers should actually be
2d/3d/cube..)
That said, I think the shaderdb shaders are a fairly, umm, unique
stress test of a shader compiler.. and the API to scrape shaders seems
to convenient to ignore..
BR,
-R
> Dylan
>
> On Tue, Jun 16, 2015 at 03:46:50PM -0400, Rob Clark wrote:
>> Attached script grabs shaders from shadertoy, and dumps them out as
>> .shader_test files which can be run through shader-db for compiler
>> testing.
>>
>> shadertoy only gives you a fragment shader (which works based on
>> gl_FragCoord), so a generic vertex shader is used. And a blurb is
>> inserted for the pre-defined uniforms and main() function (which just
>> calls shadertoy mainImage() fxn).
>>
>> ---
>> TODO I guess we'd actually have to parse the shader to figure out if
>> the sampler uniforms were meant to be 2D/cube/etc. Maybe we just
>> commit samplers we get from the script and massage them by hand?
>>
>> PS. don't make fun of my py too much.. I'm a newb and figuring it
>> out as I go
>
> I'm trying not to make fun, but I do have quite a few pointers for you.
>
>>
>> grab-shadertoy.py | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 63 insertions(+)
>> create mode 100755 grab-shadertoy.py
>>
>> diff --git a/grab-shadertoy.py b/grab-shadertoy.py
>> new file mode 100755
>> index 0000000..74e9d10
>> --- /dev/null
>> +++ b/grab-shadertoy.py
>> @@ -0,0 +1,63 @@
>> +#!/usr/bin/env python3
>> +
>> +
>> +import requests, json
>
> You're not actually using json
>
>> +
>> +url = 'https://www.shadertoy.com/api/v1/shaders'
>> +key = '?key=NdnKw7'
>> +
>> +# Get the list of shaders
>> +r = requests.get(url + key)
>> +j = r.json()
>> +print('Found ' + str(j['Shaders']) + ' shaders')
>
> If you use format you can avoid calling str() on everything, and make
> things more readable using format rather than concatenation:
> print('Found {} shaders'.format(j['Shaders']))
>
>> +
>> +shader_ids = j['Results']
>> +for id in shader_ids:
>> + print('Fetching shader: ' + str(id))
>> + r = requests.get(url + '/' + id + key)
>> + j = r.json()
>> + s = j['Shader']
>> + info = s['info']
>> + print('Name: ' + info['name'])
>> + print('Description: ' + info['description'])
>> + i = 0;
>
> python has a cool builtin called enumerate for doing this:
> for i, p in enmerate(s['renderpass']):
>
> Also, I know it's easy to forget, but python doesn't use ';' at the end
> of lines, it allows them, but they look weird to pythonistas
>
>> + for p in s['renderpass']:
>> + fobj = open('shaders/shadertoy/' + str(id) + '_' + str(i) + '.shader_test', 'w')
>
> with str.format this would look like:
> with open('shaders/shadertoy/{}_{}.shader_test'.format(id, i), 'w') as fobj:
>
>> + #print('Inputs: ' + str(p['inputs']))
>> + #print('Outputs: ' + str(p['outputs']))
>> + fobj.write('[require]\n')
>> + fobj.write('GLSL >= 1.30\n')
>> + fobj.write('\n');
>> + fobj.write('[fragment shader]\n')
>> + fobj.write('#version 130\n')
>> + # Shadertoy inserts some uniforms, so we need to do the same:
>> + fobj.write('uniform vec3 iResolution;\n');
>> + fobj.write('uniform float iGlobalTime;\n');
>> + fobj.write('uniform float iChannelTime[4];\n');
>> + fobj.write('uniform vec4 iMouse;\n');
>> + fobj.write('uniform vec4 iDate;\n');
>> + fobj.write('uniform float iSampleRate;\n');
>> + fobj.write('uniform vec3 iChannelResolution[4];\n');
>> + # TODO probably need to parse the shader to figure out if 2d/cubemap/etc
>> + fobj.write('uniform sampler2D iChannel0;\n');
>> + fobj.write('uniform sampler2D iChannel1;\n');
>> + fobj.write('uniform sampler2D iChannel2;\n');
>> + fobj.write('uniform sampler2D iChannel3;\n');
>> + # Actual shadertoy shader body:
>> + fobj.write(p['code'])
>> + # Shadertoy shader uses mainImage(out vec4 fragColor, in vec2 fragCoord)
>> + # so we need to insert a main:
>> + fobj.write('\nvoid main() { mainImage(gl_FragColor, gl_FragCoord.xy); }\n')
>> + fobj.write('\n\n')
>> + # And a generic vertex shader:
>> + fobj.write('[vertex shader]\n')
>> + fobj.write('#version 130\n')
>> + fobj.write('in vec2 position;\n')
>> + fobj.write('\n')
>> + fobj.write('void main()\n')
>> + fobj.write('{\n')
>> + fobj.write(' gl_Position = vec4(position, 0.0, 1.0);\n')
>> + fobj.write('}\n')
>> +
>> + fobj.close()
>> + i = 1 + i
>
> You can simplify this quite a bit:
>
> import textwrap
>
> header = textwrap.dedent("""\
> [require]
> GLSL >= 1.30
>
> [fragment shader]
> #version 130
> uniform vec3 iResolution;
> uniform float iGlobalTime;
> uniform float iChannelTime[4];
> uniform vec4 iMouse;
> uniform vec4 iDate;
> uniform float iSampleRate;
> uniform vec3 iChannelResolution[4];
> uniform sampler2D iChannel0;
> uniform sampler2D iChannel1;
> uniform sampler2D iChannel2;
> uniform sampler2D iChannel3;
> """)
>
> footer = textwarp.dedent("""\
> void main() { mainImage(gl_FragColor, gl_FragCoord.xy); }
>
> [vertex shader]
> #version 130
> in vec2 position;
>
> void main()
> {
> gl_Position = vec4(position, 0.0, 1.0);
> }
> """)
>
> for id in shader_id:
> ...
>
> for i, p in enumerate(s['renderpass']):
> with open('shaders/shadertoy/{}_{}.shader_test'.format(id, i), 'w') as f:
> f.write(header)
> f.write(p['code'])
> f.write(footer)
>
> Also, I recommend using `with open(...)` since it automatically closes
> the fd at the end of the block.
>
> You should probably also ensure that shaders/shadertoy exists, I don't
> think file.write will succeed if it doesn't.
> Something like this should do:
>
> import os
> if not os.path.exists('shaders/shadertoy'):
> os.makedirs('shaders/shadertoy')
>
>> --
>> 2.4.2
>>
>> _______________________________________________
>> mesa-dev mailing list
>> mesa-dev at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
More information about the mesa-dev
mailing list