[Mesa-dev] [RFC shader-db] Add support for shadertoy tests
Tom Stellard
tom at stellard.net
Tue Jun 23 18:47:01 PDT 2015
On Tue, Jun 23, 2015 at 08:18:45PM -0400, Rob Clark wrote:
> On Tue, Jun 23, 2015 at 7:27 PM, Dylan Baker <baker.dylan.c at gmail.com> wrote:
> > I have a couple of python pointers for you, feel free to take them or
> > leave them.
>
> cool, thanks..
>
> What do others think about including shadertoy in shader-db? If it is
> a useful thing, I'll clean up my script and re-submit..
>
> And if we include it, should we commit the scripts we pull down for
> repeatable results and just keep the script as something to resync and
> pull in new shaders? (Esp. given that a small percentage need some
> hand massaging.. I haven't figured out a good way to
> reliably/programatically figure out if the samplers should actually be
> 2d/3d/cube..)
>
I'm in favor of including shadertoy shaders.
-Tom
> That said, I think the shaderdb shaders are a fairly, umm, unique
> stress test of a shader compiler.. and the API to scrape shaders seems
> to convenient to ignore..
>
> BR,
> -R
>
> > Dylan
> >
> > On Tue, Jun 16, 2015 at 03:46:50PM -0400, Rob Clark wrote:
> >> Attached script grabs shaders from shadertoy, and dumps them out as
> >> .shader_test files which can be run through shader-db for compiler
> >> testing.
> >>
> >> shadertoy only gives you a fragment shader (which works based on
> >> gl_FragCoord), so a generic vertex shader is used. And a blurb is
> >> inserted for the pre-defined uniforms and main() function (which just
> >> calls shadertoy mainImage() fxn).
> >>
> >> ---
> >> TODO I guess we'd actually have to parse the shader to figure out if
> >> the sampler uniforms were meant to be 2D/cube/etc. Maybe we just
> >> commit samplers we get from the script and massage them by hand?
> >>
> >> PS. don't make fun of my py too much.. I'm a newb and figuring it
> >> out as I go
> >
> > I'm trying not to make fun, but I do have quite a few pointers for you.
> >
> >>
> >> grab-shadertoy.py | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> 1 file changed, 63 insertions(+)
> >> create mode 100755 grab-shadertoy.py
> >>
> >> diff --git a/grab-shadertoy.py b/grab-shadertoy.py
> >> new file mode 100755
> >> index 0000000..74e9d10
> >> --- /dev/null
> >> +++ b/grab-shadertoy.py
> >> @@ -0,0 +1,63 @@
> >> +#!/usr/bin/env python3
> >> +
> >> +
> >> +import requests, json
> >
> > You're not actually using json
> >
> >> +
> >> +url = 'https://www.shadertoy.com/api/v1/shaders'
> >> +key = '?key=NdnKw7'
> >> +
> >> +# Get the list of shaders
> >> +r = requests.get(url + key)
> >> +j = r.json()
> >> +print('Found ' + str(j['Shaders']) + ' shaders')
> >
> > If you use format you can avoid calling str() on everything, and make
> > things more readable using format rather than concatenation:
> > print('Found {} shaders'.format(j['Shaders']))
> >
> >> +
> >> +shader_ids = j['Results']
> >> +for id in shader_ids:
> >> + print('Fetching shader: ' + str(id))
> >> + r = requests.get(url + '/' + id + key)
> >> + j = r.json()
> >> + s = j['Shader']
> >> + info = s['info']
> >> + print('Name: ' + info['name'])
> >> + print('Description: ' + info['description'])
> >> + i = 0;
> >
> > python has a cool builtin called enumerate for doing this:
> > for i, p in enmerate(s['renderpass']):
> >
> > Also, I know it's easy to forget, but python doesn't use ';' at the end
> > of lines, it allows them, but they look weird to pythonistas
> >
> >> + for p in s['renderpass']:
> >> + fobj = open('shaders/shadertoy/' + str(id) + '_' + str(i) + '.shader_test', 'w')
> >
> > with str.format this would look like:
> > with open('shaders/shadertoy/{}_{}.shader_test'.format(id, i), 'w') as fobj:
> >
> >> + #print('Inputs: ' + str(p['inputs']))
> >> + #print('Outputs: ' + str(p['outputs']))
> >> + fobj.write('[require]\n')
> >> + fobj.write('GLSL >= 1.30\n')
> >> + fobj.write('\n');
> >> + fobj.write('[fragment shader]\n')
> >> + fobj.write('#version 130\n')
> >> + # Shadertoy inserts some uniforms, so we need to do the same:
> >> + fobj.write('uniform vec3 iResolution;\n');
> >> + fobj.write('uniform float iGlobalTime;\n');
> >> + fobj.write('uniform float iChannelTime[4];\n');
> >> + fobj.write('uniform vec4 iMouse;\n');
> >> + fobj.write('uniform vec4 iDate;\n');
> >> + fobj.write('uniform float iSampleRate;\n');
> >> + fobj.write('uniform vec3 iChannelResolution[4];\n');
> >> + # TODO probably need to parse the shader to figure out if 2d/cubemap/etc
> >> + fobj.write('uniform sampler2D iChannel0;\n');
> >> + fobj.write('uniform sampler2D iChannel1;\n');
> >> + fobj.write('uniform sampler2D iChannel2;\n');
> >> + fobj.write('uniform sampler2D iChannel3;\n');
> >> + # Actual shadertoy shader body:
> >> + fobj.write(p['code'])
> >> + # Shadertoy shader uses mainImage(out vec4 fragColor, in vec2 fragCoord)
> >> + # so we need to insert a main:
> >> + fobj.write('\nvoid main() { mainImage(gl_FragColor, gl_FragCoord.xy); }\n')
> >> + fobj.write('\n\n')
> >> + # And a generic vertex shader:
> >> + fobj.write('[vertex shader]\n')
> >> + fobj.write('#version 130\n')
> >> + fobj.write('in vec2 position;\n')
> >> + fobj.write('\n')
> >> + fobj.write('void main()\n')
> >> + fobj.write('{\n')
> >> + fobj.write(' gl_Position = vec4(position, 0.0, 1.0);\n')
> >> + fobj.write('}\n')
> >> +
> >> + fobj.close()
> >> + i = 1 + i
> >
> > You can simplify this quite a bit:
> >
> > import textwrap
> >
> > header = textwrap.dedent("""\
> > [require]
> > GLSL >= 1.30
> >
> > [fragment shader]
> > #version 130
> > uniform vec3 iResolution;
> > uniform float iGlobalTime;
> > uniform float iChannelTime[4];
> > uniform vec4 iMouse;
> > uniform vec4 iDate;
> > uniform float iSampleRate;
> > uniform vec3 iChannelResolution[4];
> > uniform sampler2D iChannel0;
> > uniform sampler2D iChannel1;
> > uniform sampler2D iChannel2;
> > uniform sampler2D iChannel3;
> > """)
> >
> > footer = textwarp.dedent("""\
> > void main() { mainImage(gl_FragColor, gl_FragCoord.xy); }
> >
> > [vertex shader]
> > #version 130
> > in vec2 position;
> >
> > void main()
> > {
> > gl_Position = vec4(position, 0.0, 1.0);
> > }
> > """)
> >
> > for id in shader_id:
> > ...
> >
> > for i, p in enumerate(s['renderpass']):
> > with open('shaders/shadertoy/{}_{}.shader_test'.format(id, i), 'w') as f:
> > f.write(header)
> > f.write(p['code'])
> > f.write(footer)
> >
> > Also, I recommend using `with open(...)` since it automatically closes
> > the fd at the end of the block.
> >
> > You should probably also ensure that shaders/shadertoy exists, I don't
> > think file.write will succeed if it doesn't.
> > Something like this should do:
> >
> > import os
> > if not os.path.exists('shaders/shadertoy'):
> > os.makedirs('shaders/shadertoy')
> >
> >> --
> >> 2.4.2
> >>
> >> _______________________________________________
> >> mesa-dev mailing list
> >> mesa-dev at lists.freedesktop.org
> >> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
More information about the mesa-dev
mailing list