[Mesa-dev] [RFC shader-db] Add support for shadertoy tests

[Dylan Baker]

I have a couple of python pointers for you, feel free to take them or
leave them.

Dylan

On Tue, Jun 16, 2015 at 03:46:50PM -0400, Rob Clark wrote:
> Attached script grabs shaders from shadertoy, and dumps them out as
> .shader_test files which can be run through shader-db for compiler
> testing.
> 
> shadertoy only gives you a fragment shader (which works based on
> gl_FragCoord), so a generic vertex shader is used.  And a blurb is
> inserted for the pre-defined uniforms and main() function (which just
> calls shadertoy mainImage() fxn).
> 
> ---
> TODO I guess we'd actually have to parse the shader to figure out if
> the sampler uniforms were meant to be 2D/cube/etc.  Maybe we just
> commit samplers we get from the script and massage them by hand?
> 
> PS. don't make fun of my py too much.. I'm a newb and figuring it
> out as I go

I'm trying not to make fun, but I do have quite a few pointers for you.

> 
>  grab-shadertoy.py | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 63 insertions(+)
>  create mode 100755 grab-shadertoy.py
> 
> diff --git a/grab-shadertoy.py b/grab-shadertoy.py
> new file mode 100755
> index 0000000..74e9d10
> --- /dev/null
> +++ b/grab-shadertoy.py
> @@ -0,0 +1,63 @@
> +#!/usr/bin/env python3
> +
> +
> +import requests, json

You're not actually using json

> +
> +url = 'https://www.shadertoy.com/api/v1/shaders'
> +key = '?key=NdnKw7'
> +
> +# Get the list of shaders
> +r = requests.get(url + key)
> +j = r.json()
> +print('Found ' + str(j['Shaders']) + ' shaders')

If you use format you can avoid calling str() on everything, and make
things more readable using format rather than concatenation:
print('Found {} shaders'.format(j['Shaders']))

> +
> +shader_ids = j['Results']
> +for id in shader_ids:
> +    print('Fetching shader: ' + str(id))
> +    r = requests.get(url + '/' + id + key)
> +    j = r.json()
> +    s = j['Shader']
> +    info = s['info']
> +    print('Name: ' + info['name'])
> +    print('Description: ' + info['description'])
> +    i = 0;

python has a cool builtin called enumerate for doing this:
for i, p in enmerate(s['renderpass']):

Also, I know it's easy to forget, but python doesn't use ';' at the end
of lines, it allows them, but they look weird to pythonistas

> +    for p in s['renderpass']:
> +        fobj = open('shaders/shadertoy/' + str(id) + '_' + str(i) + '.shader_test', 'w')

with str.format this would look like:
with open('shaders/shadertoy/{}_{}.shader_test'.format(id, i), 'w') as fobj:

> +        #print('Inputs: ' + str(p['inputs']))
> +        #print('Outputs: ' + str(p['outputs']))
> +        fobj.write('[require]\n')
> +        fobj.write('GLSL >= 1.30\n')
> +        fobj.write('\n');
> +        fobj.write('[fragment shader]\n')
> +        fobj.write('#version 130\n')
> +        # Shadertoy inserts some uniforms, so we need to do the same:
> +        fobj.write('uniform vec3      iResolution;\n');
> +        fobj.write('uniform float     iGlobalTime;\n');
> +        fobj.write('uniform float     iChannelTime[4];\n');
> +        fobj.write('uniform vec4      iMouse;\n');
> +        fobj.write('uniform vec4      iDate;\n');
> +        fobj.write('uniform float     iSampleRate;\n');
> +        fobj.write('uniform vec3      iChannelResolution[4];\n');
> +        # TODO probably need to parse the shader to figure out if 2d/cubemap/etc
> +        fobj.write('uniform sampler2D iChannel0;\n');
> +        fobj.write('uniform sampler2D iChannel1;\n');
> +        fobj.write('uniform sampler2D iChannel2;\n');
> +        fobj.write('uniform sampler2D iChannel3;\n');
> +        # Actual shadertoy shader body:
> +        fobj.write(p['code'])
> +        # Shadertoy shader uses mainImage(out vec4 fragColor, in vec2 fragCoord)
> +        # so we need to insert a main:
> +        fobj.write('\nvoid main() { mainImage(gl_FragColor, gl_FragCoord.xy); }\n')
> +        fobj.write('\n\n')
> +        # And a generic vertex shader:
> +        fobj.write('[vertex shader]\n')
> +        fobj.write('#version 130\n')
> +        fobj.write('in vec2 position;\n')
> +        fobj.write('\n')
> +        fobj.write('void main()\n')
> +        fobj.write('{\n')
> +        fobj.write('   gl_Position = vec4(position, 0.0, 1.0);\n')
> +        fobj.write('}\n')
> +
> +        fobj.close()
> +        i = 1 + i

You can simplify this quite a bit:

import textwrap

header = textwrap.dedent("""\
    [require]
    GLSL >= 1.30

    [fragment shader]
    #version 130
    uniform vec3      iResolution;
    uniform float     iGlobalTime;
    uniform float     iChannelTime[4];
    uniform vec4      iMouse;
    uniform vec4      iDate;
    uniform float     iSampleRate;
    uniform vec3      iChannelResolution[4];
    uniform sampler2D iChannel0;
    uniform sampler2D iChannel1;
    uniform sampler2D iChannel2;
    uniform sampler2D iChannel3;
""")

footer = textwarp.dedent("""\
    void main() { mainImage(gl_FragColor, gl_FragCoord.xy); }

    [vertex shader]
    #version 130
    in vec2 position;

    void main()
    {
       gl_Position = vec4(position, 0.0, 1.0);
    }
""")

for id in shader_id:
    ...
    
    for i, p in enumerate(s['renderpass']):
        with open('shaders/shadertoy/{}_{}.shader_test'.format(id, i), 'w') as f:
            f.write(header)
            f.write(p['code'])
            f.write(footer)

Also, I recommend using `with open(...)` since it automatically closes
the fd at the end of the block.

You should probably also ensure that shaders/shadertoy exists, I don't
think file.write will succeed if it doesn't.
Something like this should do:

import os
if not os.path.exists('shaders/shadertoy'):
    os.makedirs('shaders/shadertoy')

> -- 
> 2.4.2
> 
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: Digital signature
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20150623/2f37a2f8/attachment-0001.sig>