[ghns] Re : about SoC proposal at Blender Foundation on GHNS + questions
Jonathan-David SCHRODER
myselfhimself at free.fr
Wed Apr 9 23:00:34 PDT 2008
Hi
sorry for spamming
I've discovered there's an xmlrpclib inside Python's standard library :
http://docs.python.org/modindex.html =>
http://docs.python.org/lib/module-xmlrpclib.html
well anyway, I'd be glad to use the REST API, unless the SOAP API has many
more features.
I have a made a wiki site up for the project here :
http://www.ece.fr/~schroder/SoC2008/doku.php?id=blenderproject<http://www.ece.fr/%7Eschroder/SoC2008/doku.php?id=blenderproject>
Sincerely,
Jonathan
On Thu, Apr 10, 2008 at 1:20 AM, Jonathan-David SCHRODER <
myselfhimself at free.fr> wrote:
> Hi !
>
> Do you think that there could be more than 1 category level ?
> The REST api seems to show that there's only one level, and so does
> the feed view with all the xml files.
> If you want I could try and digg into the code myself and see how I
> could implement that with you... Or maybe you'll hophop my friend sit
> back, you've got other work to do.
>
> This is in some way, customizing work that I'd be supposed to do as
> part of my SoC. So please tell me, and I shall study and work on the
> code on SVN (where is the repository actually ?)
> I'd need more than 1 level of category (that is to say 2 for now : )
> but this number may change).
> If I could customize something else, that would be the possibility to
> put search terms/regexps in the REST api.
> I should see if I need to add search criteria.
>
> I think I'll setup some wiki page somewhere on my personal web site
> domain first with what I'd need for the web site.
>
> Sincerely,
> Jonathan Schröder
>
>
>
> On Thu, Apr 10, 2008 at 12:31 AM, Jonathan-David SCHRODER
> <myselfhimself at free.fr> wrote:
> > Hi
> >
> > I thank you for your clear and detailed reply.
> > I hope that you do no mind if I think aloud.
> > In this mail, I think over the 3 protocol option you Frank said (within
the"
> > == ... of my reflexion ===" boundaries) and then I pose some questions
to
> > you and tell which api option I would prefer.
> >
> > === start of my reflexion ===
> > as to option 1. with autogenerated feeds located at php page.
> > you say that is doesn't support searching, and a workaround for this
would
> > be to create lots of ids. For example, I see on
> > http://www.kde-look.org/help/ghns.php the following variation of themes
:
> > soundthemes,mousethemes,kdmthemes,amarokthemes,plasmathemes,iconthemes.
> > What I would consider doing if ever I used this 1st option would be to
have
> > lots of id names for example for materials :
> >
materialswood,materialsmetal,materialsground,materialsglass,materialsnature...
> > one problem is we'd need a materials feed that would lead to this list
of
> > materials.
> > Because I do not want people in Blender to see all the different
materials
> > types from the beginning. User will want to download more kinds of data
> > block than just materials : models, scripts, compositing nodes, logic
> > bricks, armature setups, textures.
> > I think that in the end, we would need a lot of Ids which could make
the
> > feed page too long to download when running Blender's assets manager.
Unless
> > we cache that page locally and refresh once a day on the Blender's plug
in
> > launch.
> > There would not be a big problem from the web interface point of view
to
> > have as many ideas, as what kde-looks.org shows is that the web
interface
> > categories map to the different ids or vice-versa.
> >
> > (+) on the client side :
> > - direct access to what to download with little xml browsing required
> > - script is easy to implement
> > - no authentication needed
> > - load distributed over many servers around the globe
> >
> > (-) on the client side :
> > - searching has to be done by the client script
> > - if users submit data, they'll have to way < 1 hour to see it up
online
> > through the client script
> > - there needs do exist some file online or offline that tells which
category
> > has which sub-categories=id names. This is awkward to have all the
> > subcategories on the same root in the feeds list though that's not a
> > problem.
> > - there's no nb of items per page limit compared to the REST api (10
items
> > max / page and possibility to request which page to display by its
number)
> > so this makes the client potentially download a lot (ie. imagine there
a
> > lots of glass materials). Getting the N first items of a list would be
ok
> > (ie : close the http xml download right from the moment when know we
have
> > downloaded N elements), but as to the next N elements one would have to
use
> > http resume support.
> >
> > as to option 2.
> > (+) on the client side :
> > - can get content/info without having for the script to browse through
xml
> > hyperlinks, just make SOAP calls.
> > - if we use a SOAP library things are very easy
> > I've found something in the dive into python book with wsdl
> > http://www.diveintopython.org/soap_web_services/google.html which makes
> > method calling extremely easy if a wsdl file exists, one problem is
that
> > SOAPpy used there isn't in
> > - one button click really corresponds to one web SOAP call.
> > - voting and searching implemented.
> > - sorting is not done by the client.
> >
> > (-) on the client side :
> > - authentication needed
> > - as to the SOAP capabilities on Python's side : Python's standard
> > distribution (and there's no soap library at all in the python's
standard
> > distrib I think)... so I'd have to include the whole SOAPpy package
inside
> > Blender releases which would push them to be "unofficial" because I
doubt
> > that such an inclusion would be accepted. Or I'd implement the SOAPing
with
> > xml.minidom and libhttp myself. Maybe, the blender script could, not
contain
> > this package, and on the first run oblige the user to click on a
"download
> > SOAPpy package now" which would make the script download and install
the
> > package inside Blender's .blender/scripts/ directory.
> > I should talk about that to the Blender developpers maybe. I think from
at
> > first thought that : it preferable not to increase the size of Blender
> > builds too much; it would be nice that the script (some scripts are
packaged
> > with official Blender builds but they rarely - I think - depend on
external
> > libraries) could work right way one run, not requiring to download a
soap
> > library.
> > - broadband internet connection is required
> >
> > as to option 3.
> > (+)
> > - easy to implement. Coding the client could be very evident.
> > - no real need for libs not present in a standard python distribution.
> > - one click equals one internet call
> > - not really a need for a higher speed connection compared to SOAP.
> > - sorting is not done by the client
> > - voting and search implement.
> >
> > (-)
> > - authentication required
> >
> > === end of my reflexions ==
> >
> > The requirement to authenticate for using a SOAP/REST interface is not
a
> > burdain to users to me. What makes me a little reluctant to it though
is
> > that login + password are sent in clear. Instead, why not send the user
a
> > generated image with "garbled"/morphed text and ask him to type the
> > corresponding text ? Doing that kind of translation is maybe something
that
> > bots can't do and this could spare people from signing up.
> > I think that having to sign up for uploading contents would be a good
> > thing, so that what's posted is related to some user with with all
his/her
> > details at hand for visitors.
> >
> >
> > "If someone sends 100 search requests a second my server is down." do
you
> > mean for just one person sending 100 requests ? Or do you mean, that if
> > several people start flooding with searches at the same time, this will
you
> > bring your server down ?
> >
> > How do you submit content ? on the ghns feed I can't find any upload
url
> > (just a nouploadurl attribute...), and for the REST api contents I can
fetch
> > I see no upload url attribute at all. I don't know as to the SOAP
interface.
> > Would that be compulsory to use the web interface form to upload
things :
> > http://www.kde-look.org/content/add.php ?
> >
> > I'm interested in the REST API more than in the SOAP one because of the
> > external libraries requirement issue. So if I were to tell you to set
> > something up I would request the platform to have that first API.
> > However, please do not setup something for blender yet. I've just
written
> > to the bf-committers (bf=blender foundation) list. I'll let you know
about
> > the outcome of the replies.
> > The mail I posted is here :
> > http://lists.blender.org/pipermail/bf-committers/2008-April/020741.html
> >
> > Sincerely,
> > Jonathan-david Schroder
> >
> >
> >
> >
> > On Wed, Apr 9, 2008 at 4:59 PM, Frank Karlitschek <karlitschek at kde.org>
> > wrote:
> > > Hi Jonathan,
> > >
> > > sorry for the confusion.
> > > We are talking about three different interfaces here.
> > >
> > > 1. GHNS v1.0
> > > I´m generating static xml files and distribute them, together with
the
> > download files over the kde mirror network. This is done every hour.
> > > The files are accessed from the KDE Application over the host
> > download.kde.org which is doing roundrobin via DNS.
> > >
> > > You find the feeds here:
> > >
> > > http://www.kde-look.org/help/ghns.php
> > >
> > > This is used by most of the KDE applications at the moment.
> > > The benefit of this solution is that the load is distributed over
lots of
> > high capacity servers all over the globe. Access to the content is
possible
> > without authentication because a deny of service attack is very
difficult.
> > >
> > > This could be used for blender too. But I´m not sure if we are
allowed to
> > use the kde mirrors for blender content.
> > >
> > > 2. GHNS v2.0
> > > The new Specification done my Josef. Also called DXS.
> > > It is using SOAP. It is quite powerful but also heavyweight on the
server.
> > > The only serverside implementation I know is the one from Josef.
> > > I also plan to support the interface in the future but it is not
> > implemented yet.
> > >
> > >
> > >
> > > 3. openDesktop.org REST
> > > The REST interface done by me. You find the details here:
> > > http://api.opendesktop.org
> > >
> > > The interface is more powerfull than the GHNS v1.0 It is possible to
> > browse all the content from the sites and supports voting. I will also
> > implement searching in the near future.
> > >
> > > I can setup a new community website for blender in less than a week
> > together with a REST api for you. I can also integrate this website
into the
> > openDesktop.org network so it is getting promotion and users fast.
> > >
> > > The problem is that this REST interface is vulnerable for deny of
service
> > attacks or script kiddies. If someone sends 100 search requests a
second my
> > server is down.
> > > Because of that access to this interface is only possible for
> > authenticated users or from dedicated IPs.
> > >
> > > Google, Yahoo, Amazon and eBay are doing the same because of the same
> > reason.
> > >
> > >
> > > So I´m not sure what to suggest.
> > >
> > > Interface 1 has the best performance. But it only supports
downloading and
> > no searching. And you need server infrastructure.
> > >
> > > Interface 2 hast the most features. But I´m not sure if Josefs
> > Implementation is ready and if there is a community system. It also has
the
> > deny of service problem.
> > >
> > > Interface 3 is only usable from authenticated users. I´m not sure if
this
> > makes sense to require for blender users.
> > >
> > >
> > > What do you think?
> > >
> > >
> > > Best Regards
> > > Frank
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On 09.04.2008, at 15:35, Jonathan-David SCHRODER wrote:
> > >
> > > >
> > > >
> > > >
> > > > Hi again,
> > > > please forgive me for my last message that was missing words here
and
> > there.
> > > > Thank you for you answer Josef.
> > > > Ok I'll try to show the Bf (blender foundation) that I'm already
> > tinkering with the scripts (this is what I've started doing, for real
)and
> > that I'm getting advice on this list.
> > > >
> > > > I have studied some xml pages fetched thanks to the api.
> > > > Things look good. On this I have question, how is that possible to
> > submit new files to the server through the api ? Submit doesn't belong
the 3
> > functions list from api.kde-look.org .
> > > > Also, what's the difference between the data fetched thanks to the
api
> > from : api.kde-look.org , and the data fetchable by browsing through
xml
> > pages, starting from the ones mentionned here :
> > http://www.kde-look.org/help/ghns.php ?
> > > > I don't understand much how to find again the same categories
listed on
> > http://api.opendesktop.org/V1/CATEGORIES, and on
> > http://www.kde-look.org/help/ghns.php .
> > > >
> > > > I've had a look at :
> >
http://svn.ggzgamingzone.org/trac.cgi/browser/trunk/python/lib/sdlnewstuff.py?rev=head
> > . It uses the http://www.kde-look.org/help/ghns.php interface with the
> > providers.xml files. To me, this interface could also be called a REST
> > interface somehow, because there's no SOAP, and it uses the download
xml
> > -then- go to one of the hyperlinks steps which resemble what I've read
about
> > REST.
> > > > I haven't been able to run the python script because my pygame
install
> > on linux is weird (doing apt-get install pygame does install pygame but
does
> > not setup the pythonpath to add pygame's folfers... looking on the web,
this
> > seems to be an old deb. package bug while I'm using a much newer ubuntu
> > distribution...). Changing my pyhonpath the way I could guess does make
> > importing pygame work partly, but I still have problems finding some
parts
> > of the pygame library... Maybe I'll purge the sdlnewstuff script from
any
> > pygame related code and try to run it, though reading the code is
enough to
> > understand what this script does with the ghns interface.
> > > >
> > > > I'd be ready to rewrite some python script using the REST
interface,
> > maybe complementing what I could see with sdlnewstuff and making a
python
> > only library to be used by any kind of software.
> > > >
> > > > Sincerely,
> > > > Jonathan-david Schröder
> > > >
> > > > On Wed, Apr 9, 2008 at 11:38 AM, Josef Spillner <spillner at kde.org>
> > wrote:
> > > > Am Dienstag 08 April 2008 03:14:57 schrieb Jonathan-David SCHRODER:
> > > > > I've "met" on #blendercoders a blender developper interested in
my
> > proposal
> > > > > (he had access to the SoC admin side for the blender foundation)
> > because he
> > > > > worked using mysql for blender files data blocks.
> > > > > He wants to talk more about the proposal with me in a few days.
> > > >
> > > > You can tell him that you could easily get advice on GHNS-specifics
on
> > this
> > > > list even if someone from Blender becomes a mentor.
> > > >
> > > > > By then... I could have tried to use khotnewstuff, and maybe some
> > shell
> > > > > scripts I've seen on the repos to know that I can make the GHNS
system
> > work
> > > > > somehow and start discovering more about it.
> > > >
> > > > It will give a positive view if you already tinker with the scripts
and
> > > > servers prior to the application rating deadline.
> > > >
> > > > Josef
> > > >
> > > > P.S. You're always on CC during the whole mail thread. Are you
> > subscribed to
> > > > it now? We could drop the CC then.
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > http://www.jaxtr.com/myselfhimself
> > _______________________________________________
> > > > ghns mailing list
> > > > ghns at lists.freedesktop.org
> > > > http://lists.freedesktop.org/mailman/listinfo/ghns
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Frank Karlitschek
> > > karlitschek at kde.org
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > --
> > http://www.jaxtr.com/myselfhimself
>
>
>
> --
> http://www.jaxtr.com/myselfhimself
>
--
http://www.jaxtr.com/myselfhimself
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freedesktop.org/archives/ghns/attachments/20080410/932ffa8c/attachment-0001.html
More information about the ghns
mailing list