Editing Fetching scripts (section)

= Issues =

== Security ==

Running untrusted code is trouble. Some ideas

* MediaWiki allows us to protect pages so that only administrators can edit them. We could protect all approved scripts.
* Alternately, we could have a protected page that links to the approved scripts. 
* I lean towards using both of those options. Just requiring the page to be protected could cause trouble if there's an unrelated page that happens to both be protected and have a security leak due to the fact that it was never intended to be a fetchable script.
* Maybe a hybrid system where scripts have a development version and a release version? I don't want to make too much overhead, though. Current idea: setting secure=0 means you get the development version.
* We should print a warning each time a new script is fetched anyway
* Can fetched scripts persist across saved sessions? It's easy enough to save them in fetch_path.
* Plugins? This is probably more worth considering for a future version, but it would be nice to be able to load plugins as well. Since plugins are (now) installed permanently, we have to think carefully about the implications.

== Convenience options ==

The main benefit is to make things as convenient and easy as possible, especially for new users.

* Local cache. This would make reloading scripts with each new session easier and faster. You could then stick a bunch of "fetch" lines in your pymolrc. fetch_path.
* A command to list all available scripts?

== Validation ==

* How will users know that their script is doing the correct thing
* Perhaps we should have two classes of scripts: approved and validated


== Format ==

My guess is that we'll require fetchable scripts to follow a certain format on the wiki pages. That should include some metadata like

* Version number. This makes debugging easier and it makes smart caching possible
* Citation. Script authors should be able to provide a preferred citation, DOI, etc. One of the benefits is to get script authors more credit.
* Documentation. Or should this be handled in the doc string?

I think all of these things should be function attributes

<source lang="python">
def myScript(a,b,c):
    """ Documentation for myScript goes here """
    myScript.citation = "I. Coder, A Journal, 2011, Vol. 2, Issue 1, pages 64-66"
    myScript.version = 1.2

    blah
    blah

cmd.extend('myScript',myScript)
</source>

== Implementation ==

* This will obviously be written in Python.
* We'll probably make use of some screen scraping library. I don't know the state of the art here, but I've seen at least the following, and would love some comments:
** Generic interfaces
*** [http://scrapy.org scrapy] (looks reasonable, [https://github.com/clofresh/couch-crawler/blob/master/python/couchcrawler/spiders/wiki.py this page] may also be useful.
*** [http://www.crummy.com/software/BeautifulSoup/ Beautiful Soup]
*** [http://lxml.de lxml]
*** [http://wwwsearch.sourceforge.net/mechanize/ mechanize]
*** [http://code.google.com/p/webscraping/ webscraping]
*** urllib(2?) with regular expressions for that old-school feel?
** Specific interfaces
*** [http://pywikipediabot.sourceforge.net/ PyWikipedia]
*** <s>[http://code.google.com/p/python-wikitools/ python-wikitools] (this looks good to me!)</s> can never be bundled with PyMOL due to GPL v3.
*** [http://pypi.python.org/pypi?%3Aaction=search&term=wikipedia Other pages listed here]