Education/Projects/ProcessingForTheWeb

This is a project to bring a complete implementation of the Processing language to the open web, building on the existing processing.js port and other technologies.

Summary

Processing for the Web is a initiative (under the auspices of the Mozilla Education program) to create a complete and robust port of the Java-based Processing language and environment (popular for art, design, and visualization applications) to the open web, using standard technologies such as JavaScript, Canvas, etc. It builds on the existing processing.js port of Processing to JavaScript.

The goals motivating the initiative are as follows:

• to enable Processing users to leverage the full power of the open web, and help spark a new wave of innovative uses of Processing
• to serve as a forcing function to drive improvements in key aspects of Mozilla products and technologies, including JavaScript performance, Canvas functionality, applications of Bespin, Ubiquity, Multi-Touch and others
• to reach out to and increase cooperation with the design, art, and new media communities, both as part of the existing Mozilla Education program and as a component of possible future Mozilla outreach efforts directed at those communities

The initiative will incorporate the following activities:

• extending and improving the core functionality of the current processing.js port of Processing to JavaScript
• implementing a web equivalent of the current Processing development and run-time environment, with additional features to leverage the full power of the web (e.g., to support collaborative development)
• encouraging creation of new Processing applications explicitly designed for the web
• improving the underlying Mozilla infrastructure technologies required to support a powerful, robust, and conformant Processing implementation for the web

Some activities in the initiative, in particular working on the required Mozilla technology infrastructure, would be carried out by existing Mozilla core contributors. Other activities would be done in cooperation with others; in particular we will package many of the needed tasks as student projects to be done by design, programming, and other students under the auspices of the Mozilla Education program.

What is Processing?

To quote from the Processing project home page:

Processing is an open source programming language and environment for people who want to program images, animation, and interactions. It is used by students, artists, designers, researchers, and hobbyists for learning, prototyping, and production. It is created to teach fundamentals of computer programming within a visual context and to serve as a software sketchbook and professional production tool.

Processing arose out of work in the Aesthetics and Computation Group at the MIT Media Lab. It has attracted a user base of artists, designers, people doing data visualizations, and others.

Processing language syntax is based on Java, and the original Processing environment requires an underlying Java run-time. The Processing language includes standard functions for doing drawing and related operations, and the Processing environment offers a straightforward and simple way to edit and run Processing programs ("sketches") without the complexity of Java compilers and other tools.

Why Processing for the Web?

Why re-implement Processing in the context of the web? Given Processing's Java syntax and heritage, wouldn't it make more sense to encourage people to move to using web technologies like JavaScript and Canvas directly, perhaps supplemented by JavaScript toolkits where appropriate? And even if porting Processing to the web is a good idea, why should Mozilla get involved?

For the reasons discussed below, we believe that this project can bring real benefits to the Processing community, to Mozilla products and the Mozilla community, and ultimately to the open web.

It's good for the Processing community

Processing has a large and enthusiastic community of people who have demonstrated a great deal of creativity and innovation. Doing a complete and robust implementation of Processing using web technologies benefits that community in multiple ways:

• It helps get more people involved in using Processing by lowering the barrier to entry: potential Processing developers would not have to download and install the Processing envronment, but could develop and run Processing sketches directly in their browsers. The resulting sketches could also be viewed in any modern browser, without the need for browser Java plugins.
• It would help enable new types of Processing uses that leverage the power of the web, for example to access web-based data and services, enable web-based user interaction and collaboration, and so on.
• Last but not least, it could also help enable easier collaboration on development of Processing applications, given suitable support in an underlying web-based Processing development environment.

A high-quality web-based Processing implementation could help spark a new wave of innovative uses of Processing. These would be valuable both for their own sake and also for demonstrating and promoting the power of the open web technologies that made Processing for the web possible.

It's good for Firefox and other open web products and technologies

The Processing for the Web initiative could serve as a forcing function to drive improvements in key aspects of Firefox and other open web products and technologies from Mozilla and others.

• JavaScript performance. In order to compete with the traditional Java-based Processing implementation the web-based implementation of Processing will need to have performance for basic language operations that is comparable to that of Java—which means comparable to C/C++ performance, given the capabilities of today's Java compilers. Recent advances in JavaScript performance provided by the Tracemonkey, Squirrelfish, and V8 JavaScript engines have helped narrow the performance gap in modern browsers like Firefox, Safari, and Chrome. The process of testing a wide variety of existing Processing applications on a JavaScript-based Processing implementation will likely provide a wealth of information on potential areas where further performance improvements are needed.
• Canvas functionality and performance. The heart of Processing is image creation and manipulation; the current Processing implementation features no less than four different rendering modes to enable developers to achieve the fastest graphics performance possible for their particular uses of Processing. Adopting Canvas as the underlying graphics interface for Processing on the Web could help drive improvements to Canvas performance and functionality (e.g., adding 3D support).
• Applications of Bespin and similar technologies. As noted above, one of the nice things about the Processing environment is that it offers a simple straightforward interface for novices to start creating Processing "sketches". Any Processing for the Web implementation will need to duplicate this ease of use, and could also add other features enabled by the web environment, such as storing code in the cloud and providing easy but powerful ways to collaborate with others on sketch development. Processing on the Web could be a significant "customer" for Bespin, "a Mozilla Labs experiment that proposes an open, extensible web-based framework for code editing", and could help spur Bespin's transition from an experimental prototype to an actual product.

Just as a previous generation of advanced web applications drove browser vendors to improve browser performance and functionality, we believe that Processing for the Web can be a real-world application that can spur Mozilla and other browser vendors to provide solid high-performance implementations of the HTML5 family of technologies.

It's good for the Mozilla community

As has been said many times, Mozilla products like Firefox are not ends in themselves but rather a means to an end, namely creating an open web that is open, transparent, participatory, distributed, and hackable, and otherwise embodies the ideals of the Mozilla Manifesto.

In order to achieve this goal we need to continue to grow and strengthen the Mozilla community itself, in order to increase our ability to create new products and technologies for the open web, as well as reach out and work with other communities with similar goals, in order to grow the constituency of people leveraging the open web and promoting its values.

The Processing for the Web project could help achieve both these goals:

• Computer science and programming students could work on a wide variety of student projects related to Mozilla platform infrastructure needed for Processing for the Web. These would include not just coding but also performance testing and bug fixing, technical documentation, etc.
• Due to the roots of Processing in the arts and visualization, the Processing for the Web project offers an entry point into the Mozilla project for students studying design and related topics. This includes not only creating and testing new Processing applications on top of Processing for the Web, but also helping to design and build the user experience for the Processing on the Web development environment.

Processing for the Web offers an opportunity to strengthen existing Mozilla Education initiatives directed at faculty and students in computer science programs, and also to leverage Mozilla Labs initiatives (most notably the Design Challenge series) of interest to faculty and students in design programs. Given the popularity of Processing within the digital art and data visualization communities, the Processing for the Web program could also provide a key component for use in the Visualize the Web project proposed as part of the Mozilla Drumbeat initiative.

Technical strategy

Relevant technologies

The starting point for Processing on the Web is the existing processing.js JavaScript port of Processing originally created by John Resig of Mozilla and now being developed by a growing community. Processing.js provides support for the Java-based Processing language syntax, along with an implementation of the various Processing library functions.

Processing.js is usable for many Processing applications but is not yet fully compatible with the original Processing implementation. A list of what still needs to be done is available here. It also does not provide a web-based development environment, though projects like Robert O'Rourke's HasCanvas provide an initial take on what such an environment might look like.

The Bespin project could provide the underlying technology to support online creation and editing of Processing code. The challenge is to provide a Processing for the Web environment that is simple enough for complete novices to pick up quickly, compatible enough with the original Processing environment that existing Processing users can be immediately productive, and powerful and extendable enough to provide good support for advanced tasks such as debugging and performance optimization.

Other relevant projects

OpenProcessing (by Sinan Aşçıoğlu) allows people to share their Processing sketches ("a Flickr for Processing", to paraphrase one comment about it). It is an interesting example of a site that likely could benefit greatly from a Processing for the Web implementation.

OpenCode (by Kyle Buza and Takashi Okamoto) is an alternative attempt to create a web equivalent of the Processing desktop environment. To quote from the website, "What Processing has done for simplified graphics programming, OpenCode hopes to do for simplified access and interaction with Web-accessible content." It can be thought of as a Java-based Processing environment "in the cloud"; Processing sketches are based to a server to be compiled into applets, and the resulting applets are run in the user's browser using the Java plugin. The environment includes special libraries to enable interaction with web services, and advanced users can write their own libraries in Java.

High-level tasks

The following are some possible high-level tasks for this project:

• Address outstanding bugs and feature deficiencies for processing.js vs. the original Java-based Processing implementation. For more information on what needs doing see Processing.js Lighthouse tickets. The old task list is being transcribed into Lighthouse tickets.
• Surface and address bugs or performance issues in the Mozilla JavaScript engine and/or Canvas that prevent processing.js from offering comparable performance to the Java-based implementation of Processing.
• Look at Bespin architectural or other changes that would be needed in order to support a web-based development environment for Processing.
• Design and implement a first-cut prototype for a full web-based Processing environment.
• (Longer-term) Create a full Canvas 3D implementation and hook it into processing.js.
• (Longer-term) Look at possibilities for adding web-centric functionality to Processing (e.g., new functions to access web-based services from Processing sketches).
• (Longer-term) Look at possibilities for adding collaborative capabilities to a web-based Processing environment.

Current status and project team

Al MacDonald (F1LT3R on IRC) is providing advice and support to the project and is serving as the de facto technical lead.

Dave Humphrey (humph on IRC) has recruited several students at Seneca College to work on this project, as well as talking to faculty at other institutions who'd also like their students to be involved. Thus far several students at the Université Evry Val d'Essonne in France are also participating, as part of the "Course on Mozilla Education and Technologies @ Evry" (CoMETE) taught by Didier Courtaud (desiderious on IRC).

For the project status as of December 1, 2009, see Dave Humphrey's blog post "Update on the Processing for the Web Project". For ongoing information about the project, including how to get involved, check out the processing.js community page.

Notes

This section is for reviewer feedback, suggested revisions and additions, etc.

• This project could be renamed "Processing for the Open Web" (or "Processing On the Web"), with the acronym "POW" (or even "POW!" :-)
• It's been proposed that as much of the work as possible be done by students under the auspices of the Mozilla Education program. We need a good analysis of the various work that would be required, and how much could be "outsourced" to students.