Connexions for Android Updated

The Connexions for Android app has been updated. The changes are detailed on the Connexions blog. You can download the app and view screenshots on the Connexions for Android splash page. It is also available in the Android Market and in AppsLib. The source code is available in our SVN Repository. Let us know what you think at techsupport@cnx.org.

Switching from Squid to Varnish (and getting some nice benchmarking tools along the way)

We recently (November 19th) switched from Squid to Varnish for serving requests on cnx.org. The new configuration improved performance for content visitors and gave us a chance to test some new performance benchmarking tools.

We are continuing to get cnx.org ready for a major overhaul of caching to improve performance and to make sure that the tools that we use have a broad base of developer support. Some of our development and consulting partners have more experience with Varnish, and so we decided to take a look at Varnish as a potential replacement for Squid. (An earlier blog post discusses our switch to HAProxy for load balancing).

Getting repeatable tests using a cnx.org workload was the first step in our series of performance milestones. We hired Enfold Systems to develop a set of FunkLoad tests using common URLs from Connexions. The benchmark hardware mimics the cnx.org configuration and the benchmarks increase the number of concurrent requests using a set of URL's from our cnx.org logs.

This trac page has more details about our FunkLoad tests and will have links to the reports from various benchmarks as we run them.

One representative graph from the benchmarks is shown at the top of this blog. Varnish results are on top (Y axis 12/sec) and the Squid results(Y axis scale 6/sec) are below. The graphs show the number of pages returned per second as the concurrent visitors are cranked from 10 to 50. Our Varnish configuration serves about double the number of pages and doesn't experience the degradation that our Squid configuration experiences at 30 concurrent visitors. The caveat, of course, is that these results only apply for the particular benchmark and configurations that we set up for each.

The results of actual performance on cnx.org when we switched from Squid to Varnish show an improvement in the time to serve non-authenticated content (almost all of our traffic). The benefits do not appear to be helpful for authors, however. We are upgrading the benchmarks to include authenticated URLs as well so we can test performance for both. In the graph below, the Squid to Varnish change occurred at the end of week 46. The Y axis shows the time to serve a particular URL. Connexions clearly still suffers from spikes of long page delivery times, but the overall averages and the minimum serve times came down.

Comments and suggestions are welcome.

Enterprise Rhaptos 1.6 Released

Enterprise Rhaptos 1.6 has been released. It contains the new Collection Composer recently added to cnx.org as well as a handful of bug fixes.

See the upgrade instructions and the list of updated products for more details.

HA_Proxy load balancing leads to better Connexions performance

We have two different performance milestones coming up at Connexions. One to improve performance of authoring by reducing the size of certain catalogs where lookup time is related to size, and one to improve viewing times by making our slowly changing content cacheable.

Getting ready for these performance milestones, we are updating our basic request architecture and we have made some nice performance improvements in the average times for viewing content and using the authoring system. And this is before starting on the real performance work.

We were using Squid to load balance between front end zeo servers and we switched to using HA_Proxy for load balancing. Take a look at these graphs of average load times. The Y-axis shows the time to service a particular request. We measure a few different requests and each shows up in a different color. The graph on the left shows performance for the last month and the one on the right shows performance for the year. You should notice a dramatic drop in service times a week and a half ago when HA_Proxy was added for load-balancing.

The graph below shows the same timings, but for actions that authors take. It shows a similar speedup (lower height lines) for authoring. In the graphs for authoring you may also notice a major performance improvement in February of this year. That improvement was thanks to increasing the size of an application object cache.

So why would changing the load balancer have such a big benefit? We were surprised at the magnitude, but not the direction of the improvement. Squid isn't specifically designed to do load balancing, and we were using the Internet Cache Protocol (ICP) to approximate load balancing. Squid would ask frontends to respond to an ICP request and then use the speed at which they responded "no" to determine which one to choose for the request. HA_Proxy is designed to do load balancing. It did take some configuration, but it is working much better than trying to contort ICP for load balancing. Our settings for HA_Proxy choose the front end with the least number of current connections and then choose among equals using round-robin.

As background about our characteristics, Connexions serves about 2 million unique visitors per month. We receive between 50 and 60 requests per second, peaking at around 100 requests per second. Our performance is still spiky. Occasionally a request takes a very long time to serve, and that can be very frustrating for viewers and authors. We will continue to report on performance in the blog as we improve the infrastructure.

New Collection Editor

Thanks to @hedleyroos and @rochecompaan out at Upfront Systems, we're delighted to have a shiny new collection editor. There's a separate post that gives a great overview of how to use it but in this post I'll go under the hood and describe a bit about the technologies used and how they were implemented. Finally, there are a couple of "gotchas" we ran into that are worth discussing.

The editor uses two JavaScript frameworks, ExtJS and jQuery, along with Zope Page Templates (ZPT). Zope is the templating engine used to implement Connexions. The editor uses AJAX (Asynchronous Javascript And XML) to call the server on Connexions. This eliminates the need to refresh the entire page as the old editor did. Users start off with an ExtJS tree that shows how the collection is organized. The tree is reorderable with links to add/delete Subcollections or modules. On load, the collection hierarchy is requested as JSON from a ZPT. JSON (JavaScript Object Notation) is a lightweight data-interchange format.

Every time a module or subcollection is dragged and dropped, a call is made to Zope to update its objects. Since these objects can be moved anywhere in the hierarchy they may get a new parent. To handle this in Zope, we cut, paste, and reorder.

Gotcha 1: Initially the Zope UID was used to keep the Javascript and Zope objects in sync. However, when an object is pasted the UID changes. In the code both the UID and path to the object (using ids) were used but in order to support moving the node to a different parent both would have to be updated so we switched to calculating the path on each update.

The editor also has popup dialog boxes for editing properties of the collection. When a button to open a dialog box is clicked an AJAX call is made to Zope and the resulting HTML is injected directly into the page. The HTML that is sent back also contains a Javascript tag that contains all the logic needed for the dialog. Sending back HTML from the server has the advantage of using the Zope internationalization machinery to support other languages and by keeping the Javascript in the same file it is easier to keep track of which code goes with which dialog box.

Gotcha 2: Those pesky corner cases (in form submission, cancellation, and validation).

Since we request HTML from Zope, very little state is stored on the browser. To support canceling a dialog box, we had to first store the default values. When a user enteres some invalid data and submits, Zope does a nice job of rendering the form with how to fix the problem. However, if the user cancels, we needed to scrub the form clean, removing the validation errors as well.

So, that's a quick run through the new Collection Editor and if you're interested in the code, it can be viewed at our trac page.

WebCraft Textbook -- be a part of building it this week.

The Connexions blog describes the web development textbook we are building on Connexions during the Mozilla Drumbeat festival this week. Folks on this mailing list would be excellent authors of chapters and sections for the book. Join us. -- Read more here -- http://blog.cnx.org/2010/11/webcraft-101-introduction-to-web.html

Connexions App for Android Released

The Connexions team is launching an Android application for the Connexions website. I developed the application as a side project to my regular job as the Connexions development manager.

Since this is the first version of my first app, it could use a few more features. I'll be adding them in the near future. However, I believe in not waiting until software has every feature completed before releasing it. I'd rather get some feedback so I can include requested features as I add the planned ones.

The App has the following functionality:

• Search Connexions using Google Site Search and view Modules and Collections found in the search results.
  
• View the list of Lenses from Connexions, view the content of a Lens and view the Module or Collection that is part of a Lens
• Save a search, Lens, Module or Collection to a list of Favorites that is stored locally on the phone. It is not the same list of Favorites you might have on the Connexions site. There is no login required. You can select a favorite from the Favorites tab and view it.
  

The application was built using existing feeds and URLs from Connexions. I used Google Site Search because it is a feature our users have asked for in the past. It will allow you to search for items which our current search on Connexions does not allow such as searching for lenses and in the Connexions help files. The Lenses and Lens ATOM feeds provide the data for the listings. The content is displayed as it currently is on Connexions. We hope to have a more mobile friendly version in the near future.

I would like to thank Max (the Connexions designer) and all of the beta testers for their help. The app is free and can be downloaded from Connexions. It is also in the Android Market. The application help file gives a good overview. Source code for the app is released under an LGPL license like all of our other code so it is free and open source. You can browse the code from our SVN repository.

The app was tested on an Archos 32, Droid X, Droid, enTourage eDGe and a Haipad M701 using Android versions 1.6 to 2.2. I hope it will be used by students as a quick way to access our content on the go. I'll keep improving it since developing it was a lot of fun. If you try out the app, let me know what you think at techsupport@cnx.org.

Sprint with Us

Calling all developers. Come join us at the Plone Conference in Bristol, UK, October 25th to the 31st. Connexions and Connexions Consortium board member, Upfront Systems, will be running a sprint at the upcoming Plone conference. The topic is "Migrating Rhaptos to Plone 4" and should be interesting to all Rhaptos and Connexions developers, those wanting to learn about Rhaptos/Connexions development, and to Plone developers interested in migration to Plone 4.

If you miss us in Bristol, join us at the Connexions Conference, February 9th, and stay for a sprint on February 10th. We plan to have sprints regularly to help grow the developer community around Connexions and Rhaptos.

And, look for us in May at the Plone East Symposium at Pennsylvania State University. Sprints are May 20th and 21st, 2011.

Rhaptos Answers Revealed

So, you've installed Rhaptos or you are thinking about giving the new Enterprise Rhaptos 1.5 a try and you have questions. Well, we have some answers in our Buildout FAQ. We solve mysteries such as...

• Where is the source code for Rhaptos?
• What are the Debian dependencies for Buildout?
• How do I delete a user?
• How to improve the performance of Rhaptos?

This is just a sampling of the content of the FAQ and we are updating it regularly. If you run into a Rhaptos question or problem, check the Buildout FAQ. If the FAQ does not have the answer you need, consult the Rhaptos Developer mailing list.

Enterprise Rhaptos 1.5 Released

We are proud to announce the release of Enterprise Rhaptos 1.5. There are several new features.

• Express Edit - If there is a published collection or module that you would like to edit from your Personal Workspace or a shared Workgroup, you can now quickly check out a copy directly from the content page itself.
• Enhanced Google Analytics - Google Analytics tracking was added to all download links.
• EPUB - Module and Collections can be downloaded as EPUB ebook files.
• Offline HTML - a zip file containing all information about a module or collection along with an HTML version that can be viewed offline. This is created as part of the EPUB generation.
• SWORD Interface - Modules can be created by sending a POST request to a URL along with a zip file containing a Word document and optional mets.xml metadata file.
• Updated Help files covering new features.

Documentation

• Installing Enterprise Rhaptos
• Upgrading Enterprise Rhaptos
• Express Edit
  
• Enhanced Google Analytics
  
• EPUB:
• Help
• EPUB Guidelines
• EPUB Help
• EPUB FAQ
• Code Documentation
• Sword Interface
  • Sword Help
  • Code Documentation
    

Feedback
If you have a question or think you found a bug, try posting it on the rhaptos developer mailing list.

Introducing Connexions SWORD API

In collaboration with the Public Knowledge Project, makers of the Open Journal System (OJS) software, Connexions has rolled out a new API allowing content in Word format from OJS journals to be imported for republication in Connexions/Rhaptos. The transfer is accomplished via SWORD (Simple Web-service Offering Repository Deposit). See our development site for technical documentation.

From an OJS journal that has been set up to work with Connexions, an author or editor enters their Connexions username and password and is provided a list of their work areas in Connexions. Having selected one of these, the user then chooses the articles in Word format that they want to deposit, and the submitted articles will be imported into the indicated location in Connexions. A new module is created with the title and metadata provided by the journal.

Since the content is likely to have already been published in the journal, features have been added to support that workflow.

• A citation to the original publication is automatically added to the end of each module (this can be removed by the author), if one is provided by the journal.


• A prominent reminder to review the citation, to review the authors/editors, and to preview the module appear in the module editing interface.


• The role management interface includes convenient links to search the repository for names of people found as role-holders in the original publication.


• The Creative Commons license must be agreed to before the content can be published in Connexions/Rhaptos.

While developing this new feature, we also provided a pathway for users of all content in Connexions to upgrade the version of their existing Creative Commons licenses. Content published under older versions of the license now require an agreement to upgrade to the latest version before it can be republished.

Since SWORD is an open standard, The Connexions SWORD API can be used outside OJS also. The API covers new module creation with appropriate metadata and importation and translation of Word documents into CNXML in new modules.

This work was supported by Indiana University as part of the Big Digital Machine project to enhance scholarly workflow for the digital age.

Introducing Nathan Yergler

"Connexions has been working with Creative Commons from the time both organizations were just visions in their founder's eye. The open, yet clear licenses that Creative Commons fostered to allow sharing and reuse are key to Connexions' mission to be a global repository of high-quality, reusable and adaptable textbooks and educational resources. Nathan, Creative Commons CTO, is our trusted guide on issues of standards, interoperability, and discoverability."
Katherine Fletcher Technology Director and Project Manager for Connexions

I'm Chief Technology Officer at Creative Commons (CC). My responsibilities include managing the team that builds the technical infrastructure behind Creative Commons legal tools. I'm also responsible for looking at ways Creative Commons technology can be applied, and how our experience and expertise with linked open data can be leveraged. Open educational resources are a core application of Creative Commons licenses, as they depend upon license interoperability to scale.

I initially became involved with Connexions through my work at CC. Connexions is one of the great exemplars of a community built around open licensing and re-use. The Creative Commons Attribution license that Connexions' contributors offer their work under reduces ambiguity and removes a friction point for reuse and redistribution. Connexions' support for remixing and reusing content is one of the best, which makes it an extremely interesting platform to be involved with. And as a long time Zope developer, I was further drawn in by the use of Zope and Plone.

I see the Technology Committee as a resource for Connexions and Rhaptos developers to use to help think about ways the already successful platform can be further scaled and improved. As a member of the committee, I hope I can provide perspectives on how other communities are using CC licenses, and how Rhaptos can continue to be an exemplar for encouraging sharing, remix, and reuse.

I'm excited about working on ways to improve search and discovery for Open Education Resources; Connexions' lenses are a great initial implementation of a curatorial system that I think will be central to this. I'm also interested in enabling the discovery of derivatives and transformations of OER across the web.

Introducing Mark Horner

“I am pleased to introduce Consortium technology committee member, Mark Horner of the Shuttleworth Foundation. Mark and the Siyavula Project have been incredible partners with Connexions in education and Open Education Resources (OER). Siyavula has contributed an entire curriculum for K-9th grade and science and math for 10th-12th to Connexions. Siyavula works directly with teachers in South Africa to effectively use Connexions and when they identify critical new features for Connexions, they help to design, fund and develop them. This combination of vision, real world experience, and technical problem solving is what makes Mark so valuable to the Consortium and the Technology Committee. Mark's vision extends to the entire teaching and learning process as you will read below."
Kathi Fletcher -- Technology Director and Project Manager at Connexions

I am the Open and Collaborative Resources Fellow at the Shuttleworth Foundation, where I am responsible for a suite of projects aimed at supporting the full teaching value chain through the creation of an enabling environment for communities of practice of educators to thrive.

I ended up in Open Educational Resources by accident, coming from a background in high-energy nuclear physics. During my graduate school years, I co-founded the Free High School Science Texts (FHSST) project in my “spare” time, a project that has successfully used a global network of volunteers, collaborating online, to write free, openly licensed textbooks for Grade 10-12 Physical Science (a mixture of Physics and Chemistry taught in South Africa) and Mathematics. This was to address the fact that many learners in South Africa just didn't have access to textbooks.

The success of FHSST helped me get a job at the Shuttleworth Foundation, originally as a project manager and then later as a Fellow, where I started the Siyavula project. The original vision for Siyavula was to extend the FHSST work to all grades and subjects. The Siyavula project has successfully released a set of English and Afrikaans workbooks for all learning areas for Grades R-9 in an open format under an open copyright license. Siyavula has since extended its role to become the umbrella under which
FHSST, and my other projects, exist.

The Siyavula project partnered with Connexions to house the workbooks and allow educators easy access as well as the opportunity to adapt, enhance and contextualise the materials. In collaboration with Upfront Systems and the Connexions team we've also dabbled in the development of Rhaptos.

Access to these resources online alone won't support teachers delivering the content effectively in the classroom and so, in collaboration with Steve Song, Interconnectedness Fellow and Roché Compaan, Upfront Systems, we have founded OpenPress, a project to deploy a print aggregation service to support teachers and give under-resourced schools access to cost-effective open educational resources through the power of collaborative purchasing. This will allow high quality hard-copy resources to be delivered to schools.

Teachers are also required to do assessment, a crucial piece of the puzzle if learning and material effectiveness is to be gauged and fed back into further improvements of material and teaching process. My most recent project, under the Siyavula banner, is FullMarks, an openly licensed, open-source assessment bank designed to be extremely simple to use and allow educators to rapidly share questions, create tests and analyse learners results.

There are a number of additional projects in the pipeline, most notably the development of a mobile front-end for the Connexions platform which will allow learners with access to a cellular telephone the ability to browse the full Connexions repository without requiring access to an internet connection through a PC. In contrast to developed countries where the mobile focus is on smartphone applications, we are focused on simpler devices to allow the vast majority of South Africans access to the content via their phones. This will ensure that the 85%+ of South Africans with mobile phones can access Connexions in contrast to the 10% of South Africans with internet access via a PC.

I sincerely believe that education is the only route to long-term, sustainable, peaceful and prosperous future for all South Africans. I blog about all of my projects and my fellowship at markhorner.net.

By participating as a member of the Connexions Consortium Technology Committee I hope to support the platform's development and reach into the K-12 (R-12 in SA) sector. Connexions' forward thinking use of XML, homogeneous repository and typesetting capability make it an ideal place to build a critical mass of K-12 content that will benefit a global community of educators.

Vote for Connexions Plone talk!

Connexions and Upfront Systems will be running a sprint at the upcoming Plone Conference in Bristol, UK, October 25th to the 31st. This is part of our developer community expansion to broaden the base of technical expertise for Connexions and Rhaptos. These events build on the momentum of Enterprise Rhaptos and the Connexions Consortium.

We have also proposed a high level talk "Connexions and Enterprise Rhaptos -- A Global Education and Plone Success Story" and we need your vote of interest." For the talk, we need votes from the community so the program committee knows that the talk will be well received. Please vote for the Connexions talk using the link below. (You will need to scroll down to our talk and click to vote.)
http://www.ploneconf2010.org/the-event/talks

Two of our consortium members (Shuttleworth and Upfront Systems) are proposing talks. Mark Horner from the Shuttleworth Siyavula project will talk about FullMarks, an open assessment bank. Roché Compaan from Upfront Systems will talk about tuning Plone for speed. Jazkarta, who worked with Connexions on virtualization, will talk about their work with Amazon's AWS for Enterprise Rhaptos.

Each of the talks is listed below. Please vote for us.

1. Connexions: "Connexions and Enterprise Rhaptos -- A Global Education and Plone Success Story"


2. Upfront Systems: "Lose weight now, ask me how!"


3. Mark Horner: "FullMarks for Plone :)"


4. Jazkarta : "Scalable and high availability Plone hosting with Amazon EC2 for Rice University's Enterprise Rhaptos open education platform"

EPUB Development - The Details

Connexions recently introduced EPUB files for Collections and Modules. EPUB files are electronic books (ebooks) that can be read on mobile devices, dedicated electronic book readers, and on computers running ebook reader (eReader) software. Creating the EPUB files had a unique set of problems and solutions.

Our content is stored as XML files at the Collection (collxml) and Module (cnxml) levels. To convert a Collection to an EPUB, we first retrieve the Collection and Module XML files. Then the files are converted to Docbook and then into HTML which is used to create the EPUB files. Docbook already had a Docbook-to-EPUB transformation that generates the EPUB specific configuration files and directory structures. The Docbook-to-EPUB transformation is geared toward standard books, not textbooks. It was lacking some transformations for biblographic information, annotations and more detailed footnotes. Linkage between Exercises and Solutions was also added. All of this is triggered by a Ruby script that was originally from Docbook.

Because of the small screen size that EPUBs are usually displayed on, a new stylesheet had to be created. The new style has less indentation so the text uses all of the available screen area. Boxes around notes and examples were also modified to keep the separation of the content, but to limit the space taken up by the box.

We tested the EPUBs on a variety of devices. The initial developer testing was done using the ebook reader in Calibre or the Firefox EpubReader plugin. Our formal testing used an iPad and an iPod Touch. We tested using Stanza and iBooks on both devices. More recently, we have tested on the enTourage eDGe. Some outside testing was performed by "friends and family" on a variety of platforms including iPads, iPhones, iPod Touch and Droid phones.

There were several challenges in creating the EPUB files:

• Handling MathML: EPUB readers do not support MathML, so the obvious solution was to convert simple MathML to text and convert all of the more complex MathML to images. We first tried converting to SVG, but found that although SVG is part of the EPUB spec, it is not supported by ebook readers. Converting to PNG solved the problem. It is supported by all of the readers.
• Cover Images: Book cover images were needed for the "bookshelf" display in ebook readers. The image needed to be customized with the title for each EPUB and needed to be generated when the EPUB was created. Our team created an SVG image of the cover. When an EPUB is created, the title of the EPUB is added to the SVG and it is converted to a PNG image for the cover. The same image is used as part of the title page of the EPUB. The image is created using part of the as-yet-unreleased SVG1.1 spec for automatic text reflow (using Inkscape).
• Metadata: CNXML had a problem with Module metadata that was corrected in version 0.7. However, most of our content is still stored as CNXML 0.5. We solved this by creating a CNXML 0.7 version of the CNXML for every module as part of the Collection source zip and the Module source zip. The EPUB generation uses these source zip files to retrieve the XML. This upgraded version of the CNXML allowed access to the needed metadata and will allow developers to use the source zip files without having to be concerned about the version of the CNXML.
  
• EPUB Limitations: As we have tested our EPUB files, we discovered some limitations with the EPUB format and readers. Since Connexions authors have not been entering content with EPUBs in mind, some content does not display correctly as an EPUB. We created a set of author guidelines that discusses the limitations found and offers some suggestions for entering content that will display correctly as an EPUB.
• Offline HTML Zip: There have been numerous requests for a downloadable HTML version of Modules and Collections. Creating EPUBs required the creation of a new HTML version of the content, so it seemed like a good time to offer an Offline HTML. We soon discovered that what looked good in an EPUB reader does not always look good in a browser. Our original thought was to reuse the HTML generated for the EPUB, but that was abandoned for a separate HTML generation that corrected the problems. See the downloads help page for more information on the files available.
  

The EPUB files are an important step in allowing Connexions users to have their content easily available on multiple platforms. We welcome any feedback so we can continue to improve them.

Introducing Minh Do

“Next in our series of introductions of the members of the Connexions Consortium Technology Committee, I am delighted to introduce Minh Do, the director of a team that was one of the first to embrace Connexions and Enterprise Rhaptos internationally. The Vietnam Foundation has been working with Connexions and running Rhaptos in Vietnam since 2006 and they have been pioneers in the areas of internationalization, proxy caching, language support in PDF generation, customization, training and local support for Rhaptos. Their vision for a 21^st century education in Vietnam is inspiring and their commitment unwavering. Minh and his team have been trusted advisors in the development of Enterprise Rhaptos and they have been key beta testers for new features in the software, offering their time on short notice and helping ensure quality."
Kathi Fletcher -- Technology Director and Project Manager at Connexions

I am Minh Do and I am the Vietnam Open Education Resources (VOER) Program Director at the Vietnam Foundation (VNF). My background is in eLearning, open course ware, and information technology. Before joining the VNF, I was the Vietnam Open Course Ware Program Manager for the Vietnam Education Foundation (VEF). And prior to that, I was the Deputy Director of the Center for e-Learning and Online Testing Technology at the Information Technology Institute (ITI) at the Vietnam National University in Hanoi (VNU Hanoi). There, I coordinated the "in-country training course," for the second phase of the "Vietnam Information Technology Training Project" hosted by JICA Japan and VNU Hanoi.

My involvement with Connexions began in June 2006 at the CNX training workshop at VNU Hanoi. I met Hung Tran from the VEF technical team and Rich Baraniuk, Sidney Burrus, and Chuck Bearden from the CNX team. I was impressed by the software features and the LEGO idea and kept in touch from that time on.

My e-Learning center collaborated with the VEF team to increase the Open Coursware (OCW) activities in Vietnam by installing a version of Rhaptos in Vietnam and training users how to use Rhaptos to create and share their knowledge. We volunteered to translate the Rhaptos interface into Vietnamese and I myself went around the country with Hung Tran to set up the caching servers to prepare for the launching of the VOCW Program. Day by day, I got more and more deeply involved in VOCW. Finally, I decided to fully join the VEF team and lead the program with Hung Tran.

The transition from VOCW to VOER is quite a long story, but now, let’s talk about where VOER is heading with its “3 development legs” model (Technology, Content, and Community):

• Technology Development: Having an innovative, reliable, and inexpensive platform plays a vital role for any OER project. Thanks to the release of Enterprise Rhaptos (ER) from Connexions, we were able to deploy a robust software platform for VOER with a major data center located in Hanoi. The VOER website is now available for free public access from the Internet. We are also cooperating with an open source development company named NextG Solutions. NextG specializes in Zope/Plone solutions and will test and customize ER, and develop more functions for ER as the community needs. A support center named Connexcent (Asia-Pacific Connexions Support Center) was established under the umbrella of the VNF in order to provide best practices and support services to the regional community in particular and the global community in general.

• Content Development: Besides the technology, content is the heart of VOER. To provide the seed content for the project, we aimed to collect teaching materials from various sources and make them available on the VOER website. Connexions Rhaptos makes the usage of textbooks more effective by dividing a textbook into independent parts, or modules, and allowing combination of modules from different textbooks to create a new one. In the first year of VOER, we plan to publish 20,000 modules on Rhaptos. The first source of available textbooks is a library of 1000 textbooks from MOET (Ministry of Education and Training). Other sources include universities, companies, and research institutions. After just a few months working with these organizations, we have around 10,000 modules available in VOER.

• Community Development: The last leg makes the project sustainable. We are in the process of building the community for VOER by cooperating with our partners in technology development and content production, providing training to students and faculty members on using and sharing content, and sharing information about VOER with the public. At the moment we have established the Alliance for Vietnam Open Learning Technologies (VOLT); VOLT has early members from important entities in Vietnam such as FPT (the largest technology company in the country), the Institute of Sociology (the leading research on sociology in the country), the Online Management Training Company, the Center for Promotion of Advancement of Society (a non-profit organization, belonging to the Vietnam Union of Science and Technology Associations) and the Hanoi Obstetrics And Gynecology Hospital (the leading and first-of-its-kind hospital in Reproductive Health Care in Hanoi). Members of this community are now using customized versions of Enterprise Rhaptos to create and share knowledge both internally and externally. In addition, we have won participation commitments from major universities in Vietnam such as the University of Danang, the University of Education, and the Vietnam National University in Hanoi and HoChiMinh City.

Based on my experience with VOER, as a Connexions Consortium Technology Committee member, I really want to help accelerate all the “3 development legs” activities for both Connexions and Rhaptos. The software needs to be more user-friendly, more flexible to customize, as well as able to handle large amounts of uploading and sharing. The community needs expansion beyond individuals to organizations like universities or institutes, because the number of users will determine the success of CNX. Besides that, content on different Rhaptos instances should be linked together to help people use modules and collections more flexibly and easily. As I am focusing on the development of Enterprise Rhaptos, I hope our technology team will be instrumental in the very near future in designing and implementing a way to link all the needed modules from any Rhaptos repository into collections.

Please visit the links below to see more about what we are doing.

• The Vietnam Foundation: www.vnfoundation.org
• The Vietnam Open Educational Resources Program website: www.voer.edu.vn

Introducing Roché Compaan

“I am pleased to introduce technology committee member, Roché Compaan of Upfront Systems. Roché's expertise in software development and Plone and Zope technologies has been invaluable, and Upfront Systems has rapidly become expert in Connexions/Rhaptos development. Over the past year and a half, Upfront built several new lens features used by Siyavula, a proxy-cache to speed up access in South Africa, the new Connexions rating system, and the new Express Edit feature that helps authors quickly check out content for editing and helps readers derive a copy to adapt. Roché's performance expertise and advice led to a configuration change that halved the time authoring tasks consume. We are very lucky to have his involvement in the Consortium and Technology Committee."
Kathi Fletcher -- Technology Director and Project Manager at Connexions

In 1998 I co-founded a software development company called Upfront Systems located in Stellenbosch, South Africa. Very early on I felt myself drawn to the open source movement and thought that this was a very healthy an productive protest against the establishment. I didn't think it was crazy to build a business on open source principles, but my partners did and since 2000 I have been the sole owner of the company.

Around 2001, a colleague of mine encouraged me to look at Zope. It was a web framework that was years ahead of its time. It was a significant departure from the then common cgi style web apps and it boasted an object database, a multi-threaded web server and publisher that could traverse and publish objects. I have been involved with Zope and the community around it ever since, and saw Plone grow up to become one of the major content management systems in the world. Upfront Systems was the first Zope solution provider in South Africa and we contributed a Zope and Plone training course to the community early on. We used this same course to train developers at Computer Associates in New York when they had a brief flirt with Plone in 2004.

About 3 years ago I met Mark Horner, the Open and Collaborative Resources Fellow in the Shuttleworth Foundation. As part of his Siyavula project, Mark was looking for a platform to use for the publication of a whole curriculum of workbooks bought from a private school in South Africa. Connexions caught his eye, mainly because it did a darn good job at producing printed books by using Latex for typesetting. Mark is a Latex junky. So when Latex junky and Plone pundit met, no other framework stood a chance. Admittedly the patience and charm of the Connexions team had a lot to do with the choice to go with Connexions as a platform. Over the past few years we've undertaken numerous 20 hour trips to Houston to scope and plan the development of extensions to Connexions. The extensions helped us present content in a way that is a familiar to South African teachers, while ensuring that the features that we develop are generally useful to other Connexions users.

We were the first external development team that worked on features that would be released on cnx.org itself. It shouldn't come as a surprise that this wasn't smooth sailing in the beginning. But I believe this is exactly what Connexions needed - a remote development team that can help surface development practices and knowledge that were mostly held by members of the Connexions team and not visible to the outside world.

As a committee member I would like to focus on growing the Rhaptos developer community. As a long time member of the Plone community I will naturally look there to recruit developers. At the upcoming Plone Conference in Bristol I will lead a sprint where we will start the migration of Rhaptos to Plone 4.0. Rhaptos is still running on Plone 2.5 and moving it forward to the lastest Plone version would make developing for it significantly more attractive for existing Plone developers.

Looking forward to see you all at the Plone Conference in Bristol!

Introducing the Connexions Consortium Technology Committee Members

Over the next few weeks, the Connexions Consortium Technology Committee members will be introducing themselves on this blog to the wider developer community. The Connexions Consortium is a group of organizations and individuals, including the world's foremost leaders in education, who work together to advance open source educational technology and open access educational content.

The technology committee is responsible for the technical aspects of the Connexions Consortium. Among other things, the technology committee is responsible for the technical development, implementation, and maintenance responsibilities that occur on the Connexions platform. The committee reviews and makes recommendations to the Board about the technical affairs and policies of the Connexions Consortium.

We are lucky to have a really outstanding and committed group of eight individuals on the technology committee. They participate in conference calls together about every 6 weeks, and they provide expert input to Connexions, the Connexions Consortium, and Connexions and Rhaptos partners. I look forward to introducing them to you.

EPUB mobile development underway

This summer Connexions/Rhaptos development team at Rice is developing mobile versions for all of the Connexions content based on the EPUB standard. The development fortuitously builds on code one of our developers created in his spare time, highlighted in our showcase for partial and experimental work, Rhaptos Lab. The William and Flora Hewlett Foundation generously awarded us a specific grant to finish an EPUB export for Connexions content this summer.

EPUB is supported by the major mobile devices and e-book readers including Apple's iPad and iPhone, Sony’s eReader, and enTourage’s eDGe. EPUB books can also be read offline on notepads and laptops and the EPUB format is used by accessibility services that enhance content for use by individuals needing assistive technologies for reading.

The new EPUB export will work like our current PDF export system and when authors update any part of their content, new EPUB versions will be automatically created, just like we do now for PDFs. The EPUB exports will be available on cnx.org and will be available for Enterprise Rhaptos installations also.

For developers interested in following the design and development of EPUB, see our development milestone.

Connexions/Rhaptos Sprint Topics at Plone Conference 2010

Connexions plans to organize a sprint on Connexions and Rhaptos development at the Eighth Annual Plone Conference in Bristol, UK (October 25th - 31st). We would like to hear your ideas for topics to sprint on. We also want to know whether you plan to attend the conference this year and would be interested in learning more about Rhaptos and Connexions and getting some experience coding. Our goal would be to increase the pool of developers familiar with Rhaptos, let experienced developers get to know each other better, and explore topics that may become development priorities in the future. Some of the topic ideas we have been considering include integrating an HTML editor that converts to our document XML (CNXML), implementing an existing specification for enhancing member profile pages (Author profiles), migrating to Plone 3 or Plone 4, or working on some aspect of internationalization. Post your thoughts here, email me, kef at cnx dot org, or email our developer mailing list (go here to join). We will send out more detailed ideas in a future post.

Ready for pre-alpha virtualization testers

We are ready for pre-alpha external testing of running Rhaptos on Amazon Web Services. You will need an AWS account. We have tested starting instances from an Ubuntu laptop and from Mac OS X and now we are ready for others to test these initial configurations.

The instructions are here: https://trac.rhaptos.org/trac/rhaptos/wiki/AWSQuickInstall

The following two configurations should work:

1. rhaptos-partial-32 : Rhaptos instance connected to a sample set of data for demos and exploration. It takes about 5 - 10 minutes to build and then another 5-10 minutes for reindexing the sample data set to use the amazon URL your instance is running on.
2. rhaptos-empty-32-dev : A Rhaptos instance that starts with a machine image with system dependencies installed and then builds all the Rhaptos code from a known working release. Takes about 5-10 minutes to start the machine that does the building and then about 20 minutes for the build.
   

Known issues so far: The workgroup links in the left navigation bar have the URL doubled and thus 404. You can hand fix the URL to get to the workgroup. We are looking into this. You will probably find something we didn't.

We are testing the clarity of the instructions, as well as the functionality of the result. We would love to hear feedback on how it goes. Please email cnx@cnx.org with your results and the rhaptos software development list (rhaptos@cnx.rice.edu) (which you have to join first).

Consortium member, Shuttleworth, funds performance improvement specification

Read Mark Horner's blog entry "Connexions Authoring Performance" for the full detail about the specification they funded for improving the performance of authoring on Connexions. Mark is a Shuttleworth fellow and The Shuttleworth Foundation is a Connexions consortium member. They have been using Connexions, developing new features for Connexions, and funding the creation of specifications for new features that may need additional funding to get off the ground

From Mark's blog:

"... the bottom-line is that Upfront Systems have shown that there are massive potential performance improvements (in some cases 5 times faster) for Connexions on the authoring side and analysed what it will take to implement them. I estimate that the total cost of implementing the specification is $7500."

This is exactly the sort of dynamic engagement we hoped would come out of the Connexions Consortium.

You can read the full reports at:

• Analysis of performance (Open Office format)
• Draft specification to develop the fixes to the problems identified. (Open Office format)

Supporting the Life Cycle of Scholarly Work

From research to the classroom

What students are learning today is often the result of cutting edge research in the past, and what advanced students are learning and incorporating is often today's research. So we are making it easier for content to move from publication in open research journals to publication in the Connexion open education repository. We are collaborating with the Public Knowledge Project, makers of OJS (Open Journal System) software, to support exporting content from journals and importing it to Connexions. The two teams are each implementing SWORD (Simple Web Service Offering Repository Deposit) to accomplish the transfer. Connexions SWORD support design is shown on our development site and implementation began in March.

From Connexions to reuse anywhere

We collect all sorts of statistics about use of the Connexions content from Connexions website, but often students and teachers download the content in various formats and use it remotely. Connexions explicitly supports reuse of content inside OR outside the repository. We try to make it easy to use the content in whatever way is most appropriate for educators and learners. But, we still want to know when the content is downloaded. Our current Google Analytics setup doesn't provide that information and so we have updated the code to collect information about content downloads. Testing is underway on the statistics enhancements. As an added benefit, consistent use of analytics across services makes comparisons and research on content usage more accurate.

Virtualization Update

As many of you know, Connexions has teamed up with two contractor teams to develop virtualized versions of Connexions and Rhaptos. Virtualization of the Connexions platform will allow production environments to be scaled as demand increases and decreases, will be fault tolerant regarding hardware malfunction, will support rapid testing, and will support providing demonstration instances of the software when needed. Virtualization will also make running Enterprise Rhaptos instances easier and make providing hosting services for Rhaptos instances a cost-effective third-party service.

We have a diagram of the various options that are being created. To follow along with some of the work as it is in progress, we have a milestone for virtualization in trac. Because the contractors use their own development systems, the milestone doesn't have everything that is going on.

As this work progresses, we will be requesting that external teams start testing some of the intermediate milestones of the virtualization project. Please let us know (email cnx@cnx.org) if you want to be included in that beta testing.

Feedback from community members on the Connexions Conference

Tom Caswell makes several very interesting comments about Connexions, Rhaptos, and the Connexions conference on the OCW blog, "Good things brewing at 2nd Connexions conference." He was particularly interested in the release of Enterprise Rhaptos, the MathML editor, Google Analytics for authors, and potential Common Cartridge support. He also encouraged us as a developer community to really concentrate on a simple HTML editor for Connexions and on how Enterprise Rhaptos and cnx.org (the "mothership") will cohere as network of repositories in the future. My response has pointers to more information on the features he was interested in and pointers to experimental work on HTML editing in the Connexions shop.

Extreme Makeover - Documentation Edition

With the release of Enterprise Rhaptos, we have improved the documentation on the Rhaptos Trac wiki. The documentation has been expanded and rearranged, and includes much deeper technical documentation.

The Rhaptos Trac wiki home page now has quick links at the top of the page to information about Enterprise Rhaptos, Connexions/Rhaptos Development, APIs for Accessing Data and Test and Quality Assurance.

Enterprise Rhaptos

The Enterprise Rhaptos documentation includes a quick install page that can have you running Rhaptos in only a few minutes. The install uses Buildout and can install the Rhaptos software on your Debian Linux server. Enterprise Rhaptos is also changing the way we do development internally at Connexions. We can build development instances much faster than before and the code structure in Subversion (SVN) has changed. There is a clean separation between Rhaptos and Connexions code which has led us to have 2 top level folders in SVN, rhaptos and cnx. The previous method was for each product to be in a top level folder.

Connexions/Rhaptos Development

Connexions/Rhaptos Development starts with links to our mailing lists, this blog and some older developer blogs on the rhaptos.org site. If you are thinking of doing some development, please sign up for our mailing lists. There is lots of good information being passed around, but not so much that it will fill your inbox everyday. There is a Roadmap of our plans for Connexions development through June 2010.

Technical Documentation

Technical Documentation is where we have expanded and added the most documentation. We now have instructions for installing Rhaptos and Connexions. There is also a Buildout FAQ to help troubleshoot your installation. As part of Enterprise Rhaptos, Connexions has created a large body of Selenium tests for the site. These have been exported into Python. Details of running these tests with Buildbot or manually are now documented. We are starting to use Buildbot as a continuous build system for our development. This should help us find defects earlier in the development process and improve our code quality.

Code Documentation has been added and brings some much needed information about Connexions and Rhaptos source code. There is documentation covering each product in Connexions, some general Plone information and a few tips and tricks. Also covered is most of the functionality in Connexions such as Collection Editing, Content Rendering, Edit In Place, Lenses, MathML Message Behavior, Module Editing, Monkey Patching, Printing, Schemas and Workgroups.

APIs for Accessing Data

This section has links to information about our RSS feeds, URL's to get rendered pages of content, member profiles, lenses, statistics, etc. and Open Archives metadata harvesting. Also included are links about downloaded and branding Connexions content. Kathi wrote an earlier blog post about this on the Connexions blog.

Test and Quality Assurance

Test and Quality Assurance remains to be improved and cleaned up. However, there is some useful information in the Test Plan section. We have used several of these to create our new Selenium tests.

Our hope is to continue improving our documentation with help from the Connexions team, Consortium members, authors and users. If you are interested in contributing or have a comment about the new documentation, just let us know at cnx@cnx.org.

Welcome to Connexions and Rhaptos Software Development Blog

What is Rhaptos?

• The name for the open-source licensed software developed and used by Connexions.
• An Ancient Greek adjective that refers to things stitched or sewn together.
• Software to enable authors, instructors, and students to create, select, and assemble modular educational content into collections customized to meet their teaching and learning needs.

Features

• Modular content that can be combined and reused in many ways
  
• Version control with permanently referenceable versions
• Semantic XML-based content
• Lenses for endorsing, recommending, or suggesting content
  

Getting Started with Rhaptos Development

• Installation Guide
• Mailing Lists
• Development Roadmap Overview
• Bug System Some of our most active development and design is detailed in the Trac bug system
• Request an account Need to contact us or request an account? -- Email cnx@cnx.org and tell us a bit about yourself. You do not need an account to add or comment about bugs in the bug system or to see or use any of the source code.