Jenkins forces reinstall of server images on Rackspace

This is much more DevOps than Dev, but it is important as well ! One difficulty we have doing remote development with a team spread over at least a million cities (I haven't actually counted) is that the local development LAN for Bob is frankly inaccessible for Alice and Charlie. Maintaining n-1 VPN links is just going to drive us all mad.

So the simple solution is to rent a couple of cloud servers and host the development builds on there. The cost is pretty minimal, and it seemed good - until I realised that Jenkins, our friendly Continuous Integration Butler, was unable to re-install a cloud server from an image. Or rather it *could* - but Jenkins did not know when the server was rebuilt, and the server itself, which could signal jenkins, would have no state, so could not signal jenkins with *why* it had been rebuilt. So, Pantry Bell (https://github.com/lifeisstillgood/pantrybell). We simply run a webserver, that when called (perhaps from one Jenkins job) will force a re-imaging of a Rackspace server. This has a simple callback in rc.local (wget http://jenkins.frozone.mikadosoftware.com/pantrybell//`hostname`) So - a server forces a rebuild, on successful rebuild, the server calls home, and then the "real" jenkins job can be triggered using the Jenkins API calls. Amazingly enough it works, has trivial security, no REST compliance but it works

OpenStack and Rackspace

OpenStack and Rackspace

I suspect that if you find this as exciting as I do, one or both of us need to go out looking for a life. However, OpenStack is nearly here, and Connexions is in the Early Access Program. WooHoo.

OpenStack is in the frankly hilariously named Essex release, and is a collaboration between NASA (yes!) and Rackspace (and now 150 other companies) all of whom bascially want some of Jeff Bezos’ AWS lunch.

Now, I am currently trying to get Jenkins to nicely play with my cloud servers (see Pantry Bell) and it will use the pythonic nova Real Soon Now.

But the real pull is that we will be using rackspace to integrate all the different components involved in the new system, early as possible. The system currently builds the repository and the (old) editor, and I am working on bringing in the new Aloha editor and the webview asap. The sooner we see everything working together the sooner we shall find those great moments of code serendipity.

We have an opportunity as a bunch of Open Source developers to use the first real F/OSS cloud based offering and even influence a little part of its development. I am excited to see where it will go, and excited to see if it can provide measurable benefits to Connexions, either in development simplicity or sysadmin time savings. Interesting Times are ahead.

The Usual Suspects

Starting Blogging !

This is the first of hopefully a regular series of blogs from the developers behind Connexions.

Connexions

Connexions provides free (beer and speech) educational resources to students and teachers around the world. It has developed over ten years a eco-system of authoring tools, repository and means to serve the textbooks, exercises, snippets of knowledge to 2 million people a month.

And they are building a major update of that code.

And its all Open Source - and being developed right in front of your eyes. (Mail us if you want to join in.)

And thats what we are doing here ...

The people¶

(If your gravatar is not here, please let me know the right email address to use)

Jessica Burnett		Paul Brian		Ed Woodward
Philip Schatz		Michael Mulich		Marvin Reimer
Ross Reedstrom

Google Summer of Code Students

Debajyoti Datta		Saket Choudray		Alasdair Corbett
Yanchai Ye

The code:

https://github.com/Connexions

The plan

I am hoping we can link to the specs, and have the developers involved in each component post about their updates or changes on a regular basis.

We shall see, as the old man said.

Connexions Development Update 2-3-2012

Well, the dream of posting a weekly update is not working out.  Things are very busy here and it is a challenge to find the time to write this post.  I am committed to writing an update periodically so I'm going to modify my plan to make this bi-weekly.

CCAP

• The Physics rendering in Prince XML is complete except for some math issues.  The code is on the development server and should move to production soon with Sociology.
• Our current focus is on getting the Sociology rendering tested and released.  The current release date is Feb. 8th or 9th.
• There are several Math issues in the Physics book.  Some of the problems were caused by the Word importer, some by the original structure of the math in the Word documents and others are font issues.  When the Physics book is migrated to production, we will run a script to clean up some of the import problems to minimize the need for human intervention.  Some of the font issues will not be able to be resolved because the Stix fonts do not have the needed font.  An example is an italicized delta.

CNX Conference Preparation

• We have over 50 people signed up for the Sprints!  We are very excited about this.  As best we can tell, about 30 of these will be developers/coders/designers.  With this many people involved, we are going to make a huge effort to have clear easy install instructions for the various options.  Next week, we will be testing everything and finalizing the instructions.
• Once the instructions are complete, we will post the link on the Rhaptos list so you can install Rhaptos or anything else you might need prior to arriving at the conference.
• Many thanks to all that are planning to Sprint with us!

OERPub API

• We released the latest version of the OERPub API last week.  It will be used during the Conference Sprints by the sprinters working on content.

HTML Editor Discussion

• We have had a little time to continue our discussion in house regarding a new editor for Connexions.
• Most of the discussion has been Microdata vs Microformats.  Neither has much traction in the wild.  
• CSS3 seems to have support for Microdata which would allow us to use it to decorate semantic elements.  Microformats also have CSS support.
• There has also been discussion on which editor to select.  TinyMCE and Aloha have been our focus.
• TinyMCE has a nice UI, but does not support all of HTML5.
• Aloha supports all of HTML5, but has an odd UI.  The UI is not bad, just takes a little getting use to.  The demos on Aloha's site are very specific so we need to install it and play with the configurations of the UI.
• I'm sure the editor will be discussed at the conference so let us know your thoughts if you are attending. 

Connexions Dev Update for Jan. 20, 2012

Lots of info to share since last week's update was missed.


CCAP

• We tested the Physics book PDF generation and the Word Importer changes in QA last week.    The code was using FOP and Docbook to create more professional looking PDFs.  Our plan was to release this code and in a couple of weeks release the Prince XML version of the PDF generation.  The urgency was caused by the lack of harddrive space on the development server where the content is being created.  We have now fixed the space issue so it was decided not to release the FOP version of the PDF code.  We are focused on the new Prince XML version.
• Documentation - We have some new styles needed for the Sociology book to our documentation of the new markup.
• Coding is continuing on the PrinceXML version of the PDF.  You can see a sample of the Sociology layout at http://mountainbunker.org/~schatz/sociology.pdf.  It has Physics content, but the color scheme of Sociology.
• Some minor issues have been brought up by our content team regarding Math display in the Physics book.  Some of this will be fixed in the Prince XML code, but others will have to be fixed when we do the migration to production by tweaking the MathML generated by the Word Importer.

CNX Conference Preparation

• As the CNX Conference approaches, we are preparing for the sprints that will happen after the conference.
• The Bug List for the Sprint has been completed.
• Documentation for the Sprint has started.

Donate Button Change

• The donate button now has a minimum donation of $1.  The original minimum was $10. We hope this will lead to more donations.

iPhone App

• An Apple Developer Program for iOS was finally purchased last week.  It has taken months for all the documentation problems to get ironed out.

Consortium Tech Committee Meeting

• The Connexions Consortium Tech Committee held its monthly meeting last week.
• Discussions included current Connexions dev work, OERPub API update, Sprint topics and other member updates.  Check the minutes for more details.

OERPub API

• We are currently testing a new release of the OERPub API which uses Sword to add and modify Connexions content.
• The latest release allows users to add content to a Lens, add Featured Links to a module and to create a collection in a Workspace or Workgroup.  The code should be released early next week.
• The latest documentation for the API is at https://trac.rhaptos.org/trac/rhaptos/wiki/TechnicalDocumentation/Code/Sword

HTML Editor Discussion

• We have been talking within our team and on the Rhaptos list about the possibility of using HTML5 Microdata in the new WYSIWYG editor to document non-HTML elements in CNXML in the editor.
• HTML5 has some traction because of support from schema.org, but it also has problems such as very limited browser support.
• The thread on the Rhaptos list has more information.  

Weekly Dev Report for Jan. 6, 2012

As part of an effort to increase our transparency, the dev team at Connexions will be posting an update on the projects we are working on.  This is our first report and we will make a best effort to do an update every Friday.  We will not be detailing everything we are working on, just the highlights that should be of interest to the Connexions Community.  Feel free to comment here on the blog or on the Rhaptos Development mailing list.

This week the team returned to work after a Holiday break.  Our work included:

• CAP Code Testing - CAP (College Access Project) is a rebuilding of our PDF generation system to use Docbook instead of LaTeX.  We were testing the single column textbook style on a development server prior to our break.  This week, we moved the code to QA (merged to trunk and created eggs).  Testing should finish up early next week.  The code will be moved to production, but will not be used by any collections yet.  You can view a several chapter sample from the code here.
• CAP Double Column PDF - Prior to our Holiday break, we determined that Docbook cannot handle the 2 column layout that is required for some of the CAP books.  We reviewed a couple of commercial products (Antenna House and Prince XML) and decided to move to using Prince XML.  Prince XML converts HTML to PDF, so it is a new direction for our PDF generation.  Our plan is to create the 2 column PDF needed in Prince XML and then replace the 1 column Docbook PDF code with code that will work in Prince XML.  Coding of the 2 column PDF has begun.
• Design Discussions - We are planning on moving to a component based architecture in 2012 and are having daily discussions about how to procede.  Part of the complication of this plan has been the lack of an agreement with Google so we know how to move forward.  That situation has stabilized somewhat so our planning is moving forward in ernest.  As soon as we have some details to share, links will be posted on the Rhaptos list and in the weekly update.  We want and need feedback from the Connexions tech community on this project.
• Connexions Conference - We are planning on at least a 2 day sprint as part of the Conference.  A successful sprint requires planning and preparation so that developers can get what they need installed quickly and start coding.  We are testing our Rhaptos install along with Roché's Plone 4 install to make sure they work as needed.  We will also create some VirtualBox appliances so Mac and Windows users can get up and running quickly.  Those of us doing talks are doing slide preparation.  The conference starts Feb. 15.  You can see more details at the conference site.

Please let us know what you think of our first update.  We would like these to be informative, but also a quick and easy read.