Planet Sourcesense

As I briefly mentioned in this post I founded a company called OSWay – The Open Source Way – back in 2000 with a friend of mine, exactly 10 years ago!

A bit of history: in 1997 I went live with my first ever website on Geocities, it was called C++Warriors and it was all about…C++ News, articles and tutorials were in Italian and that was a scarse resource at the time so much so that it got featured on a few magazine (the physical ones!) and got quite some traction. That’s when I met all kinds of interesting people like Alex Martelli of Python and Google fame (at the time he was able to spit out a 30-part tutorial on Win32 programming in a matter of days) and Ugo Landini to name a few.

In 1998 I started thinking about the success of C++Warriors and decided that if a little amateurish website about C++ was so successful it was due to the sheer lack of programming resources in Italian and I started planning what would then become Programmazione.it in 1999 (although I have nothing to do with it since 2003 it still exists and it’s 11 years old now!): the first Italian website about programming with news, articles, tutorials and soft dev product reviews all in Italian.

To do that I needed to find people willing to write and publish material in Italian about as many programming languages as possible and that’s when I met Filippo on the #programmazione IRCNet channel in 1998 who will then become my business partner in both Programmazione.it and OSWay (I now regret a lot letting osway.com, .net, .org and .it expire a few years ago )

At the end of 1999 Filippo and I also started working together for a few customers and came up with a product idea around “making information free”, sharing and collaborating on-line, etc, etc (all pretty common stuff nowadays), founded OSWay S.r.l., sold part of it to a public company to get some capital and invest it into the product development. We eventually froze the product and kept working as a software house specialised in the use of Open Source. We did all sorts of things from partnering with SuSe Italy, to developing the world first Kylix enterprise-grade POS application (there used to be our case history on Borland‘s website before the CodeGear split), community websites in Java and PHP (Freestation.it), Linux-based embedded software for touch-screen, industrial-rugged appliances and more.

We started working on the product a few months before incorporating but we eventually incorporated in September 2000, exactly 10 years ago!!

There are many noteworthy things about that venture, including the fact that Open Source was at the center of everything we did, starting from the company very own name 3 years before the Open Source Initiative adopted bylaws and applied for recognition as 501(c)3 nonprofit in 2003 but I just wanted to celebrate the 10th aniversary with a post and this is it

UPDATE: Filippo dug out the flash teaser that we prepared in 2000 for OSWay and the product. Yes, it’s flash and yes it’s heavy but it’s still beautiful!

Warning: there is sound and at the end it tries to redirect you to a page that clearly doesn’t exist anymore….

If you were wondering why I wasn’t updating this blog very often (or at all) during the last few months, I have an excuse: I was too busy writing a book!

Finally, Alfresco 3 Web Services, Build Alfresco applications using Web Services, WebScripts and CMIS has been published and I just received my complimentary author copies. You can order it, either as a PDF or as a dead-tree version, from the publisher’s website, or from Amazon.com.

This is the book you need if you are considering developing applications that need to use the services of the Alfresco Open Source ECM system, and it covers everything, from SOAP-based Web Services, to Web Scripts and REST, to CMIS. If you want to get a sample of the book’s content, you can download a sample chapter (Chapter 5: Using the Alfresco Web Services from .NET ) freely.

I have a couple copies that I could sign and give away. I will have to think of something, like a contest, who knows?

I would also like to thank my co-author and colleague, Piergiorgio Lucidi, for accepting to join me in this crazy enterprise and working hard to make it finally see the light of day.

While I'm happy with BoxGrinder, I don't use it on a daily basis and keep forgetting the trivial instructions; so instead of stressing the project mantainers with the same questions I'll write the instructions here as a reference for myself and a tutorial for whoever might be interested.

What is BoxGrinder?

It's a Ruby tool which creates virtual machines from simple definition files; these Appliance Definition files are cloud-vendor and virtualization tecnology neutral, so you define your appliance once and with different plugins you have them running on different virtualization platforms. So while I'm experimenting my builds with Amazon's EC2, I know I'll be able to provide the same appliance to VMWare systems, VirtualBox, KVM, even create bootable USB keys.

It currently supports Fedora, Red Hat, CentOS operating systems and built images can be run on several targets; it also uploads the built image, in my case using EC2 it will register the new AMI for me.

For a complete introduction, there's a nice video on the official website.

Following instructions are collected from the official documentation which you can find here, I only cherry-picked what I'm interested in for my own goals, so make sure to look at the full documentation in case you need more options or target different virtualization providers.

Meta-Appliance

A meta-appliance is an appliance containing almost all what we need to build other appliances; so instead of installing all of the needed software locally, you can grab one from EC2's AMI catalogue and use it. I like this option as you'll end up having to upload your AMI to S3, so if you assemble it directly on Amazon's systems this process will be way quicker and spare your bandwith.

When selecting a meta-appliance, make sure you select the appropriate architecture: as of BoxGrinder 0.5.1 you need a 64bit meta-appliance to build 64bit AMIs, or a 32bit meta-appliance to build a 23bit AMI (this last limitation will likely be resolved soon as BGBUILD-46).

There we go:

start an instance of ami-96db30ff and connect with your private key using SSH.

Then update the system, and install some tools:

yum update

rpm -Uvh http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.noarch.rpm

yum install createrepo subversion

gem install boxgrinder-build boxgrinder-build-ec2-platform-plugin boxgrinder-build-s3-delivery-plugin boxgrinder-build-fedora-os-plugin

Ec2-ami-tools is a requirement to bundle the AMIs on S3, createrepo is useful to integrate own-built RPMs, and finally I use subversion to manage the appliance definition files.

Now create some directories and the local RPM repositories:

mkdir -p /opt/repo/RPMS/i386 /opt/repo/RPMS/noarch /opt/repo/RPMS/x86_64 ~/.boxgrinder/plugins

createrepo /opt/repo/RPMS/i386 && createrepo /opt/repo/RPMS/x86_64 && createrepo /opt/repo/RPMS/noarch

Enter S3 credentials in the plugin configuration to be able to upload your AMI:

vi ~/.boxgrinder/plugins/s3

The configuration looks like this, just omitting the passwords:

access_key: xx

secret_access_key: xxx

bucket: scarlet-private-amis

account_number: 3441-4397-4060

cert_file: /root/cert.pem

key_file: /root/soseaws.pem

You have to upload the cert.pem and soseaws.pem files, these are your private keys for accessing all of AWS services.
Now get the appliance definitions:

svn co https://dev.sourcesense.com/repos/dev/scarlet/trunk/appliance-build/appliances appliances --no-auth-cache --username s.grinovero

And create an AMI:

boxgrinder-build appliances/common-scarlet.appl -p ec2 -d ami

This will end as:

I, [2010-08-26T09:14:52.070877 #18565] INFO -- : Bundling AMI...
I, [2010-08-26T09:16:44.546519 #18565] INFO -- : Bundling AMI finished.
I, [2010-08-26T09:16:44.547023 #18565] INFO -- : Uploading common-scarlet AMI to bucket 'scarlet-private-amis'...
I, [2010-08-26T09:17:43.906981 #18565] INFO -- : Image successfully registered under id: ami-723ed41b.

Done! Now take a note of the registered AMI id and use it to start N-copies of it.

Appliance Definition files
Again, to see all the options read the Stormgrind documentation, anyway an appliance definition looks like:

Example definition

name: common-scarlet

summary: Base AMI definition common to all scarlet AMIs

version: 2

release: 1

os:

name: fedora

version: "13"

password: xxx

hardware:

cpus: 2

memory: 512

partitions:

size: 3

packages:

includes:

- bash

- yum

- vim-minimal

- openssh-server

- chkconfig

- acpid

- dhclient

- openssh-clients

- mc

- subversion

- mutt

- rsync

- s3cmd

repos:

- name: "local-noarch"

baseurl: "file:///opt/repo/RPMS/noarch"

ephemeral: true

- name: "local-#ARCH#"

baseurl: "file:///opt/repo/RPMS/#ARCH#"

ephemeral: true

Note the pointers to the local RPM repository, there you can add RPM packages which you want to be included in the built appliance, in my case my own application which is not available in the default Fedora repositories.

Hudson Definition
to have a nice Fedora 13 with all latest updates running latest Hudson, just have BoxGrinder grind this definition:

name: hudson-scarlet
summary: Hudson instance to run builds of Scarlet
version: 2
release: 1
os:
  name: fedora
  version: 13
  password: yyy
hardware:
  cpus: 2
  memory: 512
  partitions:
   /:
   size: 3
   /var/lib/hudson/:
   size: 50
appliances:
  - common-scarlet
packages:
  includes:
   - hudson
   - java-1.6.0-openjdk
repos:
  - name: "hudson-repo"
   baseurl: "http://hudson-ci.org/redhat/"
post:
  base:
   - "/sbin/chkconfig --level 345 hudson on"

Note the inheritance to the appliance of the previous example: this appliance will contain also the parent's packages, then an additional repository is enabled to download Hudson and a large partitions is dedicated to it.

Finally, a post-build script is started which will make sure Hudson is started, you won't even need to login to the machine to configure it.

Planet Sourcesense

September 02, 2010

Marco Abis

OSWay 10 years later

Ugo Cei

Alfresco 3 Web Services Book

August 26, 2010

Sanne Grinovero

Using BoxGrinder meta-appliance to create custom EC2 AMIs

Subscriptions

Planetarium: