Wednesday, December 15, 2010

Accelerating Innovation with Apache Extras

Google and the Apache Software Foundation just launched Apache Extras.

Apache Extras is a home for communities and open source projects related to Apache technologies, which cannot be hosted by the Apache Software Foundation because they are unable or unwilling to use the Apache v2 license or the Apache collaborative  consensus based processes (a.k.a. The Apache Way).

This new channel will help build collaborative communities around Apache Open Source related technologies. Apache Extras provides the infrastructure services typically required by open source projects, such as code repositories, bug tracking, and project web sites/wiki. It is hosted by Google Code Project Hosting, so it will be very familiar to developers already using Google Code Project Hosting.

The announce is here, more on the Google code blog, my friend Luciano's blog, the Apache Software Foundation blog, the Apache Extras management home, the FAQ, guidelines and home page.

Wednesday, November 24, 2010

Ski Season Opening!

Tahoe got a lot of snow in the last few days and the ski resorts are now open.

I already went to Squaw Valley on Saturday for the opening day. Only one lift was open but it was enough for my daugther and I to have a great time skiing there. The drive back was a little long as we had a big snow storm but it was worth it.

Going back to Squaw Valley again tomorrow morning with my kids and a friend and hoping for a great day as the  snow accumulation is already about 9ft at 8200ft!

and the weather forecast looks really good too!

4:30am leaving San Carlos... ouch that's early :)
5:00am pick up a friend in San Francisco
6:30am pick up my daugther in Davis
8:30am breakfast in Squaw Valley
9:00am on the slopes!

Wednesday, November 17, 2010

JSON vs XML, structured vs semi-structured data

That debate has been going for years now... When should you use JSON vs XML?

Interesting blog posts on this today from Norm Walsh and Noah Mendelsohn.

Both agree that:
"XML wasn't designed to solve the problem of transmitting structured bundles of atomic values. XML was designed to solve the problem of unstructured data. In a word or two: mixed content."

Noah then adds:
"If you want to send along a list of job applicants and their recent salaries, JSON does fine;  if you want to send their resumes, well JSON isn’t quite as helpful.   A surprising amount of the world’s important information is in just such semi-structured documents.  Think insurance policies, shop manuals, and even Web pages themselves."

+1 to that! I've worked -- and struggled sometimes :) -- with XML for years, and more recently with JSON, and couldn't agree more!

Adding my 2c:
When you work with structured business data, use an object (or struct, or record) model in your favorite programming language, and an object serialization encoding like JSON.

When you work with semi-structured document resources, use a document API or a document centric programming language (like XQuery, XSLT) or even better, a document-node centric programming language (like XLinq), and an XML document encoding.

You'll need both programming approaches. Don't use a single approach for everything, unless you're looking for trouble.

When you need to communicate both structured and semi-structured data, don't bundle them. Just link these resources using URI references.

Simple enough?

Tuesday, November 16, 2010

Full support for Git at Apache

The Apache infrastructure team is starting to work on full support for Git at Apache. Woo-hoo!!

Many contributors already use the Apache Git mirrors and git-svn, but this will take Git usage at Apache to the next level.

Discussions are happening on the Apache infrastructure-dev mailing list there and JIRA issues have been opened to track the initial tasks there.

Issue INFRA-317 will be particularly interesting, I think... as it's about describing the basic Git workflows that will be used by Apache committers.

Git is a distributed version control system which makes it very easy for individual contributors to work in isolation on local private repositories for some time, then publish and merge their work with others later.

That's different from the traditional Apache way, where contributors from a project community collaborate more closely around a centralized (Subversion) repository.

So, I think it'll be interesting to see how these distributed and centralized workflows can be reconciled, to allow people to use Git for everything it's good at, while still following the Apache way.

Monday, November 15, 2010

My VI configuration file

After a few years using Eclipse, I've gone back to VI. I actually use VIM, an extended version of the original VI.

I won't expand on the reasons, but I just find it simpler, faster, and less distracting than a graphical IDE like Eclipse.

I've experimented a bit and found that I'm able to better focus on writing code with VI in a single barebones 24x80 window, instead of spending time and energy clicking and moving around outline views and windows.

I use exuberant-ctags to navigate big codebases, and Google search as a richer replacement for code-assist.

After I used VI to show and write code in my ApacheCon demos, a few folks asked me for my VI configuration supporting syntax highlighting, auto-indenting and code navigation.

Here's my $HOME/.vimrc file. Nothing fancy, but I find it useful:

" vi-improved mode
set nocompatible

" cycle buffers without writing
set hidden

" backup while writing
set writebackup

" file completion
set wildmenu
set wildmode=list:longest

" update window title
set title

" display cursor location
set ruler

" display current command
set showcmd

" short message prompts
set shortmess=atI

" silent
set noerrorbells

" switch to current file's directory
set autochdir

" remember marks, registers, searches, buffer list
set viminfo='20,<50,s10,h,%

" keep a big history
set history=1000

" syntax highligting
syntax on

" auto smart code indent
set autoindent
filetype indent on
set smartindent
set smarttab
set tabstop=4
set softtabstop=4
set expandtab
set shiftwidth=4
set shiftround

" allow backspacing in insert mode
set backspace=indent,eol,start

" incremental search
set incsearch
set nohlsearch

" ignore case
set ignorecase
set smartcase

" restore last cursor position
function! RESTORE_CURSOR()
    if line("'\"") > 0 && line ("'\"") <= line("$")
        exe "normal! g'\""
autocmd BufReadPost * call RESTORE_CURSOR()

" navigate code using exuberant ctags
set tags=tags;$HOME
set tags+=$HOME/.vim/tags/cpp
set tags+=$HOME/.vim/tags/curl
set tags+=$HOME/.vim/tags/httpd
set tags+=$HOME/.vim/tags/libmozjs
set tags+=$HOME/.vim/tags/libxml2

Hope this helps.

Friday, November 5, 2010

Tuesday, October 19, 2010

JavaOne 2010 - Composite Apps for the Cloud with Apache Tuscany

Time flies! Almost a month has passed since JavaOne... I wanted to post my JavaOne 2010 presentation earlier, but have been busy with work... and a nice vacation in France.

Luciano Resende and I presented our experience with various cloud infrastructures and platforms, the challenges we're seeing, and how SCA composition and the Apache Tuscany and Apache Nuvem projects can help.

Here's the presentation abstract:

Today's cloud environments pose new challenges for application developers: hiding cloud infrastructure from business logic, assembling components on heterogeneous and distributed cloud environments, and optimizing the provisioning of the required cloud resources.
This session will demonstrate how to use Apache Tuscany and the Service Component Architecture (SCA) to develop, build, and run an application composed of several service components in a distributed cloud environment.
We'll illustrate how to encapsulate cloud infrastructure services as SCA components to simplify the construction and assembly of the application and how to move components around and rewire the application to adjust to new business and cloud deployment conditions.

... and the slides:

The session went pretty well. The room was full, the demos worked (see slides 22 and 27 for diagrams of what we demo'ed). There were interesting questions and a good discussion after the presentation.

Wednesday, August 18, 2010

New book: Tuscany SCA in Action available

A new book by Simon Laws, Mark Combellack, Raymond Feng, Haleh Mahbod and Simon Nash, Tuscany SCA in Action provides a simple step-by-step guide on how to develop applications leveraging SCA (Service Component Architecture) and Apache Tuscany.

InfoQ has just published an interview of the authors with interesting questions and answers on SCA and an excerpt of the book.

There's a few things I really like about it:
  • The authors are the founders of the Apache Tuscany project and have been involved in SCA from the beginning, so they know what they're talking about.
  • The book uses a lot of examples, available in the Tuscany project, easy to download and try as you read.
  • The examples are actually part of an official Tuscany release, in sync with the Tuscany v1.6 release (and I'm hoping that they'll be ported later to the evolving Tuscany v2.x release stream).
I think this book is a very good introduction to SCA and its implementation in Apache Tuscany. The printed book is not available yet, but an ebook version is already available there.

Thursday, May 13, 2010

Apache Libcloud v0.3.1 released

The Apache Software Foundation and the Apache Libcloud project have announced the release of version 0.3.1 of Apache Libcloud. Apache Libcloud is a pure Python client library for interacting with many of the popular cloud server providers.

Apache Libcloud v0.3.1 is available for download from the project download page as well as PyPi.

I've been playing around with Libcloud to experiment and deploy SCA apps running on the Tuscany native (aka Tuscany-cpp) and Tuscany Java runtimes to EC2, and it's been working pretty well for me. The Libcloud API is very simple and makes it really easy to write Python scripts that automate the deployment, start, stop etc of compute nodes running SCA apps.

If I find some free time this weekend, I'll try to clean up a bit the Libcloud-based deployment scripts I've been playing with, and add them to the Tuscany native sample apps.

Also, I think it should be relatively easy to hook similar scripts with the Tuscany SCA domain manager Web app, which currently only supports local nodes but could be extended to deploy SCA components to remote nodes in the cloud.

Tuesday, May 4, 2010

The Security Exchange Commission, Python, Excel and Functional Programming

What does the SEC have to to do with Python, Excel and Functional Programming? Well, the title of this blog is Random Thoughts, so here are a few random thoughts.

Two weeks ago the US SEC Security and Exchange Commission proposed significant revisions to the rules governing the offering process, disclosure and reporting for ABS (Asset Backed Securities).

One of the proposals is to require ABS issuers to file Python computer programs describing the flow of funds (called waterfall) in ABS transactions. I think that's pretty cool!

That's cool for investors

Waterfall provisions in modern securitizations and CDOs (Collateralized Debt Obligations) are complicated. They're described in 'prospectus' documents written in plain english. That's verbose, error prone, and makes it difficult for investors to create credit and cash flow models and conduct their own evaluations of ABS. The last financial crisis revealed that many investors relied on ratings provided by third party credit rating agencies instead of doing their own evaluation homework, and were not fully aware of the risk in the underlying mortgages within the pools of securitized assets.

Providing a Python program modeling the waterfall for each ABS would help investors conduct their own evaluations and make more informed investment decisions. The Python program source code would be filed on EDGAR (Electronic Data-Gathering, Analysis, and Retrieval). Investors could just download the program for an ABS, input their data assumptions and model the waterfall right away.

That's cool for Python and programming

I've dissected a few ABS prospectus documents and I'm starting to think that these Python programs could look just like Excel spreadsheets (which most investors are familiar with), with user defined formulas written as Python functions, in a functional programming style.

Monday, April 19, 2010

Functional Programming, a perfect fit for Web Programming?

The last few months I've become really interested in Functional Programming. I've learned Haskell, Scheme, and F#. I've also looked at how to do Functional Programming in Imperative Programming languages like Python, Java, C and C++.

I've been doing that as a hobby on evenings and weekends, as I'm only using Java at work. Evenings and weekends have added up and I've actually done quite a bit of real functional programming in Scheme and F# now, as well as some functional 'style' programming in Python and C++ in weekend contributions to the Apache Tuscany C++ SCA project.

I'm only starting to digest that Functional Programming experience.

I've also done Web programming for 15 years and have been lucky to work on projects using so many Web frameworks and languages that I can't remember all of them... Well, I still remember when I almost gave up on programming altogether, out of frustration as I was trying to do something like... AJAX in 1996... and Internet Explorer and Netscape were driving me crazy and crashing all the time :). We didn't have a fancy name like AJAX at the time, just dynamic HTML and Javascript.

I'm not sure I've completely digested Web programming yet, but I've got over a number of good and bad, or let's say 'learning', experiences.

Last week I was bored and I started to think about similarities between Functional Programming and Web Programming. I thought about a few fun analogies:

Wednesday, April 14, 2010

GCC 4.5 released! now supports Lambda Expressions

GCC 4.5 just got released! with support for lambda expressions (see first class functions also), as part of some experimental support for C++0x (the upcoming ISO C++ standard.)

There are many other changes in GCC 4.5 but this new support for lambda expressions is great news for all the Linux developers out there interested in functional programming!

Tuesday, April 13, 2010

Experimenting with Databases

I've been playing around with various Open-Source databases for a little weekend database project.

Last weekend was really rainy. April usually feels like summer here, but this year is an el Nino year so we're getting a lot of rain... I shouldn't complain though, as the rain turns into great powder in Tahoe and we've had a great ski season!!

But I didn't go skiing last weekend... For my database project I had two buckets of data to work with:

Monday, April 5, 2010

Apache Vysper XMPP Server first release

The Apache MINA project has announced the very first release of their XMPP server product Apache Vysper.

Apache Vysper 0.5 is easy to install, easily extensible and embeddable. It is based, of course, on MINA.

You can find the Vysper release here:

Wednesday, March 31, 2010

Google Summer of Code 2010 now accepting student applications

The Google Summer of Code 2010 program is now accepting student applications!

Google Summer of Code is a global program to introduce students, ages 18 and over, to the wonderful world of Open Source development. GSoC offers student developers stipends to write code for various Open Source projects. Students can choose from 150 Free and Open Source software projects, in technical areas as diverse as gaming to humanitarian efforts to operating system design.

More info on the Google open-source blog and the GSoC home page.

Monday, March 29, 2010

CERN Large Hadron Collider ready for record particle beam collisions

"We are opening the door to New Physics, to a new period of discovery in the history of humankind," said Rolf-Dieter Heuer, director-general of CERN, the European Organization for Nuclear Research. 

Pretty exciting news from the Swiss / French border! In a few hours, high energy particle beams will start circulating in opposite directions around the CERN Large Hadron Collider (LHC), causing particle collisions that could help unlock secrets of the  dark matter, dark energy and what happened in the universe 13.7 billion years ago... 

Sunday, March 28, 2010

Binding to port 80 without running as root

I've been looking for a way to allow server programs like Apache httpd, memcached, Couchdb, Qpid etc to bind to privileged IP ports under 1024 like 80 or 443 for example, without running them as the root user. Normally, on Linux you need to run as root to be allowed to bind to these standard privileged ports

The postings on this site are my own and don’t necessarily represent positions, strategies or opinions of my employer IBM.