A commercial open source project in Python

Developing a Commercial Open Source Projectwith Python

OSDC 2007, Brisbane

Juergen BrendelPrincipal Software Engineer

Slide 2

Developing a Commercial Open Source Project with PythonOSDC 2007, Brisbane

Agenda

• The SnapLogic project• Open source opportunities / challenges• Python opportunities / challenges

Slide 3


The SnapLogic project

• Started in 2005• Open source data integration framework• REST• Targeted at developers

Slide 4


Example: Browser based GUI

Slide 5


Example: Resources

SnapLogic Server

ComponentHTTP

Resource Definition

Databases

Files

Applications

Atom / RSS

HTTP://server1.example.com/customer_list

Client HTTP Request and Response

• Resource Name• HTTP://server1.example.com/customer_list• Component and Behavior• SQL Query or filename• credentials• parameters

Slide 6


Example: Resources

SnapLogic Server

ComponentHTTP

Resource Definition

Databases

Files

Applications

Atom / RSS

HTTP://server1.example.com/processed_customer_list

Client HTTP Request and Response

Component

Resource Definition

Component

Resource Definition

Read Geocode Sort

Slide 7


Why open source?

• Practical reasons• Philosophical reasons• Business reasons

Slide 8


Business reason: Lower cost

Traditional sales funnel

Potential customersPotential customers

InterestedInterested

QualifiedQualified

OpportunityOpportunity

SaleSale

Slide 9



Open source sales funnel

Potential customersPotential customers

InterestedInterested

QualifiedQualified

OpportunityOpportunity

SaleSale

Slide 10



Traditional FOSS

Sales cost

Development cost

Income

Slide 11


Business reason: Broad adoption

• Broad adoption– More 'sales'– More feedback– Contributions– Lower development cost

• OSS gets in through the side door

Slide 12


Challenge: Little OSS experience

• The open source community• What's public, what's private?• You are being watched!

• Solution:– Clarify policies– Embrace openness

Slide 13


Challenge: Licenses

• Which license to choose?• Modified standard license?

• Solution:– Get professional advice– Keep it simple– Use GPL

Slide 14


Challenge: Source as a feature

• People choose OSS because of source• Source becomes a feature (!) of the product

• Solution: Make source beautiful– Documentation– Consistency– Style and documentation guide

Slide 15


Challenge: Enabling contributions

• Open APIs more important than open source!

• Enable 'deeper' contributions

• Solution:– Don't rest on your laurels– Use your unique advantage: Deep contributions– Repeat after me: “APIs, modules, APIs, modules,

APIs, modules, APIs, modules...”

Slide 16


Why Python?

• One of the founders likes it• Rapid prototyping and development• Reasonably fast• Readable code• Suitable even for large projects

Slide 17


Challenge: Little experience

• Can't hire Python developers• Even in the Silicon Valley

• Solution:– Just hire good (generic) developers– They can learn

Slide 18


Challenge: Tainted by past life

• Team experience in other languages• Not using Python to its full potential

• Solution:– Understand pythonic– Read: Python is not Java– Share knowledge: Code reviews

Slide 19


Challenge: Getting carried away

• Powerful and very (!) dynamic:

• Solution:– Create conventional looking APIs– Use those APIs as documentation

>>> class Foo(object):>>> def hello(self):>>> print “Hello World!”

>>> x = Foo()>>> dir(x)[ .......... , 'hello']>>> x.bar = 123>>> dir(x)[ .......... , 'bar', 'hello']

Slide 20


Challenge: Why didn't you just use...?

• Python comes with Batteries included• Don't know all the libraries have to offer• Write yourself? Oh, the temptation!

• Solution:– Learn to use Google– Read Python Cookbook and others– Share knowledge: Code reviews

Slide 21


Challenge: Third-party package risks 1

• Import chain madness• Some packages rely on C extensions• Normally: Don't reinvent the wheel

• Solution:– Watch what you import– Inventing a little wheel can be helpful

Slide 22


Challenge: Third-party package risks 2

• So far, no bugs in core libraries• Bugs in third-party packages• Included patched packages in the installer

• Solution:– Speed up acceptance time for submitted patches– Provide patch, test cases, detailed docs

Slide 23


Challenge: The GIL

• Can't use multiple cores with threads

• Solution 1:– Use multi-processing

• Solution 2:– mod_wsgi

Slide 24


Challenge: Hidden bugs

• No compiler to catch typos• Duck typing

• Solution:– Unit testing: pyunit, buildbot– Coverage testing: figleaf, buildbot

Slide 25


The end

Any questions?

[email protected]

Date post:	19-May-2015
Category:	Technology
Upload:	jbrendel
View:	2,651 times
Download:	1 times

A commercial open source project in Python

Technology