Sharing knowledge
Things I've discovered or uncovered that have expanded my mind in some way.
Cisco Telepresence On-Stage
18 Apr. 2010
It certainly looks cool. I'm not sure how important this is until the quality is better. It looks a bit too much like a projector.
[ Permanent link ]
Rise of the Graffiti Robots!
13 Jan. 2010
Oh man! Robots perform graffiti by using a markup language (GML) designed to represent the graffiti lettering.
Amazing. All-encompassingly.
[ Permanent link ]
MongoDB
12 Jan. 2010
MongoDB is a scalable, high-performance, open source, schema-free, document-oriented database. Mike Dirolf of 10gen, the company behind MongoDB, will discuss the features that make it an interesting choice as a datastore for web applications.
Read more: http://pivotallabs.com/talks/85-mongodb
[ Permanent link ]
Economies, pools and piss
11 Jan. 2010
My career history
I have been challenged by some friends of mine about my career history with finance and, specifically, the involvement of Bear Stearns in the recent economic turmoil. I was challenged because I was a Vice-President at Bear Stearns at the time Bear collapsed. I have decided to write some thoughts on the topic because I don't want to repeat myself anymore.
I was a software developer and worked on a team building a new derivatives trading system. We were building a system to replace the previous trading system. I started as a developer working in one of the hotter parts of Bear's business. That's essentially what it was to me. I didn't know too much about derivatives themselves but knew a thing or two about Foreign Exchange from my previous job at Calypso Technology. Foreign Exchange generally seems harmless when compared to the infamous Credit Default Swap.
As I asked questions I would get quite reasonable answers. For example: What is a Credit Derivative?. The answer was: It's insurance taken out against people who might not make their bond payments. Hrm... sounds reasonable enough.
Tasks
My daily tasks were somewhat typical of any software developer. I worked on a team that built the middle-office reporting system. We built the report generation tools and the iterated on improving them. We reached some short-comings about the system and eventually built a read-only datastore that the entire firm could access. It was, simply, a SOAP system that would return a CSV with data relevant to the structure of the query. The query was essentially a filter into our stream of daily data. From there I built a testing system, along with a colleague, that could do advanced diffing into CSV's. It could compare CSV's by column or whether one value is less than 5% off the other value and so on...
I suspect my techie friends probably already have questions regarding how we did some of that, which brings me to my next point. It's very easy to lose yourself in an exciting and high-demand environment. Especially one as lean and agile as Bear was. If I wanted to build something that could be useful in two weeks, they'd just let me do it. It was amazing. I suspect many googlers feel this way or most people working at startups. But... please take the time to remember what all of this technology was supporting.
Dang... Kind of a bummer, right?
That is what I eventually walked away from. Each time I tell the story about leaving JPMorgan I can tell there is some uncertainty in the listener as to whether I voluntarily left or was booted. To the shock of my former team and the other teams that offered me positions inside JPMorgan, I just didn't want to be there. It just felt dirty.
I must confess one thing: I lost no money on Bear's collapse. While working there I devoted a fair bit of my time to learning how Bear's business worked. I couldn't properly understand what they were doing. The leverage levels, once I understood them, seemed insane. It wasn't quite as bad as LTCM, but it was bad. I'm not a financial expert but I listen to the advice of Warren Buffet and don't invest in anything I don't actually understand. Some people lost all of their life savings! In the context of capitalism, this is exactly what's supposed to happen, but it's a shame to see people so ruined. It is, however, fitting that some people at the top get ruined as badly as those at the bottom. Currently, this is heavily weighted towards just damaging those at the bottom. And they weren't even involved directly! Ugh...
Kool-Aid
It's hard to make it past the thick clouds of optimism, profiteering and heroism. Even Lloyd Blankfein has come out publicly saying he believes Goldman Sachs is doing God's work. Employees at investment banks, especially the leaders, do this internally just as much as externally. It's called drinking the kool-aid, as we've all heard, but no one ever comments on just how crappy drinking kool-aid actually is. It's basically sugar water. Hardly any substance at all in a typical glass. And did you know the term actually originated with a massacre that killed 918 people? Almost seems fitting that Wall St. would include such a term in it's vernacular.
During my plans to leave I met with a few groups interested in having me on their team. I was even offered a team of 10 people and frequent travel to London. It didn't carry much weight on me though. This is when some of the more Ayn Randian types came out to convince me of the value of our work. I'm not naive enough to think turning humanity against itself in almost every context will somehow yield good results. Wasn't John Nash's nobel prize awarded for something along the lines of saying groups function better when they take each other's interests into account? Anyway...
The thing about Capitalism is that it can turn into theft really easily if it's not checked. That's basically what has happened during the last thirty years. Our former, more-sensible system, was replaced with a climate ripe for disaster. The repeal of Glass-Stegall, the Government backing mortgages made by people who don't care about the quality of the mortgage, the lies from the credit agencies, the incredible greed that ignored the incredible risks... There isn't any single culprit, yet we are left with a new environment that has consolidated power towards many of the people who created this mess. On top of that, many of the folks involved have the audacity to hand out record breaking bonuses. Oh? The bonus isn't cash? It's in stock instead? That doesn't actually make much of a difference unless you think the stock they're receiving will be drying up soon. Frankly, I don't see Goldman, or their stock value, going anywhere...
Upsides of capitalism
Well, to start, it has lifted 2B people out of poverty in the last 30 years. Just look at what's happened in Asia or India. These two places suffered (and still suffer) from incredible poverty. But it's getting much better and doing so quickly! Partly because of capitalism and free markets. In fact, the investment opportunities over there are far more vast than here and we're seeing this in the draught of US investment compared to the still heavy investment over there. In a single generation, a family in the US can go from absolute poverty to the upper echelons of society. That happens here frequently. I personally know many people who've done this and I know many people who immigrated here specifically to try for themselves. How many other nations can you name where this happens as easily?
This system has also created TV's, cell phones, computers, a gigantic entertainment industry and all kinds of things that allow people to find ways to live successful or happy lives. There is a catch, what they do must be worthwhile enough for the rest of the world to pay for it. I think this is a great thing because we have so many like-minded people that I can probably do whatever I want and find other people who will pay for it. I'm fortunate that I love writing software. This would be harder for me if I was a painter. But even painters make wonderful careers for themselves or they can move out of the expensive areas and choose a life that matches their income. They're free to do that. Or they're free to choose a different career that pays more, if they want.
With all that said, let's not forget that the US Government invented the Internet. The Government can help when market forces have no interest. This was understood long before our time too. An example is the constitutional mandate for the Post Office to serve everyone instead of just the market. Or, it can provide balance when market forces have complete control and need to be restrained. Sure, sometimes regulation is bad or not well-considered, but it's rare to find a person that has problems with anti-trust laws. We just don't refer to those as regulation as often so it seems to get overlooked frequently.
The thing that is perhaps the best aspect of capitalism is it's relentless drive to be better. Consider it's approach to handling the distribution of capital. Kings used to hoard the stuff for as long as they could. Today, we have things like venture capital, angel investment, hedge funds, etc, that can make use of idle capital. They can lend it to other people in the hopes that those people figure out how to make more than they borrowed. I can borrow money from a bank or an individual or a group of individuals that believe in me and off I go! Hell, the US will even open up it's borders to foreigners who want to be productive. Got an idea for starting a company? Do it in the US! We'd love to have you. It might even be cheaper (meaning we could pay more people to work) if you stay put and do your work in your home country. That spreads the wealth across the entire globe!
But what about the people who don't have the necessary education? Well... the system can't help everyone and this is tricky topic. I consider myself a liberaltarian precisely because I value all the good points of capitalism and free markets and small government, but I understand the randomness involved with how people are placed and what that means for their advantages. The Government is our best chance at creating a fairer playing field. One where the best ideas and participants truly win, rather than just those who were born lucky. I was born lucky. Hungry kids in Africa were not. The nuances of covering those issues is better done over a beer.
Economies, pools and piss.
Economies are like a pool [1]. Sometimes people piss in them and you gotta clean 'em out a bit. It's a pain in the ass and most people don't like the idea of swimming in piss but no one notices until it's really pissy. The pool gets cleaned and everyone moves on. Some people remember how annoying that is and resolve never to piss in it again. Some are too young to remember. Maybe the pool will even last beyond the generations that were there when everything turned yellow.
We still have to worry about younger generations that still think it's funny or satisfying to pee now rather than later. A tradition of not pissing in the pool might help. It's certainly tempting for people to seek the satisfaction of release right then and there. Especially (!!) with the high probability of not being noticed. The main concern is probably just hiding the hot piss in the cold water quickly.
The thing that's awesome about pools is that in spite of the fact they're tricky to maintain, you still get a lot of fun and enjoyment out of them. I love pools. I can swim on my own or I can get a group together to play a game or maybe make a team to compete in games of some kind. I definitely wouldn't want any piss in my pool if I'm trying to concentrate on getting my speed up. In fact, I want a rule against pissing in the pool!
Hmm... How the do you even detect that? I've heard about chemicals that can turn the pool water a different color when it detects pee. That might lead to a few years or torture for a young kid who didn't know any better... Pool pisser!!
In the end, if the pool is too pissy, you just gotta walk away and find a new place to hang.
[ Permanent link ]
NYC is perfect for social startups.
07 Jan. 2010
While watching Alli Mooney and Meghan Keane on Good Morning Interwebs, I felt something I've felt a lot lately. Encouraged by the NYC start up community. First Round Capital has been good, and frequent, about promoting their interest in NYC lately. They were at NYTM Tuesday night too. Charlie O’Donnel from First Round Opens this morning's Good Morning Interwebs and describes his interest in NYC
Next, they interview Justin Shaffer of Hot Potato. Hot Potato is a service that captures all the media of a particular event. When someone is signing in, what they have to say about it, what media has been captured so far, or after. They do this via mobile tools and offer a clean web experience too. Please tweet anything you might have to say about Hot Potato with a #hotpotato. Trust me, they want to hear what you have to say. You can help design how events should work with mobile devices!
While this interview was taking place it occurred to me how much I believe in NYC as a place for successful startups. GMI's crazy eyes contest is an example of the type of thinking people are using for viral promotion. I suspect it was on-the-spot, which makes it even better. Simply, It's encouraging people who are watching to take part in an event that symbolizes a partnership in marketing. It's positive for Hot Potato because it promotes the tool and it's a positive for Good Morning Interwebs because everyone is coming together for a GMI reason and they use the service that needs promotion as the mechanism.
This idea is so simple and clever that I see a future for the TV media outlets. The icons of the old media must crumble, it's just the way of things. But I now see what will replace it. We have a merger of Tech Crunch and The Today Show. And, it's in New York City!
[ Permanent link ]
Peter Norvig: What to demand from a Scientific Computing Language
20 Dec. 2009
I have posted on the wikitrans blog about this :: http://wikitrans.github.com/2009/12/20/peter-norvig-scientific-computing.html
[ Permanent link ]
The Cult of Statistical Significance
16 Dec. 2009
The book opens by exploring two distinct sides of statistics and then argues that too much science uses only one side. For example, he says statistics are a sophisticated counting tool which lets you answer two questions: 1) whether or not something exists and, 2) how much. With both of these, you can determine how much of an impact something has. If you only ask the first question, you are going to be chasing false positives. It is vital to understand how to determine if something is strongly true. A degree of magnitude is considered, instead of the black and white; yes or no.
I have made examples like this before that I found useful. Consider a regular six-sided die. Am I going to roll a 1? Yes or no. Choose one or choose the other. It’s just one out of two. But wait… There’s five other faces. The chances of rolling a 1 are 1 out of 6.
In a political context, to only ask whether or not something exists is to completely negate any compromise of ideas because it forces a dichotomy on the thinker.
Do you support miscellaneous issue that probably doesn’t actually affect you?
YES or NO?
[ Permanent link ]
Johnathan Zittrain: Minds for Sale
07 Dec. 2009
I wrote my thoughts on Zittrain's lecture to Stanford about the cloud and it's ability to put minds up for sale.
The post is on the wikitrans blog: here.
[ Permanent link ]
Python, Amazon Mechanical Turk and Boto
03 Dec. 2009
I have recently become a commiter for the Python Amazon Web Services (AWS) module named boto. This module is hosted on Google Code. I am pretty stoked to help build out the Mechanical Turk portions of the software. It appears to cover everything in AWS but the MTurk pieces aren't complete.
I have written about my experience using MTurk from python, and eventually from boto, on the wikitrans blog.
[ Permanent link ]
Django South
17 Nov. 2009
Django-NYC South Demo from Sean OConnor on Vimeo.
South is a database schema migration tool for Django. Some have claimed to like it so much they would like to see it in Django core.
[ Permanent link ]
Richard Feynman, the Challenger Disaster, and Software Engineering
16 Nov. 2009
This blog entry from Gustavo Duarte is a neat comparison of bottom-up software design principles and Richard Feynman's problem-solving approaches.
I am a big fan of the bottom-up software design principle. Solve each problem properly and snowball your code into a grander system.
[ Permanent link ]
Outsourcing reviews with Mechanical Turk
15 Nov. 2009
People appear to be gaming review systems through Mechanical Turk. Here is a specific instance: someone is asking to buy positive reviews on vpgames.com in this shot.
Just after a glance, vpgames.com doesn't seem like the kinda place I'd buy games from anyway, but it's interesting to find the marketplace on Amazon. This particular Human Intelligence Task (HIT) is reachable here :: https://www.mturk.com/mturk/preview?groupId=4YJM16ZMR0V2XPZ0NWKZ
[ Permanent link ]
Shazam not magic after all
29 Oct. 2009
The Revolutions Blog has a fascinating post regarding the mathematical genius of Shazam, an iPhone app that listens to music and identifies it after about 10 seconds of listening. The app actually works too!
I was in Baltimore for a meeting with the people at Johns Hopkins when I asked Glen Coppersmith how this could be done. We were at an art festival where Cake happened to playing, so I pulled out my iPhone and showed him how the app worked. I turned it on, not knowing if it'd actually identify Cake's live performance, but sure enough it did and Glen and I were both impressed and immediately consumed with questions on how they're pulling this off so well. We first identified the problems in designing such a tool.
- We have a ten second window to get all of our information.
- The tool can match different versions of a particular song (eg. live or album)
- It needs to find a match very quickly once that window of information is captured and sent to the server.
It would be too painful to simply send along the ten second window so some compression would have to occur. Not musical compression (ala wav=>mp3) but some kind of reduction in information more abstract. I first considered counting beats per minute, but beats are not always so clearly defined, especially in small windows of time. Glen then thought if we could capture the melody somehow and send that, we could iterate over a database. This seemed quite reasonable.
But what can we capture? The melody itself won't be enough. The melody over a section of time might do it, but that seems like a lot of information to store on the backend. And the query has to be quick or users will remove the app thinking it doesn't work. We felt we needed to be able to reduce the melody down to just it's key points but had no idea how this could be done.
Turns out we were on the right track. The team at Shazam had done almost exactly that. They built a spectrograph of the loudest notes over time. It's precisely a reduction of the melody into it's core identifying characteristics! Fascinating!
Here is the article from Revolution :: Shazam not magic after all
[ Permanent link ]
Thinking and business opportunity
06 Oct. 2009
Miguel de Icaza is discussing a battle between him and Richard Stallman in a blog entry and quotes a story which goes like this
Two shoe salesmen were sent to Africa in the early 1900's to scout the territory. One telegraphed back: "Situation hopeless. Stop. No one wears shoes." The other telegraphed: "Business opportunity. Stop. They have no shoes."
A great story for highlighting the right way to think with regard to opportunity.
[ Permanent link ]
Subscribing to Google Groups without a Gmail address
30 Sep. 2009
I use my j2labs.net email address for everything related to development, including my subscriptions to mailing lists. Google Groups, unfortunately, makes it a bit tricky to subscribe without a Gmail address.
...but I have a solution! Just modify the URL Google uses a little bit to feed it the email address and group you are interested in.
http://groups.google.com/group/<GROUPNAME>/boxsubscribe?email=<EMAIL_ADDRESS>
[ Permanent link ]
Linus Torvalds on Git
27 Sep. 2009
At around 3 minutes he rip on subversion pretty hard. As a fan of git and fellow hater of CVS and SVN, I quite liked hearing him do it!
From Linus's mouth, I see Subversion as being the most pointless project ever started. Ouch!
[ Permanent link ]