inside the 'cube' - interview at hadoop world

Abhi Mehta on Hadoop as a Major Disruptive Force
Abhi Mehta talks about Hadoop as the most significant open source movement since Linux. Mehta puts forth a vision of how analytics and big data, powered by Hadoop is fundamentally and permanently changing businesses.

data factories - spoken word

Abhi Mehta on Big Data Factories
Abhi Mehta is a visionary technologist who visited SiliconAngle's John Furrier inside the Cube at Hadoop World 2010. He talks about the next industrial revolution that will be fueled by data that is stored, mined, cleaned and analyzed. These new data factories, contends Mehta will be the drivers of innovation in the 21st century.

new factories of the world – powering the next industrial revolution

I have been toying with this concept for a while…and finally decided to share it knowing fully the risk that entails as I am still toying with it. this may well be the best way for me to develop a concept I so strongly believe in that I have committed my career and many passions to it.

I would like to thank my friend mayank (@aster) for having so beautifully framed the concept as we were talking about it 5 months ago (march 30 2010 to be precise) and give him credit for coining the term – DATA FACTORY - as a consequence of our discussion.

Now for the concept.

We are witnessing the germination of a new industrial revolution powered by 'factories' of the future – data factories. The next economic renaissance will be fueled, funded and floated with the biggest asset ever known to man – data.

And I see the emergence of these new factories in almost every segment, geography and vertical. What these factories are doing (and will do) is propel the creation of business models – by disruption or innovation – that will fundamentally change the economics of capital allocation, management and returns like never before.

So how do I spot these factories and what are their defining characteristics…here is my first pass at putting that picture together.

Data factories are universally recognizable by the asset they all seek to commercialize – data. And use data as a key competitive differentiator. Their business models are all built (and juiced) around their ability to monetize data, in a variety of ways. While currently most of the commercialization opportunity has been fueled by advertising, other business models are slowly starting to emerge.

You will instantly recognize a lot of the 21^st century superstars have actually done just that. From Google, Amazon and Netflix to upstarts like Facebook, Twitter and Zynga. And Walmart, Goldman Sachs and P&G prove that it is not just web/media properties that have built successful companies fueled by data.

All these companies have made a BIG investment in and around data – the storage, management and analysis of it. And will reap the benefits for a long time to come.

In this series I will chronicle not just the evolution of this concept but also the key players driving it forward. I already have and will continue to spend time with these players and share with you what has enabled them to change the game…as well as outline the forces that shape the creation of these 'factories' and will continue to do so.

big data innovators – of grunge t’s and software

this Sunday i saw my friend wearing one of the coolest T's I have seen in a while. It had the tagline – DO IT IN PUBLIC!

Now you do not know my friend, but I do, and was quite surprised with his choice of garment. Till I looked at the back of the shirt and realized it was one from tableau software.

Now if you read my blog, you know I am a big fan of this company…simply because no other BI tool can do what they can on BIG DATA. And now having seen the company up close, met their people, seen the users and checked out their limitless production of the coolest company T's ever, I know these guys have fun with what they do. And it makes me an even bigger fan!

And as I promised Noele@ Tableau, here's my line for a shirt –

SIZE DOES (not) MATTER.

Yes, you can use it…I just get the first one that comes off the line!

big data innovators – betting against the open source community

As you will realize (preconditioned on becoming regular followers of this post) I like BOLD…to be bold automatically sets expectations for fiery exchanges, passionate interactions and cocksure personalities…perfect recipe for a fantastic afternoon

I met one such BOLD company yesterday who proclaimed that the open source community will not invest in improving Hadoop fundamentals with added features critical for large scale enterprise deployment…as it is not of interest to the community

I have to give them credit that they made this BOLD statement based on some assumptions that seem to be ‘well researched’:

1. Hadoop MR/HDFS today has some gaps (around fault tolerance, resiliency, HW optimization, other…for details please contact MapR)

2. Based on what was said at the yahoo hadoop summit (very smart guys at yahoo doing this, but they do not care less if Hadoop did not go much beyond their offices…can’t accuse them of not being focused) said at the Hadoop Summit last month, yahoo (still by far the biggest committer to the open source Hadoop project), is not interested in plugging those gaps

3. The Community will only work on things that excite them, not problems that need fixing for Hadoop to build a Linux like cult with enterprises

4. Large companies will experience challenges in implementing Hadoop unless features that are ‘must haves’ for any enterprise deployment are built

and as you may have guessed by now, MapR is building those very features.

having tried to build from scratch one of the largest Hadoop clusters in the world, this pitch surely caught my attention. not to mention the company’s CEO and CTO have a history of building successful startups..and have hired some smart guys (tomar is a star)…

the company is still very new (1 year anniv this month) and plans to have a beta out end of the year..with a GA in Q1…and claim to have a bevy of ‘customers’ (i don’t get it…in my book a customer is one who pays and uses your product…everyone else is a prospect)

so while I am not quite ready to drink their koolaid…I am not willing to bet against them either…

as a wise man once told me…the line between BOLD and stupid is a thin one!

ps – shout-out to my friend arif j at lightspeed for introducing me to these guys