Everyone knows that Google is the founder the of the MapReduce framework, they even protected it by patent signed at June 2004. Google have been using MR for probably more then 10 years now, gaining a huge competitive and technology edge on all relevant competitors in terms of product, costs and scale.
At the end of 2004, Google introduced a whitepaper: MapReduce: Simplified Data Processing on Large Clusters that changed the “big data” industry as we know it. This paper initiated the foundation of Hadoop, built by a Doug Cutting.
Although Google was the true founder of the concept - today when you say MapReduce, you say Hadoop. Not any more. Google is working to re-gain it’s position as the leader of big data processing technology, also for their cloud services platform – Google App Engine.
At Google’s IO 2010, Google introduced it’s strategy for parallel processing in GAE, revealing the Mapper phase.
One year after, at Google IO 2011 - we got the second phase of the puzzle (still in early stages) – the full MapReduce mechanism.
What fascinate me the most is the Google message to the world - as I see it, GAE up until now, was considered to be “just” an alternative for developing simple web sites, a competitor to GoDaddy and php if you will. You’ve got Python or Java, a NoSQL datastore, simple scalability - and you’re ready to go.
Batch processing was always a mystery on GAE, and thus, they probably missed the opportunities for really big stuff to be built on their infrastructure.
Their latest announcement puts Google in a new position, they are telling us - go a head, we can take it. Go and build your super complicated applications on our platform – social networks, advertising companies, web crawlers, video processors, everything.
The giants are wrestling and the industry is winning; The race for building the richest cloud platform is getting to finals and it’s tight - AWS, Azure, GAE and Force are leading.
The race for 1B$ company built entirely on cloud services is getting to into a critical phase as well.