Putting the "BI" in Big Data
Last week, at the PASS (Professional Association for SQL Server) Summit in Seattle, Microsoft held a coming out party, not only for SQL Server 2012 (formerly “Denali”), but also for the company’s “Big Data” initiative. Microsoft’s banner headline announcement: it is developing of a version of Apache Hadoop that will run on Windows Server and Windows Azure. Hadoop is the open source implementation of Google’s proprietary MapReduce parallel computation engine and environment, and it's used (quite widely now) in the processing of streams of data that go well beyond even the largest enterprise data sets in size. Whether it’s sensor, clickstream, social media, location-based or other data that is generated and collected in large gobs, Hadoop is often on the scene in the service of processing and analyzing it.
Microsoft’s Hadoop release will be a bona fide contribution to the venerable open source project. It will be built in conjunction with Hortonworks, a company with an appropriately elephant-themed name (“Hadoop” was the name the toy elephant of its inventor’s son) and strong Yahoo-Hadoop pedigree. Even before PASS, Microsoft had announced Hadoop connectors for its SQL Server Parallel Data Warehouse Edition (SQL PDW) appliance. But last week Microsoft announced things that would make Hadoop its own – in more ways than one.
Yes, Hadoop will run natively on Windows and integrate with PDW. But Microsoft will also make available an ODBC driver for Hive, the data warehousing front-end for Hadoop developed by FaceBook. What’s the big deal about an ODBC driver? The combination of that driver and Hive will allow PowerPivot and SQL Server Analysis Services (in its new “Tabular mode”) to connect to Hadoop and query it freely. And that, in turn, will allow any Analysis Services front end, including PowerView (until last week known by its “Crescent” code name), to perform enterprise-quality analysis and data visualization on Hadoop data. Not only is that useful, but it’s even a bit radical.
As powerful as Hadoop is, it’s more of a computer scientist’s or academically-trained analyst’s tool than it is an enterprise analytics product. Hadoop tends to deal in data that is less formally schematized than an enterprise’s transactional data, and Hadoop itself is controlled through programming code rather than anything that looks like it was designed for business unit personnel. Hadoop data is often more “raw” and “wild” than data typically fed to data warehouse and OLAP (Online Analytical Processing) systems. Likewise, Hadoop practitioners have had to be a bit wild too, producing analytical output perhaps a bit more raw than what business users are accustomed to.
But assuming Microsoft makes good on its announcements (and I have pretty specific knowledge that indicates it will), then business users will be able to get at big data, on-premise and in-cloud, and will be able to do so using Excel, PowerPivot, and other tools that they already know, like and with which they are productive.
Microsoft’s Big Data announcements show that Redmond’s BI (Business Intelligence) team keeps on moving. They’re building great products, and they’re doing so in a way that makes powerful technology accessible by a wide commercial audience. For the last seven years, SQL Server’s biggest innovations have been on the BI side of the product. This shows no sign of stopping any time soon, especially since Microsoft saw fit to promote Amir Netz, the engineering brain trust behind Microsoft BI since its inception, to Technical Fellow. This distinction is well-deserved by Mr. Netz and its bestowal is a move well-played by Microsoft.
Last week’s announcements aren’t about just Big Data; they’re about Big BI, now open for Big Business.
Posted by Andrew J. Brust on 10/17/2011