Executive Summary Reprint: R1210E Senior leaders who write off the move toward big data as a lot of big talk are making, well, a big mistake. So argue McKinsey’s Barton and Court, who worked with dozens of companies to figure out how to translate advanced analytics into nuts-and-bolts practices that affect daily operations on the front lines. The authors offer a useful guide for leaders and managers who want to take a deliberative approach to big data—but who also want to get started now. First, companies must identify the right data for their business, seek to acquire the information creatively from diverse sources, and secure the necessary IT support. Second, they need to build analytics models that are tightly focused on improving performance, making the models only as complex as business goals demand. Third, and most important, companies must transform their capabilities and culture so that the analytical results can be implemented from the C-suite to the front lines. That means developing simple tools that everyone in the organization can understand and teaching people why the data really matter.

• Advanced Analytics Definition
• What Is Advanced Data Analytics

Embracing big data is as much about changing mind-sets as it is about crunching numbers. Executed with the right care and flexibility, this cultural shift could have payoffs that are, well, bigger than you expect. Artwork: Tamar Cohen, The Big Quick, 2010, silk screen collage on vintage book pages, 40″ x 50″ Big data and analytics have rocketed to the top of the corporate agenda. Executives look with admiration at how Google, Amazon, and others have eclipsed competitors with powerful new business models that derive from an ability to exploit data. They also see that big data is attracting serious investment from technology leaders such as IBM and Hewlett-Packard. Meanwhile, the tide of private-equity and venture-capital investments in big data continues to swell. The trend is generating plenty of hype, but we believe that senior leaders are right to pay attention.

Big data could transform the way companies do business, delivering the kind of performance gains last seen in the 1990s, when organizations redesigned their core processes. As data-driven strategies take hold, they will become an increasingly important point of competitive differentiation. According to research by Andrew McAfee and Erik Brynjolfsson, of MIT, companies that inject big data and analytics into their operations show productivity rates and profitability that are 5% to 6% higher than those of their peers (see “Big Data: The Management Revolution” in this issue).

Even so, our experience reveals that most companies are unsure how to proceed. Leaders are understandably leery of making substantial investments in big data and advanced analytics. They’re convinced that their organizations simply aren’t ready. After all, companies may not fully understand the data they already have, or perhaps they’ve lost piles of money on data-warehousing programs that never meshed with business processes, or maybe their current analytics programs are too complicated or don’t yield insights that can be put to use. Or all of the above.

No wonder skepticism abounds. Many CEOs, too, recall their experiences with customer relationship management in the mid-1990s, when new CRM software products often prompted great enthusiasm. Experts descended on boardrooms promising impressive results if new IT systems were built to collect massive amounts of customer data. It didn’t turn out that way.

Too many C-suites were blind to the practical implications of new CRM technologies—namely, that to capitalize on them, organizations would have to make complex process changes and build employees’ skills. The promised gains in performance were often slow in coming, because the systems remained stubbornly disconnected from how companies and frontline managers actually made decisions, and new demands for data management added complexity to operations. To be fair, most companies eventually managed to get their CRM programs on track, but not before some had suffered sizable losses and several CRM champions had lost career momentum. Given this history, we empathize with executives who are cautious about big data.

Nevertheless, we believe that the time has come to define a pragmatic approach to big data and advanced analytics—one tightly focused on how to use the data to make better decisions. In our work with dozens of companies in six data-rich industries, we have found that fully exploiting data and analytics requires three mutually supportive capabilities. (See the exhibit “How to Benefit from Big Data.”) First, companies must be able to identify, combine, and manage multiple sources of data. Second, they need the capability to build advanced analytics models for predicting and optimizing outcomes. Third, and most critical, management must possess the muscle to transform the organization so that the data and models actually yield better decisions. Two important features underpin those activities: a clear strategy for how to use data and analytics to compete, and deployment of the right technology architecture and capabilities. How to Benefit from Big Data.

To improve performance with advanced analytics, companies need to develop strengths in three areas. Multiple Data Sources Creatively source internal and external data.

Upgrade IT architecture and infrastructure for easy merging of data. Prediction and Optimization Models Focus on the biggest drivers of performance. Build models that balance complexity with ease of use. Organizational Transformation Create simple, understandable tools for people on the front lines. Update processes and develop capabilities to enable tool use. Equally important, the desired business impact must drive an integrated approach to data sourcing, model building, and organizational transformation.

Advanced Analytics Definition

That’s how you avoid the common trap of starting with the data and simply asking what it can do for you. Leaders should invest sufficient time and energy in aligning managers across the organization in support of the mission. A shipping firm improved on-time performance of its fleet by tapping data on weather and port availability that it hadn’t realized were available. Choose the Right Data The universe of data and modeling has changed vastly over the past few years. The sheer volume of information, particularly from new sources such as social media and machine sensors, is growing rapidly. The opportunity to expand insights by combining data is also accelerating, as more-powerful, less costly software abounds and information can be accessed from almost anywhere at any time. Bigger and better data give companies both more-panoramic and more-granular views of their business environment.

The ability to see what was previously invisible improves operations, customer experiences, and strategy. But mastering that environment means upping your game, finding deliberate and creative ways to identify usable data you already have, and exploring surprising sources of information. Source data creatively. Often companies already have the data they need to tackle business problems, but managers simply don’t know how the information can be used for key decisions.

Operations executives, for instance, might not grasp the potential value of the daily or hourly factory and customer-service data they possess. Companies can impel a more comprehensive look at information sources by being specific about business problems they want to solve or opportunities they hope to exploit. For example, a banking team that needed to improve the efficiency of its customer-service operations created a 360-degree view by combining information from ATM transactions, online queries, customer complaints, and so on. That allowed duplicative interactions to be identified, thereby reducing costs and streamlining the customer experience. Managers also need to get creative about the potential of external and new sources of data.

Social media are generating terabytes of nontraditional, unstructured data in the form of conversations, photos, and video. Add to that the streams of data flowing in from sensors, monitoring processes, and external sources that range from local demographics to weather forecasts. One way to prompt broader thinking about potential data is to ask, “What decisions could we make if we had all the information we need?” Using that logic, one shipping company improved the on-time performance of its fleet by tapping specialized weather forecast data and live information about port availability that it hadn’t realized were available.

Senior executives can take the lead here. The CEO of one major packaged-goods company told us that he views data as a strategic asset whose value he takes into account when assessing potential acquisitions. But leaders at all levels must also be attuned to novel approaches to gathering and husbanding information. As business practices in the internet era continue to evolve, inspiration can often arise from a scan of the external environment. A corporate finance executive, for instance, might look to a company such as Kabbage, a start-up that supplies working capital to online businesses. To slash the time required to underwrite loans, Kabbage asks merchants to opt in to sharing their customer-feedback ratings, Facebook interactions, and electronic shipping records.

Those with the strongest feedback and highest business volume receive greater financing. Get the necessary IT support. Legacy IT structures may hinder new types of data sourcing, storage, and analysis. Existing IT architecture may prevent the integration of siloed information, and managing unstructured data often remains beyond traditional IT capabilities. Many legacy systems were built to deliver data in batches, so they can’t furnish continuous flows of information for real-time decisions. Fully resolving these issues often takes years. However, business leaders can address short-term big data needs by working with CIOs to prioritize requirements.

This means quickly identifying and connecting the most important data for use in analytics, followed by a cleanup operation to synchronize and merge overlapping data and then to work around missing information. Such short-term tactics may lead companies to vendors that focus on analytics services or emerging software.

New cloud-based technologies may also offer ways to scale computing power up or down to meet big data demands cost-effectively. Together those approaches establish an IT infrastructure that propels innovation by facilitating collaboration, rapid analysis, and experimentation. Build Models That Predict and Optimize Business Outcomes Data are essential, but performance improvements and competitive advantage arise from analytics models that allow managers to predict and optimize outcomes. More important, the most effective approach to building a model rarely starts with the data; instead it originates with identifying the business opportunity and determining how the model can improve performance. Unfortunately, not all model building follows this course. One approach that gets inconsistent results, for instance, is simple data mining. Corralling huge data sets allows companies to run dozens of statistical tests to identify submerged patterns, but that provides little benefit if managers can’t effectively use the correlations to enhance business performance.

A pure data-mining approach often leads to an endless search for what the data really say. One company followed a more targeted strategy to optimize complex product pricing. At its core was a model based on the historical price elasticity of its products, sales data, competitors’ responses, and other variables.

To improve its chances of success, the company began the modeling process by positing which factors affected sales volumes (for instance, competitors’ pricing and promotions) and then asked what data and which model would best deliver insights that were useful for making business decisions. We have found that such hypothesis-led modeling generates faster outcomes and also roots models in practical data relationships that are more broadly understood by managers. Remember, too, that any modeling exercise has inherent risk. Although advanced statistical methods indisputably make for better models, statistics experts sometimes design models that are too complex to be practical.

For example, a predictive model with 30 variables may explain historical data with high accuracy, but managing so many variables will exhaust most organizations’ capabilities. Companies should repeatedly ask, “What’s the least complex model that would improve our performance?” 3.

Transform Your Company’s Capabilities The lead concern expressed to us by senior executives is that their managers don’t understand or trust big data–based models. One large retailer intended its model to optimize returns on advertising spending, but despite considerable investment, it wasn’t being used. The reason soon became evident: The frontline marketers who made key decisions on ad spending didn’t believe the model’s results and had little familiarity with how it worked. Many companies grapple with such problems, often because of a mismatch between the organization’s existing culture and capabilities and the emerging tactics to exploit analytics successfully. In short, the new approaches don’t align with how companies actually arrive at decisions, or they fail to provide a clear blueprint for realizing business goals. Tools seem to be designed for experts in modeling rather than for people on the front lines, and few managers find the models engaging enough to champion their use—a key failing if companies want the new methods to permeate the organization. Bottom line: Using big data requires thoughtful organizational change, and three areas of action can get you there.

Develop business-relevant analytics that can be put to use. Like early CRM misadventures, many initial implementations of big data and analytics fail simply because they aren’t in sync with the company’s day-to-day processes and decision-making norms.

The aforementioned case of a company that aimed to optimize prices illustrates how to avoid those common pain points. The company started with an analytics task force that convened a series of meetings with pricing and promotions managers to better understand the types of decisions they made when setting prices—and how those choices ultimately affected revenue and customer retention. Model designers also inquired about the types of business judgments that managers make to align their actions with broader company goals. These conversations ensured that both pricing analytics and resulting scenario tools would complement existing decision processes. The modeling allowed the company to reach its ultimate goal: more-effective management of price and volume trade-offs as product launches proliferated. Embed analytics into simple tools for the front lines. Managers need transparent methods for using the new models and algorithms on a daily basis.

By necessity, terabytes of data and sophisticated modeling are required to sharpen marketing, risk management, and operations. The key is to separate the statistics experts and software developers from the managers who use the data-driven insights. One large industrial company, for instance, sought to better forecast workforce needs to reflect local market variations. Historically, as the company had tried to keep labor costs low, it had often found itself short-staffed in some markets, leading to significant overtime costs and service snafus.

To remedy the problem, the company convened a small working group of analysts and IT programmers who developed a series of predictive models that forecast workforce availability on the basis of factors such as vacation time, absenteeism, and work rules in labor contracts. The models incorporated millions of new data points on thousands of employees across dozens of locations. But rather than providing managers with reams of data and complex models, they created a simple visual interface that highlighted projected workforce needs and necessary actions. Ultimately, that approach of using a simple tool to deliver complex analytics substantially improved workforce planning and reduced the need for new hires and overtime. Develop capabilities to exploit big data. Even with simple and usable models, most organizations will need to upgrade their analytical skills and literacy. Managers must come to view analytics as central to solving problems and identifying opportunities—to make it part of the fabric of daily operations.

Efforts will vary depending on a company’s goals and desired time line. Adult learners often benefit from a “field and forum” approach, whereby they participate in real-world, analytics-based workplace decisions that allow them to learn by doing. Using a simple tool to deliver complex analytics substantially improved workforce planning and reduced the need for new hires and overtime.

At one industrial services company, the mission was to get basic analytics tools into the hands of its roughly 200 sales managers. Training began with an in-field assignment to read a brief document and collect basic facts about the market. Next managers met in centralized, collaborative training sessions during which they figured out how to use the tools and market facts to improve sales performance. They then returned to the field to apply what they had learned and, several weeks later, reconvened to review progress, receive coaching, and learn about second-order analysis of their data.

This process enabled a four-person team to eventually build capabilities across the entire sales management organization. Adjusting culture and mind-sets typically requires a multifaceted approach that includes training, role modeling by leaders, and incentives and metrics to reinforce behavior.

One large consumer-products company applied such an approach successfully. It created a sophisticated program to improve the profitability of promotional spending with its retailers.

What Is Advanced Data Analytics

The launch included training—led by company management—and a new promotions-analysis tool for sales representatives. However, after an initial whirlwind of activity, the program and use of the tool fizzled. The obstacle was that company incentives and reporting protocols for sales managers tracked sales and sales growth, not profits. As a result, the managers considered the profit-focused program to be bureaucratic overhead that was unrelated to their key sales goals. After a series of discussions with the managers, the company relaunched the program, offered new incentives for improving profits, and tailored reports to profit-related data. Although ongoing training and coaching was necessary, the efforts gradually produced a shift in mind-set such that the power of promotions analytics is now used to further the common goal of increasing profitability.

The era of big data is evolving rapidly, and our experience suggests that most companies should act now. But rather than undertaking massive overhauls of their companies, executives should concentrate on targeted efforts to source data, build models, and transform the organizational culture.

Best video converter for mac. Such efforts will play a part in maintaining flexibility. That nimbleness is essential, given that the information itself—along with the technology for managing and analyzing it—will continue to grow and change, yielding a constant stream of opportunities. As more companies learn the core skills of using big data, building superior capabilities may soon become a decisive competitive asset.

Data analytics is increasingly important for businesses looking to uncover insights that might be hidden in a vast sea of data. Organizations can gain a tremendously valuable perspective on their customers and business objectives using tools that are designed to organize, categorize and infer statistical conclusions from various sources of data. Keep up to date with the and beware these. Bolster your career with our. Get the latest on data analytics. Enterprises have many considerations to weigh and choices to make when evaluating data analytics tools, but finding the right application and using its features effectively can lead to dramatic transformation. We’ve reviewed dozens of providers to identify the best free data analytics software available today.

After weighing their strengths and limitations, studying reviews by industry leaders and analyzing rankings from various research firms, we’ve selected the following seven tools (presented in alphabetical order) to help you find the solution that's best for you. DataMelt, also referred to as DMelt, is a computational platform for statistical analysis of large data and scientific visualization. The program is most frequently used in natural sciences, engineering, and modeling and analysis of financial markets. The platform supports many programming languages including Python, BeanShell, Groovy, Ruby, Java and others. Organizations can access vast libraries via dynamic scripting, including more than 40,000 Java classes for computation and visualization and 500 Python modules. More advanced features require a developer or commercial license, but the free edition of DataMelt includes many of the key features required to explore, analyze and visualize data. DataMelt runs on Windows, Linux, macOS and Android devices.

The KNIME Analytics Platform is designed to help organizations manipulate, analyze and model data through visual programming. The software includes more than 1,000 modules, hundreds of ready-to-run examples and a range of integrated tools to help users discover potential insights hidden in their data and predict futures with the aid of machine learning. Instead of writing code, KNIME enables organizations to drag and drop connection points between activities. The data analysis tool also supports data blending among simple text files, databases, documents, images, networks and Hadoop-based data in a single visual workflow. KNIME Analytics Platform is open source and is updated with new releases on a bi-annual basis. KNIME is available for Windows, macOS and Linux devices.

OpenRefine, formerly Google Refine, helps organizations get a handle on messy data. Google stopped supporting the project in 2012, but the application is still available and is updated regularly by volunteers. OpenRefine can perform various tasks on data, including cleaning, transforming and formatting data to make it more suitable for data analysis and exploration. The tool also enables users to fetch data from external web services to reconcile and match data from various sources. OpenRefine is not the best tool for vast databases, but it remains an important and well regarded option for many organizations because of the significant amount of time that analysts spend cleaning data for predictive modeling. OpenRefine is available for download on Windows, macOS and Linux. Orange is an open source data analysis and visualization tool developed at the University of Ljubljana in Slovenia.

Users can mine data via visual programming or Python scripting in a terminal window; explore statistical distributions, box plots or scatter plots; and dive deeper into their data with decision trees, hierarchical clustering, heatmaps and linear projections. Orange's graphical user interface enables users to focus on exploratory data analysis instead of coding.

The tool also has components for machine learning and add-ons that extend the functionality of data mining from external sources to perform natural language processing, text mining, bioinformatics, network analysis and association rules mining. Orange supports Windows, macOS and Linux.

Tableau Public is a data analysis and visualization application that enables users to publish interactive data to the web. The free version of Tableau is limited to 1 GB of data storage and 1 million rows of data. The simplicity and intuitiveness of Tableau Public has made it one of the most popular data analysis tools. Tableau Public can mine data from Google Sheets, Microsoft Excel, CSV files, JSON files, statistical files, spatial files, web data connectors and OData. Users can generate interactive charts, graphs and maps to be shared on social media or embedded on sites for public availability. Tableau Public is available for download on Windows and macOS.

Trifacta Wrangler is another app designed to help data analysts clean and prepare messy data from diverse sources. Once datasets are imported to Trifacta Wrangler, the app will automatically organize and structure the data. Machine learning algorithms help prepare data for more detailed analysis by suggesting common transformations and aggregations.

Trifacta Wrangler can import data from Microsoft Excel, JSON files and raw CSV files. The tool also profiles data to indicate what percent of rows have missing, mismatching or inconsistent values, and visually categorizes data by type, such as the date or time, string or IP address associated with each data point. Trifacta Wrangler is limited to 100 MB of data and is available for download on Windows and macOS devices.