Techaisle Blog

In a parody of Start Trek, Silicon Valley technology companies describe their business goal as “Scale, the final frontier…”.  Mid-market companies, defined as those having 100-2500 employees, may indeed provide an opportunity to emerging technology vendors to scale their business. According to Techaisle, a market research firm, these 800,000 global companies spend $300B on IT and are sought after by technology vendors big and small. In the last decade, technologies such as Cloud, SAAS and Virtualization have reached scale with a large number of mid-market companies as early adopters. Intuit, Salesforce.com, NetSuite and Amazon are just a few examples of companies who have relied upon mid-market companies as a key building block for their business.

What does this mean for Big Data? To find out, Carpe Datum Rx spoke to “SMB Guru”, Anurag Agrawal, CEO of Techaisle and the former Head of Worldwide Research Operations at the Gartner Group. Techaisle recently talked to 3,300 global businesses about their Big Data adoption plans. Here is an excerpt from our discussion.

The SMB Market is considered the Holy Grail for technology vendors because it is hard to penetrate. Does your research show that mid-market companies will adopt Big Data before large enterprises do? Are they the early adopters of this technology?

Yes, you are right the SMB Market is the Holy Grail as it is hard to penetrate but with the highest potential. To elaborate, there are slightly over 70 million small businesses and 800,000 mid-market businesses worldwide. They constitute over 97 percent of the business segment. And their collective IT spend is projected to grow by 6.5% between 2013 and 2016 which is quite a lot faster than the Enterprise segment. To really identify the SMB segments and their type of technology spend is a mind-numbing exercise due to the sheer volume of data points. This is compared to the enterprise segment where there are fewer companies and larger dollar amounts being spent.

To answer your second question about whether mid-market businesses will adopt big data before large enterprises, let us look at some facts. Cloud computing started as an enterprise play, however, it was quickly discovered that SMBs would be the more relevant target segment with a faster path to adoption. Similarly, as enterprises adopted Virtualization, vendors shifted their focus to the SMBs with some very creative solutions. Mid-market companies, defined as those with 100 to 2500 employees could certainly be the early adopters of Big Data. We recently did a study where we surveyed 3,360 mid-market businesses worldwide covering all regions – North America, Europe, Asia/Pacific and Latin America. What we found is that the promise of superior data-driven decision making is motivating 43 percent of global mid-market businesses to at least look at Big Data technology. And above all, 18 percent of mid-market businesses are now investing in big data related projects.

In the mid-market segment, there is also a competitive imperative to understand customers, create innovate products and improve operational efficiencies. They are not burdened with too many silos and large legacy systems deployments. The absence of large legacy systems is an important point to consider because it makes mid-market businesses more agile to implement new types of solutions that solve their business problems. It is expected that in year 2016, global SMBs would spend US$3.6 Billion on big data solutions exhibiting a growth rate that is faster than what was exhibited by cloud computing solutions.

We understand that you cast a very wide net to get your 43% number. Is there a consistency in the sentiment on big data across different parts of the world? 

Yes, we had to cast a wide net to really understand the adoption and trends within mid-market businesses. And yes, there is a difference across geographies and employee sizes. North America has both the largest market and the highest level of adoption in Big Data overall. In terms of actual deployment activity, the market grows in relation to the size of the companies. Additionally, mid-market business attitude towards Big Data transitions from “Over-Hype” to must-have technology with the increase in employee size. Let me give you some examples. A small-to-mid-sized bank is developing a Proof of Concept for fraud analytics. Another example is of a small advertising agency that is trying to deploy digital advertising analytics. So big data is not only within the radar of enterprises, the same problems exist across all sizes of business, only the volume of data, available budget and the required simplicity varies. The problem is that we all get caught up in technology which instills a sense of fear. We have to shift the conversation from technology to solving business problems.

Big Data adoption is often stalled by a lack of knowledge or understanding of the technology and its capabilities. Do mid-market companies have a better understanding of this technology than large enterprises? Do they have an advantage over large enterprises in implementing effective solutions?

You are right. Three things – Technology, Resources and Data are the biggest roadblocks for big data project implementations within mid-market businesses. In recent years technology and technology options have evolved extremely rapidly for an average business to understand, evaluate, purchase and implement. Big data is no different. Mid-market businesses consider big data as very complex resulting in very steep learning curves. The complexity gets further exacerbated with lack of experience, lack of skilled manpower and innate difficulty in identifying external consultants who would be the right fit for their big data business objectives and budget availability. In spite of challenges, the study shows that there have been some successes when business units, IT & data analysts exhibit extraordinary alignment.

Our study shows that mid-market businesses typically start their big data journey in one of four ways and the highest success rates have been achieved when IT and data analysts work with external consultants from project inception. It is still very early days for these businesses to fully embrace big data but the seeds are being planted. And we believe that these businesses may very well race ahead of enterprises with their deployments as technology becomes simpler and consultants become experienced. As we like to say it, SMBs could be the path to big data simplicity.

You talk about the linking of structured and unstructured data. Why is this problem so important compared to all the others? 

The issue of analyzing data from diverse sources leads a mid-market business to naturally consider linking structured and unstructured data. If we look back, CRM solutions had first established the need for analyzing customer data. However, the data was mostly two-way transactional structured data. This changed when customers began visiting business websites to explore, browse and perhaps make purchases thus leaving behind a trail of information. And everything changed with the onset of social media, blogs, forums, wikis and opinion platforms where the identification of false positives and negatives became difficult and knowledge about the customer and resulting segmentation became an inaccurate undertaking. Big data analytics presents the possibilities of connecting together a variety of data sets from disconnected sources to produce business insights for generating sales, improving products or detecting fraud. Thus the importance of linking structured and unstructured data to analyze social media data, web data, customer and sales data along with click-stream machine generated data and even communications data in the form of emails, chat, and voice mails. But extremely limited expertise creates a major challenge. If they can figure it out, one-fourth of mid-market businesses say that they will use big data as an integral part of their overall analytics efforts. The possibility of analyzing a variety of data producing action-driven business insights is too big to ignore for mid-market businesses.

How are big data projects getting started globally? Are they championed by LOB managers? Are they getting adequate support from executive management? Are customers demanding it?

The study reveals that the initiators are marketing, finance or operations and the ultimate user of the analytics is the business user. Big data requires a new type of alignment between business heads, namely, marketing and finance (main drivers of big data projects), IT and a completely new set of players known as data scientists or data analysts. As I mentioned before, once the decision is made mid-market businesses show an extraordinary alignment across departments. Our study shows that mid-market businesses typically started their big data journey in one of four ways. However, the highest success rate was achieved when an external consultant or organization was brought in to develop proof of concept, advise on database architecture and ultimately develop the big data analytics solution right from the moment of project inception.

What is one piece of advice or Carpe Datum prescription can you share for our members?

You have adopted cloud, you have adopted mobility, you have adopted social media so do not be afraid to develop Big Data analytics proof of concepts. Do not ignore big data just because of perceived complexity and big data solution providers’ inability to create bite-sized messaging that directly address pain-points. Do not forget that business intelligence has now become one of the fastest solutions to be adopted by SMBs and mid-market businesses. If done right, big data will address three key pain points: Increased sales, More Efficient operations, Improved Customer service.

Various organizations define Big Data differently. Some use “petabytes of data” as a benchmark to isolate big data from other normalized and structured data sets that exist within an organization. However, this measure quickly boxes big data analytics into the large enterprise market segment. Small and mid-market businesses certainly do not have this extent of data but Big Data still relevant for them. In fact Big Data solutions are more relevant for Small and Mid-Market businesses. However, it will take some creativity on the part of solution providers to make Big Data accessible, easy to use and comprehend for segment that constitutes 97 percent of global businesses.

Cloud computing started as an enterprise play, however, it was quickly discovered that SMBs will be the more relevant target segment with a faster path to adoption. Similarly, as Virtualization market started getting fully penetrated within the enterprises, vendors shifted their focus to the SMBs with some very creative solutions. As far as big data is concerned SMBs are starting to show interest and even adoption. However, there is a stark difference in approaches between mid-market businesses and small businesses. While mid-market businesses are experimenting with bespoke solutions, small businesses are gravitating towards a multi-tenant, aggregated and federated big data solution that has a mix of publicly available data and their own internal data.

It is expected that in year 2016, global SMBs would spend US$1.6 Billion on big data solutions exhibiting a growth rate that is faster than what was exhibited by cloud computing solutions. Cumulatively between now and end of 2016, SMBs itself would have shelled out US$3.9 billion on big data solutions. This spending includes hardware, software and services.

So why are many big data solution providers ignoring SMBs? Simply put, because of perceived complexity and inability to create bite-sized messaging that directly address SMBs pain-points. But they should not forget that business intelligence has now become one of the fastest solutions to be adopted by SMBs. If done right, Big data address three key pain points of SMBs: Increase sales, Efficient operations, Improve Customer service.

Promise of Superior Decision Making

Let us take Techaisle’s recent global mid-market businesses’ Big Data Adoption & Trends study which clearly shows that the promise of superior data-driven decision making is motivating 43 percent of global mid-market businesses to either invest in or investigate Big Data technology. Out of these, 18 percent of mid-market businesses are actively investing in big data related projects. The possibilities of analyzing a variety of data sources, producing action-driven business insights is too big to ignore for these businesses.

Similar to cloud, the attitude towards Big Data is transitioning from “Over-Hype” to “Must-Have” technology with the size of business. Even within the businesses that consider big data to be over-hyped, 29 percent think that it will be an important part of their business decision making process in the future.

Extracting Business Perspectives

Business intelligence by itself has provided enough business insights, however, mid-market businesses are now looking for extracting business perspectives to drive superior decisions and ultimately achieve superior results.  Extracting business perspectives has become important as they rethink their marketing strategies because mobility, social media, and other transactional services have increased the number avenues for connections with their customers and partners.

CRM solutions had first established the analytics for analyzing customer data. However, the data was mostly two-way transactional data. This changed when customers began visiting business websites to explore, browse and perhaps make purchases thus leaving behind a trail of information. IT vendors and mid-market businesses figured out the need to analyze the data and combine it with transactional information.

However, everything changed with the onset of social media, blogs, forums, wikis and opinion platforms where the identification of false positives and negatives became difficult and knowledge about the customer and resulting segmentation became an inaccurate undertaking.

Big data analytics presents the possibilities of connecting together a variety of data sets from disconnected sources to produce business insights whether be for generating sales, improving products or detecting fraud.

It is therefore not surprising that global mid-market businesses are turning towards big data analytics to analyze social media data, web data, customer and sales data along with click-stream machine generated data and even communications data in the form of emails, chat, voicemails.

Leap of Faith or Solution Readiness

Analyzing data from diverse sources leads a mid-market business to naturally consider linking structured and unstructured data. This also drives them to evaluate and select the technology that can be used for simplified implementation. Simplified implementation is important because mid-market businesses do not yet have in-house capabilities to analyze unstructured data and those that have them consider the capabilities at best rudimentary.

Big data therefore is a major leap of faith for mid-market businesses resulting in treating big data analytics projects usually as separate to the existing analytics within the business. More aggressive adopters are planning to use big data analytics along with other analytics in a coordinated manner so that one does not become an inhibitor for the other.

In recent years technology and technology options have evolved extremely rapidly for an average business to understand, evaluate, purchase and implement. The complexity gets further exacerbated with lack of experience, lack of skilled manpower and innate difficulty in identifying external consultants that would be the most right fit for their business objectives and budget availability.

In spite of challenges, the study shows that there have been some successes when business units, IT & data analysts exhibit extraordinary alignment. Our study shows that mid-market businesses typically started their big data journey in one of four ways. Highest success rates for project implementation and generating new insights have been achieved when IT and data analysts work with external consultants from project inceptions.

SMBs as the Path to Big Data Simplicity

The global SMB spend on big-data related deployments will cross US$1.0 billion in 2013 which is a 32 percent increase from 2012. SMBs are still experimenting to see if big data analytics can provide newer insights into their operations and better knowledge about their customers. It is still very early days for small and mid-market businesses to fully embrace big data but they are planting the seeds in terms of re-architecting their IT infrastructure to plan for the future. But we believe that SMBs may very well race ahead of enterprises with their deployments as technology becomes simpler and consultants become experienced.

 

Techaisle’s global mid-market businesses’ Big Data Adoption & Trends study shows that the promise of superior data-driven decision making is motivating 43 percent of global mid-market businesses to either invest in or investigate Big Data technology. Out of these, 18 percent of mid-market businesses are actively investing in big data related projects. The possibilities of analyzing a variety of data sources, producing action-driven business insights is too big to ignore for mid-market businesses.

Big Data requires a certain level of IT sophistication and a history in the linear investment in Information Technology enablers to be successfully. While these factors predispose larger accounts to Big Data, the competitive imperative to understand customers, innovate products and improve operational efficiencies has already started to reach down to the mid-Market, forcing a search for how to leverage primary and secondary data that is generated by the business.

The current and planned investment represents a sizable opportunity considering that the segment is relatively new and requires a certain level of IT sophistication and a history in linear investment in Information Technology enablers to be successful. North America has both the largest market and the highest level of investment in Big Data overall in SMB and mid-market segments. Mid-Market attitude towards Big Data transitions from “Over-Hype” to “Must-Have” technology with the increase in employee size. However, nearly one-fourth of lower mid-market businesses consider big data to be over-hyped and yet 29 percent think that it will be an important part of their business decision making process in the future.

Business intelligence by itself has provided enough business insights, however, mid-market businesses are now looking for extracting business perspectives to drive superior decisions and ultimately achieve superior results.  Extracting business perspectives has become important as they rethink their marketing strategies because mobility, social media, and other transactional services have increased the number avenues for connections with their customers and partners.

In addition to understanding customers, mid-market businesses are also considering big data analytics as an important initiative to help them improve operational efficiencies.

Techaisle’s study shows that there are many different tactical objectives for deploying big data projects but the top among them are sentiment monitoring, generating new revenue streams & improving predictive analytics. It must also be said that businesses have figured out that there is a lot of publicly available data which could also be analyzed to their advantage.

The mid-market businesses actively investing in big data technologies are expecting some clear cut benefits from big data analytics such as increased sales, more efficient operations and improved customer service. These objectives differ slightly by different geographic regions. As the growth rates continue to lag in mature economies, the pressure to increase revenue grows resulting in developing robust analysis and extracting insights from all sales and customer data including transactions.

When specifically asked about preferred deployment choice in terms of on-premise vs. cloud, mid-market businesses are unsure as they are still navigating through their technology options. However, Hadoop dominates as the preferred platform but confusion exists.

In terms of analytics skill-set and long-term vision, the potential of linking structured and unstructured data sources to create new business insights is being considered very useful but at the same time mid-market businesses are not really prepared for it. In fact one-third of mid-market businesses agree that linking structured and unstructured data would be very useful for big data analytics but over 70 percent mention that they have either none or very limited capabilities of analyzing unstructured data. This is where they are turning to external help for guidance.

Needless to say, survey reveals that big data deployment is posing tremendous challenges. Technology confusion, lack of skilled resources and potential unclean data are being considered as the biggest roadblocks for big data project implementations. Big data technology and its far-reaching capabilities are being viewed by mid-market businesses as very complex resulting in very steep learning curves.

In spite of challenges, the study shows that there have been some successes when business units, IT & data analysts exhibit extraordinary alignment. Highest success rates for project implementation and generating new insights have been achieved when IT and data analysts work with external consultants from project inceptions.

Detailed Global Mid-Market Big Data Adoption and Trends report is available for purchase. Details are given here.

Underutilization and the complexity of managing growing data sprawl have spawned several trends during the last several years. Data-as-a-Service (DaaS) is one such trend which represents an opportunity to improve IT efficiency and performance through centralization of resources. DaaS strategies have increased dramatically in the last few years with the maturation of technologies such as data virtualization, data integration, MDM, SOA, BPM and Platform-as-a-service.

Within the corner offices of business heads, data scientists and analysts several questions are being asked:

• How to deliver the right data to the right place at the right time?

 

• How to “virtualize” the data often trapped inside applications?

 

• How to support changing business requirements (analytics, reporting, and performance management) in spite of ever changing data volumes and complexity?

In the early years most of DaaS initiatives were limited to financial services, telecom, and government sectors. However, in the past 24 months, we have seen a significant increase in adoption in the healthcare, insurance, retail, manufacturing, eCommerce, and media/entertainment sectors. This is because of massive amalgamation of extracting continuous insights from structured and unstructured data, liberation of data restricted and protected within silos to the enterprise level and the express desire to conduct real-time analytics.

Businesses are looking to solve tough data and process integration challenges as they once again begin to invest in new business capabilities. Data as a Service (DaaS) is based on the concept that the fragmented transaction, product, customer data can be provided on demand to the user regardless of geographic or organizational separation of provider and consumer. Additionally, the emergence of PaaS and service-oriented architecture (SOA) has rendered the actual platform on which the data resides also irrelevant.

Data as a Service (DaaS) has many use cases:

1. Providing a single version of the truth;

 

1. Integration of data from multiple systems of record

 

1. Enabling real-time business intelligence (BI),

 

1. Federating views across multiple domains;

 

1. Improving security and access;

 

1. Integrating with cloud and partner data and social media;

 

1. Delivering real-time information to mobile apps

Data as a Service (DaaS) brings the notion that data related services can happen in a centralized place – aggregation, quality, cleansing, enriching and offering it to different systems, applications or mobile users, irrespective of where they were. DaaS is a major enabler of the Master Data Management (MDM) concept.

Master Data Management is the Holy Grail in data management.  The focus for most businesses is on the single version of the truth or Golden Source “Product”, “Customer”, “Transaction” and “Supplier” data.  This is because:

• Fragmented inconsistent product data slows time-to-market, creates supply chain inefficiencies, results in weaker than expected market penetration, and drives up the cost of compliance.

 

• Fragmented inconsistent Customer data hides revenue recognition, introduces risk, creates sales inefficiencies, and results in misguided marketing campaigns and lost customer loyalty.

 

• Fragmented and inconsistent Supplier data reduces efficiency; negatively impacts spend control initiatives, and increases the risk of supplier exceptions.

MDM provides the plumbing that enables DaaS solutions. This plumbing allows for:

• Agility & Time to Market – Customers can move quickly due to the consolidation of data access and the fact that they don’t need extensive knowledge of the underlying data. If customers require a slightly different data structure or has location specific requirements, the implementation is easy because the changes are minimal.

 

• Cost-effectiveness – Providers can build a base with data experts and outsource the presentation layer, which makes for very cost-effective report and dashboard user interfaces and makes change requests at the presentation layer much more feasible.

 

• Data quality – Access to the data is controlled via data services, which tends to improve data quality, as there is a single point for updates. Once those services are tested thoroughly, they only need to be regression tested, if they remain unchanged for the next deployment.

 

• Cloud like Efficiency, High availability and Elastic capacity. These benefits derive from the virtualization foundation —one gets efficiency from high utilization of sharing physical servers, availability from clustering across multiple physical servers, and elastic capacity from the ability to dynamically resize clusters and/or migrate live cluster nodes to different physical servers.

We find that there is a common process that is appearing within the mid-market and customer customers focused on enabling and MDM strategy. It is the data logistics chain consisting of data acquisition, data stewardship, data aggregation and data servicing.

There is a sudden and dramatic shift in how data is handled in businesses as they are shifting away from a hierarchical, one-dimensional enterprise data warehouse initiative with fixed data sources to a fragmented network. This phenomenon has caused ripple effects throughout the old data logistics network.  Data-as-a-Service (DaaS) at its core is addressing this problem of fragmentation soundly enabled by MDM.