The latest issue of The Economist contains a special report on the ever-growing abundance of data being captured and analyzed across all areas of society, from business to health-care to crime.  It’s a fantastic look at how much data is available and how analysts are struggling to not only understand the implications, but to simply find ways to slice and dice immense amounts of information.

Here are a few choice tidbits that give a glimpse into the challenges facing statisticians, analysts, and more importantly, the software engineers creating the latest analytics platforms:

• Wal-Mart handles more than 1m customer transactions every hour, feeding databases estimated at more than 2.5 petabytes—the equivalent of 167 times the books in America’s Library of Congress.
• By 2013 the amount of traffic flowing over the internet annually will reach 667 exabytes, according to Cisco.
• Farecast can advise customers whether to buy an airline ticket now or wait for the price to come down by examining 225 billion flight and price records.

A few months ago, we wrote a post that mentioned some similar statistics, but the new article really blows away those numbers.

It’s going to take a really disruptive, revolutionary analytics platform to help companies digest and find the value in this kind of “big data.”  One that can analyze tens of billions of data points in just a few minutes.

I seem to recall hearing of a solution that would be perfect for this…  starts with the letter Q, I think… 

We’ve put a lot of work into Quantivo 4 and one of the highlights that we’re most proud of is our ability to dynamically segment customers based on contextually-specific behaviors.  That’s a mouthful but the potential that this brings to the market is just huge.

What context filters do for you is eliminate the need for multi-pass queries by allowing you to define the context of the query, such as items purchased in the context of a single invoice or the context of a customer’s lifetime.  Typical BI and analytics tools can tell you that a customer has purchased product A and product B, but drilling down to true context-specific answers quickly becomes difficult and time-consuming.

Let’s take a quick look at how the context impacts a real business question and dramatically changes the results of the query. The data is the same for each of these screen shots, which is point-of-sale data for a fictional home improvement retailer. In each case, we’re asking the same question: “I want to see the items (i.e. categories) purchased that contain the letters “ap” and only by customers in our “platinum” customer loyalty group, but the context of that question changes.  And remember: it takes only a single click to change the context!  And also remember:  it takes only a few seconds to answer these types of questions, even on hundreds of millions of detailed transaction or server call records!

In this first example, the context is set to “line items,” which returns line items containing the letters “ap” and purchased by platinum customers.  It’s a fairly simple question with a straight-forward answer and the only one that current BI tools can typically answer.

Read the rest of this entry »

All of us here at Quantivo are thrilled to announce the latest advancements to our award-winning and revolutionary analytics solution:  Quantivo 4!  We couldn’t be more excited to bring the world of Business Intelligence and Advanced Analytics into the the 21st Century…finally!

I could go on and on about how Quantivo 4 enhances our already powerful solution with new analytic capabilities.  I could also talk for hours about how Quantivo 4’s new, intuitive user interface enables you to simply drag-and-drop to segment your audience and customers using context-specific attributes without learning to code or write some complex querying language.  I thought about telling you how Quantivo 4 lets you immediately compare target segment results to your entire population so that you can instantly recognize opportunities and gauge their impact.  I could also fill up this post with more on Quantivo 4’s ability to analyze billions of customer clicks or transactions in just seconds.  But I should probably just get to the details, right?

I also thought about using this post to rant about how decades of BI solutions and technologies have had their chance and have failed.   But then that would lead me right into a pitch about how Quantivo has modernized analytics and introduced a new model for BI that puts answers into the hands of the people who need them, when they need them, and at a reasonable cost.  But I probably shouldn’t do that either.

Instead, let me link you to all of the details (below), and use this space to thank the entire Quantivo team for their continued hard work at developing, launching, and finalizing Quantivo 4 this quarter.  And, I’ll also thank our entire ecosystem of customers and partners for their continued support and participation in making Quantivo 4 such an exciting solution.

We are truly revolutionizing the BI and Advanced Analytics space, and this is only the beginning!

Read our Quantivo 4 press release.

Read what the press are already saying.

See Quantivo 4’s new capabilities.

Download our “What’s New in Quantivo 4″ data sheet.

I just read an interesting article that looks at BI trends and forecasts for 2010. The BI industry has been trying to solve business information access problems for many years now, using some tried and tested methods.  But, these methods have been debunked for their deficiencies multiple times over. So let me make my one, simple 2010 prediction:

Adding speed and a better user interface to data mining and predictive modeling tools is not going to make them any more agreeable or palatable to business users who do not want to use such tools in the first place!

Tweaking data warehouses with additional processing power to handle deeper analytics is still going to make the end user experience subject to the BI tool capabilities that source data off these warehouses.  And, shoving 30-year-old relational database technology on to The Cloud isn’t going to work either.

What’s the answer?  Quantivo’s unique approach to handling difficult and time-intensive data mining-style queries as a regular BI query with simple end user expectations around turnaround times (right now!) and analytic intuitiveness (just answer this simple question!).

Quantivo’s queries are interactive by design, enable the user to ask deep dynamic segmentation-style questions that return answers within a few seconds or minutes on data sets that are of the order of 100s of millions of rows.  Such queries can provide business insight at the speed of human thought dramatically enhancing the “business actionability” on the data results from such queries.

It’s great to see the BI pundits and analysts predicting this year’s trends, but without a totally new approach to analytics in general, the industry is not going to advance at the pace required by 2010 businesses.  Quantivo is that new approach, and in two weeks from today, you’ll see our latest attack on the BI status quo.  Stay tuned!

A quick conversation with a coworker this morning centered around the lack of shoppers out this past weekend.  Deemed “Super Saturday” (by some media types, I’m guessing), many retailers capture a majority of their holiday sales on this final Saturday before Christmas.  According to a recent article, “stores like Macy’s, Kohl’s, Walmart, and Toys R Us typically do as many as 60% of their sales on Super Saturday.”

The news over the weekend focused on East Coast retailers being hit by a big snowstorm that would diminish sales, but retailers out here on the West Coast lacked such an excuse.  So why the small crowds?  Is it the economy?  Are consumers moving more quickly to take advantage of Black Friday and Cyber Monday deals, so have already made most of their purchases?  Are shoppers waiting for even better deals later this week?  Are more and more consumers shopping online?  What is impacting their behaviors, and how have their behaviors changed?

Most retailers - both on- and off-line - will be struggling today to figure out their sales numbers for one of the biggest shopping days of the year.  Their Monday Morning Reports will compare year-over-year sales, and then they’ll scratch their heads as they attempt to ascertain what happened instead of what they can do in the next three days.

Behavioral Analytics is the perfect solution for this type of analysis: uncovering insights in huge volumes of transactions (holiday purchases) in an instant (today) so that they can make changes now (like, right now) in order to help maximize sales these next few days.

Understanding your customers - What have they purchased since Thanksgiving?  Where did they spent their money?  What complementary items remains to be purchased?  Which web content or products did they view over the weekend, and how often?  Who did X and Y, but not Z? - is the only way you can still impact your holiday sales.  Can you answer any of these questions with your current tools?  Can you answer them today?  If not, then you need to rethink how you ask and answer your marketing, merchandising, and promotion questions.

Can you still impact your holiday sales, given that it’s Monday, December 21st? If you had Behavioral Analytics at your fingertips, you wouldn’t be reading this post, you’d already be increasing your sales!

Happy Holidays!

We’ve been getting a great response to our just-released book, Behavioral Analytics For Dummies, and here is the reason:  decision-makers are starved for answers.

First, let me reiterate that they are starved for answers.  They aren’t starved for data - they’re already swimming in mountains of it.  They aren’t looking for queries - it takes a PhD to run most query tools, or to create a meaningful query in a so-called “user friendly” tool.  They aren’t asking for more of the same old high-level operational reports. They want the answers to the questions that are going to help them move the needle in their business.  And they want the answers now!

Behavioral Insider just published a great article on our Dummies book, “Dumbing Down Behavioral Analytics.”  It focuses on exactly that point:  analytics are meaningless unless the business managers can actually ask the questions and get the answers.

Quantivo is delivering that today and our solution not only gives marketers, merchandisers, web editors, and others access to enormous amounts of data, it lets them ask tough, complex questions and instantly get the answers that they can use to improve marketing campaigns, increase traffic, and drive sales.

We’re democratizing Behavioral Analytics by putting incredible analytics power into the hands of the people - from the marketing analyst to the statistician, from the web content manager to the PhD.

While there is definitely movement in the positive direction for retailers, it’s tough to discern how things went this past Friday.  I’ve been scanning the news sites for four days now, trying to determine if Black Friday was “good” or “bad” for retailers and the overall economy.  Here’s what I’ve found:

• On the positive side…
  • Sale Items and Curiosity Inspire Black Friday Shoppers
  • Black Friday results: Minimal growth
  • Black Friday Sales Strong, According to CEA
  • Black Friday sales inch higher
• On the negative side…
  • Black Friday Sales Weigh On Retailers
  • Retail stocks drop on lagging Black Friday sales
• And on the non-committal side…
  • Black Friday Results Mixed
  • Black Friday Shoppers Up, Spending Down

So, not much to go on there.  Looking at the numbers, Black Friday sales rose somwhere between 0.5%, according to ShopperTrak, and 13%, according to NRF.   While even a half of a percentage point may be small, it’s still in the right direction.  Given the state of the economy, I think that most retailers are happy with a small improvement over any loss.  If it truly is 13%, then that’s amazing!

As for Cyber Monday, early reports show a 6% increase over last year’s dismal results.  But again, positive is, well, positive!

I’m sure we’ll be hearing and reading about retail results more and more, each and every day between now and the end of the year.  But, as long as retail is moving in the positive direction, that’s wonderful news.

Trying to predict customer behavior by building predictive models?  That’s so 90’s.

Predictive models were fine when things moved slower, rich customer data was not available, you were simply trying to predict quantities and timings, and had no ability to personalize offers, messages or storefronts.

Today you are capturing mountains of data about customers and have the powerful ability to tune offers and messages to segments and individuals. So why in the world would you use systems developed in the 90’s - or earlier? You can’t afford to wait days or even weeks to see, much less understand, what’s happening. And since your competition is always nipping at your heels, you can’t afford to send an untargeted message to your audience.  They’re impatient and can easily go elsewhere.

The industry leaders are doing it, and doing it well.  Your competitors are doing it too.

With your tired, old solutions, getting answers to your deep, complex, valuable behavioral questions is extremely expensive, time consuming, and still won’t deliver what you need. What do you need?

• A modern architecture. One that’s elastic to grow with your needs.
• Infinitely scalable analytics, designed to provide you answers and insight without hunting and pecking, with no regard for data volumes.
• Answers that tell you what to do to get more customers, keep them coming back and how to sell them more.  Answers that tell you what to do next.
• The ability to SEE what behavior is being exhibited by your customers, SPOT the important trends that lead to increased conversions, and then ACT on this valuable insight with the right offers, content and messages.

Sound too good to be true? It’s not - it’s today’s reality.  A new approach to analytics, combined with a cloud computing architecture, and it’s called Quantivo.

Are you ready to step into this century?  Get started today!

Amazon Web Services, Quantivo’s cloud infrastructure provider, had two big announcements today.  The first release announced a 15% drop in instance pricing, with the example showing a Linux-based instance now going for the impossibly-low rate of 8.5 cents per hour of computing.  The second release announced new high-memory instances, with up to 68 GB of memory, 8 virtual cores, and 1690 GB of storage on a 64-bit platform.

While you process those overwhelming numbers (and corporate IT readers take a big gulp and start to sweat nervously), Amazon’s announcement can only be seen as one thing:  the beginning of the end of on-premise IT infrastructures.  The power, breadth, and scalability of Cloud Computing continue to surpass the capabilities of corporate IT.  The Cloud is quickly moving to such an unmatchable level, that corporate IT’s customers (i.e. business users), are going to soon demand a move to these faster, more powerful, and more flexible solutions.

Sure, as hardware prices drop for Amazon Web Services and other Cloud Computing providers, they also drop for corporate IT buyers.  But, add in the typical corporation’s cost structures and accounting practices, plus the resource limitations and other IT priorities on their plates, and it would be nearly impossible for companies to even consider ripping out and replacing hardware at such a blistering pace.

For solutions built in and for the Cloud, such as Quantivo, taking advantage of these huge advancements is instantaneous.  When Amazon announces more powerful boxes, we use more powerful boxes. Given the pace of progress in the Cloud, on-premise hardware (and software) continues to lose its appeal.

Google captured some buzz at this week’s eMetrics show in Washington, DC, with their release of new Google Analytics features.  While GA users are undoubtedly happy to see some of these new features, gaps between GA and Webtrends, Omniture, etc., still exist. Furthermore, regardless of your web analytics choice, it’s critical to consider both the depth of analytics available and the access to your underlying online data.

First, what is “analytics” when it comes to web data?

• Tracking time-on-site and pages per session is interesting, but it’s not going to tell you how to structure your next marketing campaign. Advanced analytics are required to understand how visitors’ engagement levels are driven by which content, and how you can move visitors to higher-value levels.
• Advanced filtering is reporting, not analysis. It’s an Excel feature, not an advanced capability.  Revenue is driven by slicing and dicing your customer data, asking questions and drilling down.  Better reporting is great, but it’s still not going to move the needle in your business.
• Once you do filter, reporting on, say, member vs. non-member visits still leaves you wondering both why the behaviors are different and, more importantly, what to do about it.  Without behavioral analytics, you’ll quickly hit the wall between knowing what happened vs. knowing why it happened.

Real power, flexibility and intelligence lies the ability to access and analyze ALL of the complete server calls from your website (not aggregated data), and then interactively drill into the data to discover the value within your visitors’ behaviors. You need to go beyond page views and reports to understand behavioral-based value. Knowing the average time-on-site is interesting, but it won’t drive revenue. Knowing the average time-on-site for visitors who complete >5 sessions per week for >3 weeks, have viewed content X and Y but not content Z, and have made >$100 in purchases in their lifetime is the type of analysis you need to really understand your audience, who’s valuable, who’s not, and how to drive revenue.

Second, why do I need access to the underlying server call data?

As Google continues to attack the enterprise market and put more pressure on the big web analytics players, companies looking to make a change need to consider the importance of advanced and integrated data analysis, regardless of the tool they choose:

• Does your vendor allow access to your complete server call data?  If not, then forget any advanced analytics or reporting using third-party tools.
• Can you integrate web data into other enterprise tools to drive marketing automation or other systems?
• Can you integrate other enterprise data , such as CRM, marketing, support, e-commerce, etc., with your web data to get a complete picture of my customer behaviors and value?
• Will switching cause you to lose your entire history of website data? Will your new web analytics solution allow you to import or include past historical data? Or will you need to run both side-by-side for a year or more?

Not having complete access to your web data would be a killer for most mid-sized and larger companies.  As soon as your business grows and your customer segments become even slightly complex, you’re going to need to both analyze that data and integrate it to continue your success.

Most of the fault here lies with the person who coined the term “web analytics” in the first place.  If they just would have called it what it is, “website information collection and reporting,” the need for a complementary analytics tool wouldn’t be so confusing.