BRotD - Entry 0263 AI over A/B Testing for Advertising

Best Reading of the Day

I am working to get my head around the impact Artificial Intelligence, machine learning, will have on our world so I'm pulling in any interesting article I see on the subject.

If you've watched even one episode of the landmark television series 'Mad Men' you'll have an appreciation of the work that goes in to creative decision making for advertising.  Now let the machines do a big chunk of the work.

Here is a snippet from the piece:

Even though one of the main goals of digital marketing is to serve customers the right message at the right time, we all know what it’s like to be chased around the internet by an ad that’s completely irrelevant or just plain annoying. And that’s because deciding when and where to deliver that message, not to mention the labor that goes into creating the message in the first place, has long involved human guesswork. Granted, those guesses often come after rigorous testing, but those best tests are limited to the often slow-moving process human analysis. Until now.

Artificial intelligence has already transformed everything from the IT department to the customer service experience, and now, machine learning is on track to completely change the ways we think about ad creative.

Here is the full article, another excellent one from ClickZ:

https://www.clickz.com/how-ai-could-make-a-b-testing-a-thing-of-the-past/216302/

Happy Reading,

J.W. Gant

Optimizing for Smaller Audiences

Hello folks, I've had a crazy last 6 weeks but have been capturing some stories I marked to blog when I finally carved the time to catch up and get back to my usual rhythm.  Here's the first of several stories.

I've often thought about the practice of A/B testing, or split testing (the practice of providing part of your audience with one design and the rest with another to compare and contrast).  It is especially difficult when your audience is small.  As a statistically driven person I think in terms of scientific sample size when determining the accuracy of results.

Thank about it.  Is your A/B testing useful when you have 5-10 visitors regularly?  100?  10,000?  When your numbers are in the tens of millions or more then ... definitely, but what about the smaller fish in the sea?

Here you go:

https://getuplift.co/how-to-optimize-a-low-traffic-site-without-ab-testing-step-by-step/

Here is a snippet from the piece:

Testing is not a matter of opening your testing tool and waiting for that little “Statistically Significant” marker. You need to reach your required sample size before concluding anything or else the insights will be invalid.

That’s where it gets difficult for low traffic sites.

Unless you’re detecting an incredibly large effect, you’ll need to run the test for months. The longer a test is running, the more vulnerable it is to sample pollution, which can rear its head in many ways.

Happy Reading,

J.W. Gant

Marketing Technology - Data Analysis

What's the saying in digital marketing circles?  You don't need a "growth hacker" you just need a marketer who understand data.

Yeap.  I'm onboard with that view, pushed by the best minds at Google and many other places.

Here is a great little piece to that end:

https://www.clickz.com/6-changes-marketing-measurement-will-impact-2018/206222/

Here is a snippet from the piece:

“One of the biggest pitfalls for performance measurement is to measure the ‘part’ with ignorance of the ‘whole,’” wrote Pearl Zhu, author of the Digital Master book series. Her quote perfectly reflects the growing realization of marketing measurement’s critical role. We see a requirement to evolve beyond attributing the value of a conversion to single customer touchpoints (the ‘part’) toward an understanding of marketing effectiveness throughout the ‘whole’ customer journey.

Multi-touch attribution delivers change through modeling all of a brand’s customer interactions to attribute fractional credit for each conversion to each unique touchpoint.

Happy Reading,

J.W. Gant

PS  Welcome to 2018!

Experience Experimentation or Checkout Optimization?

Any time I read about a sophisticated experimentation effort to optimize some digital process or tool I get all starry eyed.

Yes, these are the things that keep me up at night wondering about the larger meaning.

The folks over at eConsultancy have another excellent entry that goes right down this rabbit hole.  Well worth the read:

https://econsultancy.com/blog/69447-ask-the-experts-conversion-rate-optimisation-trends-challenges-strategy/

Here is a snippet from the piece:

There is a divide taking place in the industry. The businesses at the strategic and transformative level of maturity don't even use the term CRO or conversion optimization. Experimentation is the mindset change they have adopted and made part of their DNA. Brands like Amazon, Skyscanner, Booking.com…

Happy Reading,

J.W. Gant

Too Good, Too Efficient?

Is this possible?

Have you ever used a physical typewriter before?  If so you may know a bit more than the current generations know about this question.  The old long-handed key typewriters had one problem when the first "keyboard" was laid out: it was too fast for the physical hands to keep up.  The reason the letters were jumbled up a bit in to the current version of a keyboard is because the typists were too fast on a more straight-forward set of keys and the old typewriters couldn't keep up.

Fast forward to today with modern UI work for websites, mobile apps, etc.  Can these interfaces be too fast and efficient? Is there ever a benefit to slowing down?  That is the question posed in this excellent article:

https://econsultancy.com/blog/69101-why-increasingly-efficient-ux-might-not-always-be-a-good-thing/

Here is a snippet from the piece:

A recent leader in The Economist raised my interest in the little discussed area of ‘facile externality’.

The Economist summed up the concept as follows:

"Efficiency is at the heart of progress. Yet just as too much of a good thing (travel, say) can yield a bad (congestion), so excessive ease in transactions can generate costs, known in the jargon as a “facile externality”, such that less efficiency would actually be more efficient. In academic circles…. the notion is well established that innovations which eliminate too much hassle could do society harm."

The article continues, stating that "a few companies have recognised the benefits of restoring friction. Research into “the Ikea effect”, named in honour of those happy hours spent with an Allen key, a Billy bookcase and a rising hatred of Sweden, shows that people put extra value on things when they devote their own labour to them."

Happy Reading,

J.W. Gant

Txt Message Fiction

Yes that is a thing and is there really anything wrong with that?

Hey, so long as folks are reading it is okay with me.

What is especially cool is the way this little startup used A/B or "split" testing to develop this form of fiction.  Cool read,

https://medium.com/@prernagupta/how-we-got-10-million-teens-to-read-fiction-on-their-phones-19a2a475084c

Here is a snippet from the piece:

Three years ago, I was living in a small surf town in Costa Rica and writing my first novel, when I had a panic attack.

The novel was a sci-fi fantasy trilogy for young adults, set in Silicon Valley a hundred years in the future. I’m a tech entrepreneur, so it’s not surprising I chose this theme.

But there was something unusual about this sci-fi story — my protagonist was a dark-skinned Indian girl, like me. And that was what caused me to panic.

Would anyone read a sci-fi story with a dark-skinned, female protagonist? How was I going to convince an agent to take a book like this seriously? Would a publisher be able to find an audience for my strange story? And, do teenagers even read?

Happy Reading,

J.W. Gant

Do Customers WANT to Engage With You?

This headline really caught my eye.  Plenty enough to get me to read it and so should you.

Newsflash From Big Nerd Ranch: ‘Customers Really Don’t Want To Engage With Your Brand’
Here is the story:

http://www.cmo.com/interviews/articles/2017/3/20/the-cmocom-interview-aaron-hillegass-big-nerd-ranch.html#gs.vV1TDPM

Here is a snippet from the piece:

Aaron Hillegass, founder and CEO of Big Nerd Ranch, was involved in one of the most important mergers of our lifetime: He was working at NeXT when it was purchased by Apple. Eventually, though, he left Apple to create Big Nerd Ranch, a training and consulting firm that specializes in creating mobile apps and other emerging technologies.

Nineteen of the top 25 most popular apps, such as Facebook and Spotify, were created by teams who got their training from Big Nerd Ranch. Based in Atlanta, Big Nerd develops native mobile apps and web apps for some of the world’s most respected brands, including Airbnb, Smithsonian Channel, and Nextdoor.

Hillegass is the author of three popular books on software development, including “iOS Development: The Big Nerd Ranch Guide."

Happy Reading,

J.W. Gant

Split Testing for your Content Marketing

The folks over at CMO.com have a great little read on how to optimize your content marketing.

Familar with A/B Testing?  You should be as I've written extensively about it in these pages.

Here is the story:

http://www.cmo.com/opinion/articles/2016/6/1/5-ab-tests-you-should-try-to-grow-your-content-conversions.html

Here is a snippet from the piece:

With 78% of CMOs seeing custom content as the future of marketing and more than 90% of marketers using content marketing, you’re in good company.

In fact, according to DemandMetric, marketers now spend more than 25% of their budgets on content marketing. Clearly, marketers are putting a lot of effort into creating content to help brands get found online and engage with their audiences. Those efforts get a handsome return--since for every dollar spent, content marketing generates, on average, three times as many leads as traditional marketing.

But how do you guarantee that your content is conversion optimized? The short answer is that you don’t.

Happy Reading,

J.W. Gant

Windows 10

I did it and it was very painless.  I upgraded my main desktop machine at home to Windows 10.  Piece of cake to do, and wow it is a really nice OS.

The nerd in me must admit it felt good, almost comforting, to once again be going through a Windows OS upgrade process. So, how about a couple good stories on this?

First is a review by CNet:

http://www.cnet.com/products/microsoft-windows-10/

Here is a snippet from the review:

THE GOOD Windows 10 bridges the gap between PCs and tablets without alienating anyone. The new OS combines the best bits of old and new Windows features into a cohesive package, while correcting nearly all of the missteps of Windows 8. The upgrade process is mostly painless, and free for most Windows 7 and 8 users.

THE BAD Many of the new features will be lost on those who don't care about touch. Automatic, forced updates could spell trouble later on. Cortana's features are better suited for smartphones.

For those interested in the process used to develop this new version of the OS, to learn before launching, and to set up a flywheel of ongoing development for the new "Windows as a Service" concept read this excellent story that contains examples of A/B testing and much more:

http://venturebeat.com/2015/08/13/how-microsoft-built-and-is-still-building-windows-10/

Here is a snippet from that piece:

The Windows 10 team started with the basic capability of having multiple virtual desktops, then added functionality to see what users wanted: Over the release, the team would look to incorporate the top feedback items. First basic switching was added, then keyboard support was requested and implemented, then users asked about being able to sort contents in the taskbar that map to the desktop, and so on. Aul considers virtual desktops a “great example for where the feedback really led the development of the feature.”

Eventually, an A/B test was required. For the taskbar, there were two different ways to show your running apps: Display all the apps or display a filtered view, where for any given desktop you only see what you’re running in that desktop. Microsoft offered the options to two different groups and asked each how they liked their variant.

Microsoft has become a very different company today, and it looks like a change for the good based on the products and services beginning to come out of Redmond.

Happy Reading,

J.W. Gant

BRotD - Entry 0218 Netflix Redesign

Best Reading of the Day

I've been sitting on this one for a few days since finding it over the weekend.  It is a great read though. Which digital video streaming service do you currently use?  Hulu?  Amazon?  Netflix?  HBO? YouTube?

Here is a great read on Netflix.

What do you do if you design a system that is used by 10 people?  Chat with them and see how it works, maybe make some minor adjustments, and plan on the next big update to help them out.  What about if you have 10 million people using that system, or many more like say 62 million?

That becomes a pretty big challenge right?  Sure is.  Read more here.

http://www.theverge.com/2015/5/22/8642359/the-science-behind-the-new-netflix-design

Here is a snippet from the piece:

Back in March, a developer named Renan Cakirerk wrote a small piece of code that made a big impact on Netflix. Cheekily named "god mode," it addressed one of the most annoying aspects of trying to use Netflix in your browser: scrolling through the company's ever-growing list of movies. Once enabled, it would simply give you one, big list. Instead of sitting there, holding your mouse in anticipation, you could simply find the title you wanted and get on with watching.

The web lit up with stories hailing it as a crowning achievement of little-guy ingenuity. TechCrunch called it a fix for a "wonderful problem." CNET dutifully noted that it "saves you from the slow monotonous horizontal scrolling." Mashable, meanwhile asked, "Are you listening, Netflix?"

Inside Netflix, though, god mode was old news. In fact, the company had already tried it out on thousands of unsuspecting users years earlier. And it was a total disaster.

Good stuff.  I like the A/B testing at play there.  Release it to a scientifically controlled sample size with a control group and analyze the results.  Is the impact significant?  If so, in what way?  Does it "help" or "hurt" the user experience?

I'm a user of Netflix after coming back in to the fold recently.  I hadn't used them for years since I previously received DVDs through the postal service.  I like the digital service and am blown away by the 4K presentation of shows like 'House of Cards' on my fancy new LG set.  Good stuff and a fun company to follow.

Happy Reading,

J.W. Gant

Best Reading of the Day - Entry 0203 User Supported A/B Testing

I love just about any article on helping product people make better products.  A/B Testing, the practice of using statistically significant results to help choose between 2 or more design options, is right in the wheelhouse to provide that kind of assistance.

Here is a good little article on the subject:

https://econsultancy.com/blog/66243-five-ways-user-insight-improves-a-b-testing/

Here is a snippet from that piece:

In this article we examine how User Insight (gathered through online usability testing) is improving A/B Testing programmes by drawing on real-world implementations with five best practice recommendations:

1. Use customer-struggle insight to prioritise test plans.
2. Develop root-cause hypotheses.
3. Improve variant quality.
4. Tackle tough problems.
5. Get a view on competitors.

Happy Reading,

J.W. Gant

Best Reading of the Day - Entry 0197 Digital Marketing by Walmart and Sports Chek

Two stories out of Mobile Commerce Daily are worth reading, on the same overall subject, and from the same source at eTail West

First is the required shift in thinking to go digital:

http://www.mobilecommercedaily.com/brands-should-shift-focus-from-creative-to-content-on-digital-fgl-sports-exec

Here is a snippet from that piece:

Sports Chek was able to ramp up sales by converting its weekly print circular, which only 17 percent of consumers read regularly, to a digital circular viewable on Facebook. The company claims that it saw a 16 percent increase in sales each time it tapped Facebook for a social media promotion.

Although the move required Sports Chek to take a risk by pulling the allotted thousands of dollars for the print circular and giving it all to Facebook, the profit was ultimately twice as high.

Next up is a Walmart exec's take on search engine marketing, a subset in many ways of SEO:

http://www.mobilecommercedaily.com/walmart-com-exec-claims-mobile-bidding-deserved-starring-role-in-sem-product-searches

Here is a snippet from that piece:

Mobile bidding should also be at the forefront of retailers’ minds, although A/B testing is also paramount.

“Before, your ad group was used to write ads,” Mr. Gadacz said. “Now, your ad group is all about mobile bidding.

“Mobile is so big that it really does matter, and to be able to control your efficiencies, this is an absolute must.”

Happy Reading,

J.W. Gant

Best Reading of the Day - Entry 0170 A/B Testing Science

The folks over at eConsultancy have just wrapped a very good 3 part series on split testing, or A/B testing.

What is A/B testing?  How about WikiPedia to help?

http://en.wikipedia.org/wiki/A/B_testing

Here is the definition listed there:

As the name implies, two versions (A and B) are compared, which are identical except for one variation that might affect a user's behavior. Version A might be the currently used version (control), while Version B is modified in some respect (treatment). For instance, on an e-commerce website the purchase funnel is typically a good candidate for A/B testing, as even marginal improvements in drop-off rates can represent a significant gain in sales. Significant improvements can sometimes be seen through testing elements like copy text, layouts, images and colors,[1] but not always.

First up is the introductory article:

https://econsultancy.com/blog/65672-using-data-science-with-a-b-tests-sample-sizing

Secondly is Chi-Squared testing:

https://econsultancy.com/blog/65711-using-data-science-with-a-b-tests-chi-squared-testing

The third and final wraps up the set:

https://econsultancy.com/blog/65755-using-data-science-with-a-b-tests-bayesian-analysis

Very good reading over all, especially if your understanding of statistics could be improved.

Happy Reading

J.W. Gant

Best Reading of the Day - Entry 0074 Beacon Payments and More

I found a plethora of great reading today.  Let me share it with you.

First up is a simple news piece from the folks at MorningNewsBeat:

The National Retail Federation (NRF) said yesterday that "total holiday retail sales, which includes November and December sales, increased 3.8 percent to $601.8 billion, which was in line with NRF’s projected forecast of 3.9 percent and $602.1 billion. In addition, non-store holiday sales, which is an indicator of online and e-commerce sales, grew 9.3 percent to $95.7 billion."

This means eCommerce has been exploding and led the charge to much higher sales figures this year over last.  Buried in other news is the fact that tablet sales have been at the head of those figures.  If you are in eCommerce you better be looking at tablets.

This next one raises the question of mobile apps in retail.  Do we really need these?  Are people using them?  Read on:

http://econsultancy.com/blog/64136-mobile-apps-in-retail-who-needs-them

I may have been most interested in this next one.  Mobile payments is embracing BLE Beacon technology.  Wow!  Knew this wasn't far off from hitting actual stores.

http://www.mobilecommercedaily.com/ibeacon%E2%80%99s-mobile-payments-potential-opens-to-retailers

Finally is the question of A/B Testing and how to perform this with native apps.  Facebook is well known to have stated, Zuckerberg himself, that creating a non-native app for Facebook was the greatest mistake the company had made up to that time.  Native was the path for them from that time forward.  This does complicate some marketing questions for conversion rates, etc.  Great read.

http://mobilemarketingmagazine.com/ab-testing-the-future-of-apps/

Happy Reading,

J.W. Gant