:::: MENU ::::

Statistics, politics, and opinions: How they all make sense

Statistician Nate Silver has been under criticism for partisan bias because his model evaluating the presidential election has heavily favored Obama since it launched this summer. This criticism comes from a misunderstanding of statistics, both how they are calculated and what they mean.

As most polls show, this is a very close election.  The popular vote will likely be between 1-2 percentage points. The closest swing states also may be decided by only 1-2 percentage points. Nate Silver’s model understands this and seeks to explain this with more depth than any pollster or pundit.

A single poll, for example may show Obama and Romney tied with a margin of error of plus or minus three points. This means, based on the sample population polled, there is a 95 percent chance that the true poll result is within three percentage points. There is a 95 percent chance, known as the confidence interval, that the true result should have showed Romney up three points and Obama down 3 points.  There is also a five percent chance the actual result is outside of that 3 percent margin of error.

This is where poll aggregators like Real Clear Politics and Cook Political Report offer more insight. By simply averaging polls, the margin of error is reduced and the confidence interval increases. More polls means a larger sample size means a lower chance the sample population isn’t reflective of the full population.

Nate Silver takes his analysis a step further, by incorporating economic data, fundraising, incumbent status, and other, in his opinion, indicators of electoral success.  While his model is in many ways, an opinion, it’s a very transparent one that allows others to test and verify his analysis.

Silver’s success came, in part, from correctly predicating 48 out of 50 states in 2008. But this mistakes Silver’s predictions and even his claims for success.  He is providing odds, similar to odds makers for betting.  When he says a state has a 51 percent chance of going for Obama and 49 percent chance for Romney, that means 51 out of 100 times, Obama will win that state.  When Silver claims Obama has an 80 percent chance of winning this election, he’s still saying Romney has a 20 percent chance of winning. If Romney did win the election, that wouldn’t disprove Silver’s model.  Since we can’t hold the election 25,001 times (as Silver does in his simulations), we can’t know for certain how accurate his model is.

Upsets happen, in politics, sports, and any situations with odds; because we expect the unexpected to happen occasionally.  Thankfully we have tons of data related to presidential politics, from prior elections and thousands of polls for the current one.  That’s why a model like Silver has created is believable, because it is based strictly at looking at prior and current data to make the best possible prediction. When unexpected outcomes happen, that only serves to improve the model.

Many in the media would probably love to vilify Silver if Romney were to win.  Even if Silver fails to predict every state properly, that’s not his goal. He’s simply showing what the data shows – what is the more likely outcome.  Unexpected things do happen, but you likely won’t expect them.

Naming “Best of” subjects Yelp to lawsuit from Village Voice

Helping people discover great places to visit and eat has grown into a popular business. Websites and apps enjoy promoting the “Best of” a specific location. Village Voice holds trademarks on “Best of San Francisco”, “Best of Seattle", and several other best of cities and has chosen to sue competitor Yelp for using Best of language on its website.

Last year, the Village Voice sued Time Out New York for using “Best of NYC”. Time Out counter-sued claiming “Best of” is generic and should not be able to be trademarked. In April 2012, Village Voice and Time Out reached an undisclosed settlement (registration required).

More and more we are seeing companies and individuals bullying others from using basic language and ideas, both of which defenders of intellectual property will claim can’t be limited by intellectual property.  “Best of” is completely generic and has been used in books and magazines for decades. Village Voice, in fact, only registered its trademark “Best of NYC” in 2007.  This is a trademark that should never have been approved.  And Time Out not fighting the case just goes to show how easy it is to bully even sizeable companies with frivolous IP lawsuits. It will almost always be cheaper to settle than to fight in court.  Even if Time Out or Yelp were willing to fight to get the trademark invalidated, its unlikely they could recoup legal fees.  Why spend millions just to prevent the company from suing another competitor? If Yelp decides to settle, that just leaves Village Voice armed to sue more magazines, books, and websites. The “Best of” the web, all vulnerable to a company scared of some competition.


Midnight in Paris film sued over line from Faulkner novel – Updated

Woody Allen film Midnight in Paris has been sued for copyright infringement over a single line in the movie. Faulkner Literary Rights, the company that owns the copyright to author William Faulkner’s writings, including novel The Sound and the Fury, says the line from Requiem For a Nun was used without permission and is seeking unspecified damages.

In the film, the line said is: "The past is not dead! Actually, it’s not even past. You know who said that? Faulkner. And he was right. And I met him, too. I ran into him at a dinner party." The only appropriation from the book is  ten words: "The past is never dead! Actually, it’s not even past.”

This is certainly a case of fair use.  The line, which is cited to Faulkner in the script (even though that is not a requirement for fair use), is few words from a 286 page novel, far from threating the commercial value of the book. Rather, Allen citing the line can increase the commercial value of the book, introducing people to a new Faulkner book.

Unfortunately, fair use is a defense that must be made in court. This means significant court costs for Sony Picture Classics, the studio behind Midnight in Paris, just to prove they did nothing wrong.  Faulkner Literary Rights, further, does not need to prove their commercial value has been hurt. Simply the act of copyright infringement, even if it was to the holder’s benefit, can cost hundred of thousands of dollars in damages.  The result, most likely, is Sony will settle, paying a settlement in order to save on court costs.  All over ten words.

Updated 10-26-12 1:01pm – Clarified that the quote in the film was actually slightly different from the book, changing never to not.  It’s not necessarily a transformative change, but still, shows how silly this lawsuit is.

Italian scientists guilty of manslaughter for not predicting earthquake

Six Italian scientists and one former government official will spend six years in prison for failing to predict a 2009 earthquake that killed 309 people. 

Prosecutors claim the scientists made incorrectly reassuring statements. The scientists, rightfully, defended that earthquakes are still near impossible to predict. More than 5,000 scientists  wrote an open letter to Italian President Giorgio Napolitano defending the lack of predictability for earthquakes.

This ruling is tremendously anti-science and potentially stifles scientific research.  While it’s likely to be appealed, researchers of any kind may find themselves wary of Italy.  For a country already struggling with economic recession, railroading scientists and smart people seems to be a flawed strategy.

Malcolm Sperrin, the Royal Berkshire Hospital’s director of medical physics told the BBC:

If the scientific community is to be penalized for making predictions that turn out to be incorrect, or for not accurately predicting an event that subsequently occurs, then scientific endeavor will be restricted to certainties only and the benefits that are associated with findings from medicine to physics will be stalled.

Instant gratification determines decisions, even against our best interests

Harvard researchers found that wellness programs at organizations resulted in gains of $3.27 for each dollar invested. Yet programs like these are few and far between. Even asking CEOs, like CEO of the Energy Project Tony Schwartz did at Davos, whether wellness programs are beneficial for business, and all CEOs said yes. Yet again, few implement wellness programs at their companies.

From pollution to eating habits to drug use, there is a recognizable pattern of humans making decisions based on the short term gains (laziness, enjoyment, small rewards), even when the long term costs are well known. 

Our incentive systems have gravitated toward rewarding these short-term gains. Stock price now determines a CEO’s worth, rather than building a sustainable business. Daily poll tracking in politics makes politicians legislate for the immediate campaign rather than invest in long-term gains.

This makes companies more reliant on their current profits. And because politicians are always campaigning, they will pass laws to protect those profits in order to raise money for the endless election cycle.

On an individual level, neuroscientists have theories for how to retrain people to mentally account for the long-term costs and benefits in decision making, the type of thinking people more often rationalize away. 

From a business and legislative level, we need to create incentives that encourage long-term planning because that, in the end, generates more revenue and profits even if there are initial losses. Reducing stock price as the main barometer of CEO performance would be the most obvious change (or adjust the time table to judging the stock price over a two year period rather than quarter to quarter).

The Innovator’s Dilemma further explains how companies ignore the long-term. Often, companies under-estimate the speed of changes to the market, preferring to hold onto their current profits rather than invest in a long-term risk. Newspapers, for example, have continued instituting paywalls online in order to stave the loss of readers from their more profitable print publications. The long-term plan would involve investing in the online market, recognizing that that is the growing market, possibly at the print publication’s expense. Because the longer newspapers wait, the more competitors step forward to dominate the new space. Kodak, who invented digital cameras more than a decade before everyone else, focused on its profitable photo printing business. By the time that business collapse, almost every other consumer electronic company had digital cameras on the market and Kodak is facing bankruptcy.

Apple versus Samsung: Patent monopoly versus antitrust

Patents are already government granted monopolies. Though we normally think of monopolies as bad things, patents (and copyright) are nothing but exclusive power over a specific item or area or expression, allowing the patent holder to charge whatever they choose or withhold the technology or expression.

In some cases, when one or many patents cover a set technology, standard licensing agreements are set up, otherwise it would be too expensive or time consuming for a new business to negotiate all the licenses needs. These standards, however, are a band-aid to try to remedy problem that patents are hampering innovation. This type of licensing agreement covers everything from ethernet cables to web video encoding.

Absent these licensing agreements, lawsuits run rampant. Apple and Samsung are tied in a global legal battle over patents, trade dress, and other legal issues. Like most patent cases, Apple sued Samsung over patents, so Samsung sued back over its own patents. Each has filed lawsuits in the U.S., Australia, and several European countries, 10 countries in all so far, with each battle playing out on its own (and costing each company millions of dollars).

European authorities are now investigating Samsung for antitrust violates related to its patent licensing, specifically against commitments to license patents related to mobile phone technology.

The EU is accusing Samsung of antitrust violations over its government granted monopoly.

If the EU is worried about anti-competition activities, why is it granting these patents in the first place? The main function of the patent is to block others from using that technology without the patent holder’s permission. Companies form these patent licensing agreements in order to protect themselves while still blocking out competitors not included in the licensing agreements. I don’t have the specific details of Samsung’s 1998 agreement. There are agreements that because of a technology’s standardization, like some mobile technologies, that companies are required to license their patents. But again, I ask, why give them the patent? If some technology has become so standard and widespread, was it really non-obvious when first developed? Or is it something so simple, everyone working in that space knew they needed the same thing and all invented it separately (which is usually the case).

Instead of asking whether Samsung and Apple’s patents are truly non-obvious and needed to encourage innovation, the EU wants to create a more complex set of laws with antitrust that make it even harder to innovation and gives more money to lawyers.

Patent lawsuit over smart thermostat

Amid all the patent lawsuits around mobile phones, is easy to forget just how mundane patent lawsuits can get, yet still affect our daily lives.  Nest, the makers of a critically acclaimed smart thermostat, is the target of a lawsuit from Honeywell, a long time thermostat manufacturer.

Created by one of the original designers of the iPod, Nest’s thermostat learns user habits to automatically change the temperature, has an auto-away feature, and scans the thermal decay of your house to save energy. The patent lawsuit, which includes retailer Best Buy as a defendant (even though Best Buy does nothing but sell the product), lists several patents Nest allegedly infringes:

  • U.S. Patent No. 7,634,504 – "Natural Language Installer Set Up for Controller"
  • U.S. Patent No. 7,142,948 – "Controller Interface with Dynamic Schedule Display"
  • U.S. Patent No. 7,584,899 – "HVAC Controller"
  • U.S. Patent No. 7,159,789 – "Thermostat with Mechanical User Interface"
  • U.S. Patent No. 7,159,790 – "Thermostat with Offset Drive"
  • U.S. Patent No. 7,476,988 – "Power Stealing Control Devices"
  • U.S. Patent No. 6,975,958 – "Profile Based Method for Deriving a Temperature Setpoint Using a ‘Delta’ Based On Cross-Indexing a Received Price-Point Level Signal."

Honeywell actually invented a smart thermostat almost 20 years ago and has since sat on the technology. “We found that consumers prefer to control the thermostat, rather than being controlled by the thermostat,” said Honeywell’s president of environmental and combustion controls division Beth Wozniak.

Honeywell, instead of innovating on a new technology, did nothing with it.  That is not the intention of the patent system. Patents are supposed to “promote the progress of useful arts and sciences.” When Nest releases an exciting new device, Honeywell sues trying to block the innovation.  Thermostats are far from a highlight of interior decorating, but they serve an important function in a household.  The more technology we lock up in patents that go unused, the slower we’ll discover new functions to make our lives easier and more efficient.

Lodsys didn’t invent in-app purchases but wants money for every one

Lodsys, a company that produces no products, has been sending out letters to Apple App Store developers demanding license payments over their patent on in-app purchases.  While Apple itself is mixed up in more than a dozen patent lawsuits, it is becoming growingly apparent that there is no area safe from a patent lawsuit.  Lodsys, who seems to be offended at the idea that people call their company a patent troll, is only looking for 0.575 percent of in-app purchases.  Now Lodsys didn’t invent this patent. They purchased it from Intellectual Ventures.  And the patent itself is pretty obvious.  Applications and websites have long had purchases to be made after opening their program (or site).  This just applies to the mobile version of doing that same thing.

How is innovation served by paying Lodsys?  How does paying Lodsys promote the progress?

For developers, you can get some advice here on what to do if you get one of these letters.

Price regulations should fix inefficiencies, not cause them

France is push forward a law to place price regulations eBooks.  This expands current price regulations on paper books that requires all books sold in France to be sold for the same price – no discounts. The law aimed to protect small book stores, but it now going to cover eBooks regardless of who is selling them.

Our love for the corner book store is wonderfully nostalgic, but economically inefficient.  While we in the United States lament Walmart and Amazon pushing out mom and pop bookstores, the reality is authors are publishing more books than before, and people are reading more books than ever before.  And because eBooks reduce upfront and marginal costs for book publishing, authors are more able to read audiences. Amanda Hocking, for example, has been selling her own, cheap eBooks without a publisher and is making millions of dollars.  Even publishers are starting to see the success of eBook revenue – Hachette claims more than 20 percent of its U.S. revenue is from eBooks.

Let’s ignore the impossible regulatory nightmare France opens up by trying to regulate online businesses like Amazon and Apple.  Rather price regulation is a mistake France and even digital media companies are making.  Apple and Amazon set very specific prices on songs, movies, and TV shows. Almost every new video game is sold for $60.  These are price regulations even without government interface, and they only lead to economic inefficiency.  It means more popular/valuable products can’t sell for higher prices and less popular ones can’t sell for cheaper.

Like Amanda Hocking, other digital products not subject to price regulations are seeing great success with lower prices leading to great profits in greater volume.  Game publisher Valve discounted Left4Dead significantly and found sales jumped 3,000 percent – leading to more sales than its opening weekend.

Remember, price and value are not the same thing. We all value air immensely, but the price is free. When something many people value is made cheaper, then the market for that product increases.  With smart pricing and other business models, businesses can make more money with lower prices – computers are more valuable today, but are far cheaper than 10 years ago.

Price regulations can be used to fix inefficient markets, most specifically for products where the overall cost is not built into the price.  For example, oil is running out and damages the environment, but these costs are not felt by the purchaser because the costs will come years later.  Smoking also will lead to greater health costs in the future, but that is not built into the price of the initial purchase.  This is why taxes can help limit these markets for the better.

It’s a very fine line to find an inefficient market.  Price uniformity is in the form of price regulation is inefficient.  Maybe that’s why so many economists oppose it. Or tax a working market just because. Utah seems to like that idea.

Motorola Xoom is not a failure because it didn’t sell iPad numbers

I’m sorry if I sound like an Android cheerleader. I love Android phones, but have an iPod Touch and an iPad 2, so I live in both camps.  What is frustrating is Android, for all its success, gets pretty unfairly criticized by even tech circles.  Take the most recent example of allegedly low sales for the Motorola Xoom, the premiere Android tablet.

The Xoom sold an estimated 100,000 units, a spec compared to the millions of iPad 2 sales over the approximately sale timespan.  But you don’t judge success against the blockbuster.  The iPad is the Avatar blockbuster compared to the the Tourist’s healthy box office.

The same comparisons of Android phones get made to the iPhone. No single Android phone will match Apple sales 1 to 1. Android hardware competes with dozens of hardware makers for carrier and shelf space.  Apple has a focused process which works very well for them.  It would be like saying the Macbook Air outsells every Windows laptop and thinking this somehow says every Windows laptop is a failure.

Also note that for most of this sales period, there was only one version of the Xoom compared to 6 different iPads.  Yes, I’m sure the top of the line iPad sold more than the Xoom, but those numbers would likely be more comparable than all iPad versions.

Let’s do some math. The Xoom, retailing for $800, sold 100,000 units. That means Motorola made $80,000,000 in about a month.  Even for a large company like Motorola, that is some respectable revenue.  It’s very possible Motorola expected higher sales. But can we please stop calling Android hardware a failure? Let’s remember how unremarkable the first Android phone, the G1, was. And yet now the total of Android phones are outselling iPhones.  This is a marathon.