Tuesday, June 24, 2008

The End of Theory: The Data Deluge

Link to Wired.com article

"All models are wrong, but some are useful."

So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don't have to settle for wrong models. Indeed, they don't have to settle for models at all.

At the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics. It calls for an entirely different approach, one that requires us to lose the tether of data as something that can be visualized in its totality. It forces us to view data mathematically first and establish a context for it later. For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn't pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day. And Google was right.

Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two. Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.

But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete. [Consider biology:] the models we were taught in school about "dominant" and "recessive" genes steering a strictly Mendelian process have turned out to be an even greater simplification of reality than Newton's laws. The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.

In short, the more we learn about biology, the further we find ourselves from a model that can explain it.

There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.

Link to Arstechnica's response

Correlations are a way of catching a scientist's attention, but the models and mechanisms that explain them are how we make the predictions that not only advance science, but generate practical applications. One only needs to look at a promising field that lacks a strong theoretical foundation—high-temperature superconductivity springs to mind—to see how badly the lack of a theory can impact progress.

Monday, June 23, 2008

In honor of George Carlin

That's all your house is — it's a place to keep your stuff while you go out and get more stuff. Now sometimes — sometimes you gotta move. You gotta get a bigger house. Why? Too much stuff. You've gotta move all your stuff, and maybe put some of your stuff in storage. Imagine that — there's a whole industry based on keeping on eye on your stuff.

Something is wrong here: War, disease, death, destruction, hunger, filth, poverty, torture, crime, corruption and the Ice Capades. Something is definitely wrong. This is not good work. If this is the best God can do, I am not impressed. Results like these do not belong on the resume of a supreme being. This is the kinda (expletive) you'd expect from an office temp with a bad attitude.

Thursday, June 19, 2008

Surprising quote from Einstein

In 1954, a year before his death, Einstein wrote a letter to Jewish philosopher Eric Gutkind that was sold at auction for $404,000.
The word God is for me nothing more than the expression and product of human weaknesses, the Bible a collection of honorable, but still primitive legends which are nevertheless pretty childish. No interpretation no matter how subtle can (for me) change this. These subtilized interpretations are highly manifold according to their nature and have almost nothing to do with the original text. For me the Jewish religion like all other religions is an incarnation of the most childish superstitions. And the Jewish people to whom I gladly belong and with whose mentality I have a deep affinity have no different quality for me than all other people. As far as my experience goes, they are also no better than other human groups, although they are protected from the worst cancers by a lack of power. Otherwise I cannot see anything "chosen" about them.


Link to article at Slate.com

Home buying practices adjust to high gas prices

Link to Article

Stroud's choice represents a fundamental shift in the way more Americans are approaching home buying in this era of ballooning gas prices. Real estate agents, transportation officials and industry surveys indicate that home buyers are placing more importance on cutting their gas bills and commute times than they have since the oil shocks of the 1970s.

And there are some early indications that homes near urban centers, and subway, train and bus stops are often selling faster and at better prices than those in the distant suburbs.

Gas prices, which have shot up $1.07 this year, are magnifying demographic trends that show more younger buyers and empty-nest seniors are moving back to urban centers. If gas prices continue their ascent, this could have profound consequences over time on the future development of American cities and suburbs and modes of transportation.

Homes in cities and neighborhoods that require long commutes and don't provide enough public transportation alternatives are falling in value more quickly than more central locations, according to a May study by CEOs for Cities, a network of U.S. urban leaders.

Monday, June 16, 2008

How to Pack Everything You Own Into One Suitcase

Link to NPR article

Doug Dyment, whose Web site onebag.com is devoted to the art of traveling light, says the key is to make a list in advance of what to pack and stick with it. He has developed a master list over the years that people can use as a starting point for creating their own.

"If it's not on your list, it shouldn't be in your bag," Dyment tells NPR's Michele Norris. "What happens with people is that they pack before their trip, and that packing activity consists mostly of talking to yourself and saying, 'Well I might need this and I might need that and what if the queen invites me to dinner?' And that's death to light packing."

Dyment advises people to think of what their lists look like well before a trip — literally writing it down and then checking off each item.

Dyment has two big tricks for packing a bag correctly: Don't let any space go unused, and wrap your clothes in bundles. "If you're packing a pair of running shoes, say, don't forget there's a lot of space inside those shoes that you can use to pack stuff," he says. When it comes to clothing, Dyment says travelers who fold items individually, put them in a stack and force them in the suitcase are making a huge mistake. Instead, he suggests using a technique called bundle
wrapping, because it keeps clothes from getting wrinkled and takes up less space.

"Never take more than two pairs of shoes," Dyment says. "In lots of business situations these days, you can buy shoes that are quite dressy looking and yet their internal construction is more like a high-quality running shoe."

Link to Onebag.com's checklist

Saturday, June 14, 2008

A Guide to Creating a Minimalist Home


Photo from 30elm

Link to Article at Zenhabits.com

Executive summary
  • Definition of a minimalist home
    • Minimal furniture, clear surfaces (few knicknacks), accent decorations, quality over quantity
  • Benefits of a minimalist home
    • Less stressful, more appealing, easier to clean
  • How to create a minimalist home
    • One room at a time, start with furniture, only the essentials, clear floors, clear surfaces, clear walls, store stuff out of sight, declutter, simple artwork, simple decorations, plain window treatments, plain patterns, subdued colors, edit and eliminate, place for everything

How to Live With Just 100 Things

Time.com Article

David Bruno's Blog (100 Things was his idea)

From David's blog:

My buddy Todd summed it up better than I could today when we were talking about it. He said, "Things are to be used. People are to be loved." The crazy thing about our consumer culture is that we so often reverse it. We use people to get the things we think we'll love. How stupid. As if fancy cars or more shoes are really going to satisfy us more than a great friend or a close relationship with our children.

Why am I doing the 100 Thing Challenge? Because I want to challenge stuff! I believe that run-away consumerism is making many of us narcissistic jackasses. It dulls our wits. Keeps us from thinking and acting like we understand what's really important. I'm planning on writing much more about this point. The main thing to remember now is that stuff is not passive. Stuff wants your time, attention, allegiance. But you know it as well as I do, life is more important than the things we accumulate. Challenge stuff!

Wednesday, June 11, 2008

Quotes from Orson Scott Card's Ender series

"So let me tell you what I think about gods. I think a real god is not going to be so scared or angry that he tries to keep other people down…A real god doesn't care about control. A real god already has control of everything that needs controlling. Real gods would want to teach you how to be just like them."
- Xenocide

So he believed. Believed, but the seed of doubt was there, and it stayed, and every now and then sent out a little root. It changed everything, to have that seed growing. It made [him] listen more
carefully to what people meant, instead of what they said. It made him wise.
- Ender's Game

He was commander every moment they were together. He never had to remind them of it; he simply was.
- Ender's Game

You can't rule out the impossible, because you never know which of your assumptions about what was possible might turn out, in the real universe, to be false.
- Ender's Shadow

For we humans do, when the cause is sufficient, spend our own lives. We throw ourselves onto the grenade to save our buddies in the foxhole. We rise out of the trenches and charge the enemy and die like maggots under a blowtorch. We strap bombs on our bodies and blow ourselves up in the midst of our enemies. We are, when the cause is sufficient, insane. He pretended all this time that humans were rational beings, when we are really the most terrible monsters these poor creatures could ever have conceived of in their nightmares. They had no way of knowing the story of blind Samson, who pulled down the temple on his own head to slay his enemies.
- Ender's Shadow

Is the Internet making us stupid?

Link to Article

The article says "Google", but the more accurate word is "Internet". The massive amount of information available via search, and the way that information on the web is broken into small chunks, is changing the way that we read, and therefore think.

Wolf worries that the style of reading promoted by the Net, a style that puts “efficiency” and “immediacy” above all else, may be weakening our capacity for the kind of deep reading that emerged when an earlier technology, the printing press, made long and complex works of prose commonplace. When we read online, she says, we tend to become “mere decoders of information.” Our ability to interpret text, to make the rich mental connections that form when we read deeply and without distraction, remains largely disengaged.

The idea that our minds should operate as high-speed data-processing machines is not only built into the workings of the Internet, it is the network’s reigning business model as well. The faster we surf across the Web—the more links we click and pages we view—the more opportunities Google and other companies gain to collect information about us and to feed us advertisements. Most of the proprietors of the commercial Internet have a financial stake in collecting the crumbs of data we leave behind as we flit from link to link—the more crumbs, the better. The last thing these companies want is to encourage leisurely reading or slow, concentrated thought. It’s in their economic interest to drive us to distraction.

That’s the essence of Kubrick’s dark prophecy: as we come to rely on computers to mediate our understanding of the world, it is our own intelligence that flattens into artificial intelligence.

Tuesday, June 10, 2008

This is the man who wants to be our President?

Link to Article

Some of McCain’s acquaintances are less forgiving, however. They portray the politician as a self-centred womaniser who effectively abandoned his crippled wife to ‘play the field’. They accuse him of finally settling on Cindy, a former rodeo beauty queen, for financial reasons.

Ted Sampley, who fought with US Special Forces in Vietnam and is now a leading campaigner for veterans’ rights, said: ‘I have been following John McCain’s career for nearly 20 years. I know him personally. There is something wrong with this guy and let me tell you what it is – deceit.

‘When he came home and saw that Carol was not the beauty he left behind, he started running around on her almost right away. Everybody around him knew it.

‘Eventually he met Cindy and she was young and beautiful and very wealthy. At that point McCain just dumped Carol for something he thought was better.

‘This is a guy who makes such a big deal about his character. He has no character. He is a fake. If there was any character in that first marriage, it all belonged to Carol.’

Irony - When Pro-Life Women Get Abortions

Link to Article

I'm all for freedom of thought and respecting opinions other than my own. However, when people profess one opinion and behave in a diametrically opposite way, I find it difficult to take their opinions seriously.

How to Get Rid of Ants without Pesticides

Link to article

Four strategies:

1) Cornmeal (ants try to eat, can't digest, starve)
2) Vinegar (ants don't like the smell, temporary)
3) Boric acid (toxic to ants, safe for environment)
4) Boiling water on the nest (if you can find it)

Monday, June 9, 2008

Rational consumer behavior emerges at $4 per gallon

Link to Article in Washington Post

At $3 a gallon, Americans just grin and bear it, suck it up and, while complaining profusely, keep driving like crazy. At $4, it is a world transformed. Americans become rational creatures. Mass transit ridership is at a 50-year high. Driving is down 4 percent. (Any U.S. decline is something close to a miracle.) Hybrids and compacts are flying off the lots. SUV sales are in free fall.

At $4 a gallon, the fleet composition is changing spontaneously and overnight, not over the 13 years mandated by Congress. (Even Stalin had the modesty to restrict himself to five-year plans.) Just Tuesday, GM announced that it would shutter four SUV and truck plants, add a third shift to its compact and midsize sedan plants in Ohio and Michigan, and green-light for 2010 the Chevy Volt, an electric hybrid.

AutoNation CEO wants gas to stay above $4 per gallon

Link to Article

It's entirely possible that a decade from now, we'll realize that this was a pivotal moment in the auto industry's history. This could be the moment when a century of relying almost exclusively on petroleum to power personal mobility gives way to a new model, in which electricity powers our transportation.

A gaggle of small companies such as Norway's Think Global AS and Silicon Valley's Tesla Motors Inc. are all gearing up to expand the electric vehicle market if the big guys won't. But the excitement over projects like the Tesla Roadster can't compare to the significance of the shift in mindset among the people who run the world's biggest auto companies. This isn't a crowd given to green idealism, but they have come to the conclusion that remaining totally shackled to petroleum is bad for business and are re-gearing their future vehicle plans accordingly.