Tuesday, June 24, 2008
The End of Theory: The Data Deluge
"All models are wrong, but some are useful."
So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don't have to settle for wrong models. Indeed, they don't have to settle for models at all.
At the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics. It calls for an entirely different approach, one that requires us to lose the tether of data as something that can be visualized in its totality. It forces us to view data mathematically first and establish a context for it later. For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn't pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day. And Google was right.
Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two. Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.
But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete. [Consider biology:] the models we were taught in school about "dominant" and "recessive" genes steering a strictly Mendelian process have turned out to be an even greater simplification of reality than Newton's laws. The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.
In short, the more we learn about biology, the further we find ourselves from a model that can explain it.
There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
Link to Arstechnica's response
Correlations are a way of catching a scientist's attention, but the models and mechanisms that explain them are how we make the predictions that not only advance science, but generate practical applications. One only needs to look at a promising field that lacks a strong theoretical foundation—high-temperature superconductivity springs to mind—to see how badly the lack of a theory can impact progress.
Monday, June 23, 2008
In honor of George Carlin
Something is wrong here: War, disease, death, destruction, hunger, filth, poverty, torture, crime, corruption and the Ice Capades. Something is definitely wrong. This is not good work. If this is the best God can do, I am not impressed. Results like these do not belong on the resume of a supreme being. This is the kinda (expletive) you'd expect from an office temp with a bad attitude.
Thursday, June 19, 2008
Surprising quote from Einstein
The word God is for me nothing more than the expression and product of human weaknesses, the Bible a collection of honorable, but still primitive legends which are nevertheless pretty childish. No interpretation no matter how subtle can (for me) change this. These subtilized interpretations are highly manifold according to their nature and have almost nothing to do with the original text. For me the Jewish religion like all other religions is an incarnation of the most childish superstitions. And the Jewish people to whom I gladly belong and with whose mentality I have a deep affinity have no different quality for me than all other people. As far as my experience goes, they are also no better than other human groups, although they are protected from the worst cancers by a lack of power. Otherwise I cannot see anything "chosen" about them.
Link to article at Slate.com
Home buying practices adjust to high gas prices
Stroud's choice represents a fundamental shift in the way more Americans are approaching home buying in this era of ballooning gas prices. Real estate agents, transportation officials and industry surveys indicate that home buyers are placing more importance on cutting their gas bills and commute times than they have since the oil shocks of the 1970s.
And there are some early indications that homes near urban centers, and subway, train and bus stops are often selling faster and at better prices than those in the distant suburbs.
Gas prices, which have shot up $1.07 this year, are magnifying demographic trends that show more younger buyers and empty-nest seniors are moving back to urban centers. If gas prices continue their ascent, this could have profound consequences over time on the future development of American cities and suburbs and modes of transportation.
Homes in cities and neighborhoods that require long commutes and don't provide enough public transportation alternatives are falling in value more quickly than more central locations, according to a May study by CEOs for Cities, a network of U.S. urban leaders.
Tuesday, June 17, 2008
Monday, June 16, 2008
How to Pack Everything You Own Into One Suitcase
Doug Dyment, whose Web site onebag.com is devoted to the art of traveling light, says the key is to make a list in advance of what to pack and stick with it. He has developed a master list over the years that people can use as a starting point for creating their own.
"If it's not on your list, it shouldn't be in your bag," Dyment tells NPR's Michele Norris. "What happens with people is that they pack before their trip, and that packing activity consists mostly of talking to yourself and saying, 'Well I might need this and I might need that and what if the queen invites me to dinner?' And that's death to light packing."
Dyment advises people to think of what their lists look like well before a trip — literally writing it down and then checking off each item.
Dyment has two big tricks for packing a bag correctly: Don't let any space go unused, and wrap your clothes in bundles. "If you're packing a pair of running shoes, say, don't forget there's a lot of space inside those shoes that you can use to pack stuff," he says. When it comes to clothing, Dyment says travelers who fold items individually, put them in a stack and force them in the suitcase are making a huge mistake. Instead, he suggests using a technique called bundle
wrapping, because it keeps clothes from getting wrinkled and takes up less space.
"Never take more than two pairs of shoes," Dyment says. "In lots of business situations these days, you can buy shoes that are quite dressy looking and yet their internal construction is more like a high-quality running shoe."
Saturday, June 14, 2008
A Guide to Creating a Minimalist Home
Link to Article at Zenhabits.com
Executive summary
- Definition of a minimalist home
- Minimal furniture, clear surfaces (few knicknacks), accent decorations, quality over quantity
- Benefits of a minimalist home
- Less stressful, more appealing, easier to clean
- How to create a minimalist home
- One room at a time, start with furniture, only the essentials, clear floors, clear surfaces, clear walls, store stuff out of sight, declutter, simple artwork, simple decorations, plain window treatments, plain patterns, subdued colors, edit and eliminate, place for everything
How to Live With Just 100 Things
David Bruno's Blog (100 Things was his idea)
From David's blog:
My buddy Todd summed it up better than I could today when we were talking about it. He said, "Things are to be used. People are to be loved." The crazy thing about our consumer culture is that we so often reverse it. We use people to get the things we think we'll love. How stupid. As if fancy cars or more shoes are really going to satisfy us more than a great friend or a close relationship with our children.
Why am I doing the 100 Thing Challenge? Because I want to challenge stuff! I believe that run-away consumerism is making many of us narcissistic jackasses. It dulls our wits. Keeps us from thinking and acting like we understand what's really important. I'm planning on writing much more about this point. The main thing to remember now is that stuff is not passive. Stuff wants your time, attention, allegiance. But you know it as well as I do, life is more important than the things we accumulate. Challenge stuff!
Wednesday, June 11, 2008
Quotes from Orson Scott Card's Ender series
- Xenocide
So he believed. Believed, but the seed of doubt was there, and it stayed, and every now and then sent out a little root. It changed everything, to have that seed growing. It made [him] listen more
carefully to what people meant, instead of what they said. It made him wise.
- Ender's Game
He was commander every moment they were together. He never had to remind them of it; he simply was.
- Ender's Game
You can't rule out the impossible, because you never know which of your assumptions about what was possible might turn out, in the real universe, to be false.
- Ender's Shadow
For we humans do, when the cause is sufficient, spend our own lives. We throw ourselves onto the grenade to save our buddies in the foxhole. We rise out of the trenches and charge the enemy and die like maggots under a blowtorch. We strap bombs on our bodies and blow ourselves up in the midst of our enemies. We are, when the cause is sufficient, insane. He pretended all this time that humans were rational beings, when we are really the most terrible monsters these poor creatures could ever have conceived of in their nightmares. They had no way of knowing the story of blind Samson, who pulled down the temple on his own head to slay his enemies.
- Ender's Shadow
Is the Internet making us stupid?
The article says "Google", but the more accurate word is "Internet". The massive amount of information available via search, and the way that information on the web is broken into small chunks, is changing the way that we read, and therefore think.
Wolf worries that the style of reading promoted by the Net, a style that puts “efficiency” and “immediacy” above all else, may be weakening our capacity for the kind of deep reading that emerged when an earlier technology, the printing press, made long and complex works of prose commonplace. When we read online, she says, we tend to become “mere decoders of information.” Our ability to interpret text, to make the rich mental connections that form when we read deeply and without distraction, remains largely disengaged.The idea that our minds should operate as high-speed data-processing machines is not only built into the workings of the Internet, it is the network’s reigning business model as well. The faster we surf across the Web—the more links we click and pages we view—the more opportunities Google and other companies gain to collect information about us and to feed us advertisements. Most of the proprietors of the commercial Internet have a financial stake in collecting the crumbs of data we leave behind as we flit from link to link—the more crumbs, the better. The last thing these companies want is to encourage leisurely reading or slow, concentrated thought. It’s in their economic interest to drive us to distraction.
That’s the essence of Kubrick’s dark prophecy: as we come to rely on computers to mediate our understanding of the world, it is our own intelligence that flattens into artificial intelligence.
Tuesday, June 10, 2008
This is the man who wants to be our President?
Some of McCain’s acquaintances are less forgiving, however. They portray the politician as a self-centred womaniser who effectively abandoned his crippled wife to ‘play the field’. They accuse him of finally settling on Cindy, a former rodeo beauty queen, for financial reasons.
Ted Sampley, who fought with US Special Forces in Vietnam and is now a leading campaigner for veterans’ rights, said: ‘I have been following John McCain’s career for nearly 20 years. I know him personally. There is something wrong with this guy and let me tell you what it is – deceit.
‘When he came home and saw that Carol was not the beauty he left behind, he started running around on her almost right away. Everybody around him knew it.
‘Eventually he met Cindy and she was young and beautiful and very wealthy. At that point McCain just dumped Carol for something he thought was better.
‘This is a guy who makes such a big deal about his character. He has no character. He is a fake. If there was any character in that first marriage, it all belonged to Carol.’
Irony - When Pro-Life Women Get Abortions
I'm all for freedom of thought and respecting opinions other than my own. However, when people profess one opinion and behave in a diametrically opposite way, I find it difficult to take their opinions seriously.
How to Get Rid of Ants without Pesticides
Four strategies:
1) Cornmeal (ants try to eat, can't digest, starve)
2) Vinegar (ants don't like the smell, temporary)
3) Boric acid (toxic to ants, safe for environment)
4) Boiling water on the nest (if you can find it)
Monday, June 9, 2008
Rational consumer behavior emerges at $4 per gallon
At $3 a gallon, Americans just grin and bear it, suck it up and, while complaining profusely, keep driving like crazy. At $4, it is a world transformed. Americans become rational creatures. Mass transit ridership is at a 50-year high. Driving is down 4 percent. (Any U.S. decline is something close to a miracle.) Hybrids and compacts are flying off the lots. SUV sales are in free fall.
At $4 a gallon, the fleet composition is changing spontaneously and overnight, not over the 13 years mandated by Congress. (Even Stalin had the modesty to restrict himself to five-year plans.) Just Tuesday, GM announced that it would shutter four SUV and truck plants, add a third shift to its compact and midsize sedan plants in Ohio and Michigan, and green-light for 2010 the Chevy Volt, an electric hybrid.
AutoNation CEO wants gas to stay above $4 per gallon
It's entirely possible that a decade from now, we'll realize that this was a pivotal moment in the auto industry's history. This could be the moment when a century of relying almost exclusively on petroleum to power personal mobility gives way to a new model, in which electricity powers our transportation.
A gaggle of small companies such as Norway's Think Global AS and Silicon Valley's Tesla Motors Inc. are all gearing up to expand the electric vehicle market if the big guys won't. But the excitement over projects like the Tesla Roadster can't compare to the significance of the shift in mindset among the people who run the world's biggest auto companies. This isn't a crowd given to green idealism, but they have come to the conclusion that remaining totally shackled to petroleum is bad for business and are re-gearing their future vehicle plans accordingly.