Article at The Atlantic
Behind Door #1 are people of extraordinary ability: scientists, artists, educators, business people and athletes. Behind Door #2 stand a random assortment of people. Which door should the United States open?
In 2010, the United States more often chose Door #2, setting aside about 40,000 visas for people of extraordinary ability and 55,000 for people randomly chosen by lottery.
It's just one small example of our bizarre U.S. policy toward high-skill immigrants.
Showing posts with label Articles. Show all posts
Showing posts with label Articles. Show all posts
Tuesday, December 20, 2011
Thursday, December 15, 2011
The Overjustification Effect
Blog post at You Are Not So Smart
Related to intrinsic versus extrinsic motivation
According to the research, in modern America the average income required to be happy day-to-day, to experience “emotional well being” is about $75,000 a year. According to the researchers, past that point adding more to your income “does nothing for happiness, enjoyment, sadness, or stress.”
In 1980, David Rosenfield, Robert Folger and Harold Adelman at Southern Methodist University revealed a way you can defeat the overjustification effect. Seek employers who dole out reward – paychecks, bonuses, promotions, etc. – based not on quotas or task completions but instead based on competence.
The results of the study suggested when you get rewarded based on how well you perform a task, as long as those reasons are made perfectly clear, rewards will generate that electric exuberance of intrinsic validation, and the higher the reward, the better the feeling and the more likely you will try harder in the future. On the other hand, if you are getting rewarded just for being a warm body, no matter how well you do your job, no matter what you achieve, the electric feeling is absent. In those conditions greater rewards don’t lead to more output, don’t encourage you to strive for greatness. Overall, the study suggested rewards don’t have motivational power unless they make you feel competent. Money alone doesn’t do that.
Related to intrinsic versus extrinsic motivation
According to the research, in modern America the average income required to be happy day-to-day, to experience “emotional well being” is about $75,000 a year. According to the researchers, past that point adding more to your income “does nothing for happiness, enjoyment, sadness, or stress.”
In 1980, David Rosenfield, Robert Folger and Harold Adelman at Southern Methodist University revealed a way you can defeat the overjustification effect. Seek employers who dole out reward – paychecks, bonuses, promotions, etc. – based not on quotas or task completions but instead based on competence.
The results of the study suggested when you get rewarded based on how well you perform a task, as long as those reasons are made perfectly clear, rewards will generate that electric exuberance of intrinsic validation, and the higher the reward, the better the feeling and the more likely you will try harder in the future. On the other hand, if you are getting rewarded just for being a warm body, no matter how well you do your job, no matter what you achieve, the electric feeling is absent. In those conditions greater rewards don’t lead to more output, don’t encourage you to strive for greatness. Overall, the study suggested rewards don’t have motivational power unless they make you feel competent. Money alone doesn’t do that.
Monday, September 5, 2011
Teach practical math
Article at the New York Times
A math curriculum that focused on real-life problems would still expose students to the abstract tools of mathematics, especially the manipulation of unknown quantities. But there is a world of difference between teaching “pure” math, with no context, and teaching relevant problems that will lead students to appreciate how a mathematical formula models and clarifies real-world situations.
Imagine replacing the sequence of algebra, geometry and calculus with a sequence of finance, data and basic engineering. In the finance course, students would learn the exponential function, use formulas in spreadsheets and study the budgets of people, companies and governments. In the data course, students would gather their own data sets and learn how, in fields as diverse as sports and medicine, larger samples give better estimates of averages. In the basic engineering course, students would learn the workings of engines, sound waves, TV signals and computers. Science and math were originally discovered together, and they are best learned together now.
A math curriculum that focused on real-life problems would still expose students to the abstract tools of mathematics, especially the manipulation of unknown quantities. But there is a world of difference between teaching “pure” math, with no context, and teaching relevant problems that will lead students to appreciate how a mathematical formula models and clarifies real-world situations.
Imagine replacing the sequence of algebra, geometry and calculus with a sequence of finance, data and basic engineering. In the finance course, students would learn the exponential function, use formulas in spreadsheets and study the budgets of people, companies and governments. In the data course, students would gather their own data sets and learn how, in fields as diverse as sports and medicine, larger samples give better estimates of averages. In the basic engineering course, students would learn the workings of engines, sound waves, TV signals and computers. Science and math were originally discovered together, and they are best learned together now.
Wednesday, October 13, 2010
Watching "the game"
Something of a misnomer...
Football (article from the Wall Street Journal)
According to a Wall Street Journal study of four recent broadcasts, and similar estimates by researchers, the average amount of time the ball is in play on the field during an NFL game is about 11 minutes.
The typical length of a broadcast is 185 minutes, making the actual game ~6% of your typical broadcast.
Baseball (article from the Wall Street Journal)
A similar study of two nine-inning baseball games, one from Fox and another from ESPN.
The result is that during these games, there was a nearly identical amount of action: about 14 minutes. To put that in context, that's about 10.9% of the total broadcast time (excluding commercials).
Add in commercials, and the proportion drops even lower.
Reminds me of the "human" body, where human cells are outnumbered 10 to 1 by bacteria.
Football (article from the Wall Street Journal)
According to a Wall Street Journal study of four recent broadcasts, and similar estimates by researchers, the average amount of time the ball is in play on the field during an NFL game is about 11 minutes.
The typical length of a broadcast is 185 minutes, making the actual game ~6% of your typical broadcast.
Baseball (article from the Wall Street Journal)
A similar study of two nine-inning baseball games, one from Fox and another from ESPN.
The result is that during these games, there was a nearly identical amount of action: about 14 minutes. To put that in context, that's about 10.9% of the total broadcast time (excluding commercials).
Add in commercials, and the proportion drops even lower.
Reminds me of the "human" body, where human cells are outnumbered 10 to 1 by bacteria.
Monday, August 2, 2010
The value of authenticity
Article at Wired.com
If I had access to a secret stash of iPhone knockoffs — a phone that worked identically to the real iPhone, but was a bootleg made of inauthentic parts — how much could I charge? Could I sell them for $10 less than the purchase price of a real iPhone? What about 25 percent off? How much is authenticity worth?
This is a great summary line:
There are many blankets in the world. But there is only one blankie. The best brands are blankies.
Wednesday, March 3, 2010
Copying Creates Trends
Post at the Freakonomics blog about Oscar fashions
“Ten minutes after any big awards telecast, the Faviana design team is already working on our newest ‘celebrity look-alike gowns,’” says Faviana CEO Omid Moradi.
The existence of firms like Faviana (or ABS, Promgirl, or any of a number of similar houses) raises fascinating questions about intellectual property. First, how can Faviana get away with blatantly copying a dress that someone else has designed? And second, why doesn’t this rampant and very rapid copying destroy the fashion industry?
The ability of a firm like Faviana to copy a dress means that hot designs spread rapidly, and trends rise and fall. Copying helps to create trends. It then helps to destroy them: as more and more designers hop on to a trend, the look becomes overdone, and the most fashion-forward consumers hop off. Copying, in other words, accelerates the fashion cycle.
“Ten minutes after any big awards telecast, the Faviana design team is already working on our newest ‘celebrity look-alike gowns,’” says Faviana CEO Omid Moradi.
The existence of firms like Faviana (or ABS, Promgirl, or any of a number of similar houses) raises fascinating questions about intellectual property. First, how can Faviana get away with blatantly copying a dress that someone else has designed? And second, why doesn’t this rampant and very rapid copying destroy the fashion industry?
The ability of a firm like Faviana to copy a dress means that hot designs spread rapidly, and trends rise and fall. Copying helps to create trends. It then helps to destroy them: as more and more designers hop on to a trend, the look becomes overdone, and the most fashion-forward consumers hop off. Copying, in other words, accelerates the fashion cycle.
Tuesday, February 23, 2010
Data-Driven, rather than Hypothesis-Driven
Article about Google's reliance on data at Wired.com
Instead of using a semantic framework to build up a theory of language, Google mines its massive trove of data to find contextual word associations.
As Google crawled and archived billions of documents and Web pages, it analyzed what words were close to each other... "Today, if you type 'Gandhi bio,' we know that bio means biography," Singhal says. "And if you type 'bio warfare,' it means biological."
Want to introduce a new feature? Forget focus groups or relying on management to make decisions, run experiments on actual users!
But Google also has a larger army of testers — its billions of users, virtually all of whom are unwittingly participating in its constant quality experiments. Every time engineers want to test a tweak, they run the new algorithm on a tiny percentage of random users, letting the rest of the site’s searchers serve as a massive control group.
Blog post about data-driven versus hypothesis-driven science
The new data-driven approach suggests that we collect data first, then see what it tells us.
More info can be seen at a previous blog post.
Instead of using a semantic framework to build up a theory of language, Google mines its massive trove of data to find contextual word associations.
As Google crawled and archived billions of documents and Web pages, it analyzed what words were close to each other... "Today, if you type 'Gandhi bio,' we know that bio means biography," Singhal says. "And if you type 'bio warfare,' it means biological."
Want to introduce a new feature? Forget focus groups or relying on management to make decisions, run experiments on actual users!
But Google also has a larger army of testers — its billions of users, virtually all of whom are unwittingly participating in its constant quality experiments. Every time engineers want to test a tweak, they run the new algorithm on a tiny percentage of random users, letting the rest of the site’s searchers serve as a massive control group.
Blog post about data-driven versus hypothesis-driven science
The new data-driven approach suggests that we collect data first, then see what it tells us.
More info can be seen at a previous blog post.
Compressed Sensing
Article at Wired.com
Compressed sensing works something like this: You’ve got a picture — of a kidney, of the president, doesn’t matter. The picture is made of 1 million pixels. In traditional imaging, that’s a million measurements you have to make. In compressed sensing, you measure only a small fraction — say, 100,000 pixels randomly selected from various parts of the image. From that starting point there is a gigantic, effectively infinite number of ways the remaining 900,000 pixels could be filled in.
The key to finding the single correct representation is a notion called sparsity, a mathematical way of describing an image’s complexity, or lack thereof. A picture made up of a few simple, understandable elements — like solid blocks of color or wiggly lines — is sparse; a screenful of random, chaotic dots is not. It turns out that out of all the bazillion possible reconstructions, the simplest, or sparsest, image is almost always the right one or very close to it.
This question really highlighted the utility of compressed sensing:
Digital cameras, he explains, gather huge amounts of information and then compress the images. But compression, at least if CS is available, is a gigantic waste. If your camera is going to record a vast amount of data only to throw away 90 percent of it when you compress, why not just save battery power and memory and record 90 percent less data in the first place?
Compressed sensing works something like this: You’ve got a picture — of a kidney, of the president, doesn’t matter. The picture is made of 1 million pixels. In traditional imaging, that’s a million measurements you have to make. In compressed sensing, you measure only a small fraction — say, 100,000 pixels randomly selected from various parts of the image. From that starting point there is a gigantic, effectively infinite number of ways the remaining 900,000 pixels could be filled in.
The key to finding the single correct representation is a notion called sparsity, a mathematical way of describing an image’s complexity, or lack thereof. A picture made up of a few simple, understandable elements — like solid blocks of color or wiggly lines — is sparse; a screenful of random, chaotic dots is not. It turns out that out of all the bazillion possible reconstructions, the simplest, or sparsest, image is almost always the right one or very close to it.
This question really highlighted the utility of compressed sensing:
Digital cameras, he explains, gather huge amounts of information and then compress the images. But compression, at least if CS is available, is a gigantic waste. If your camera is going to record a vast amount of data only to throw away 90 percent of it when you compress, why not just save battery power and memory and record 90 percent less data in the first place?
Tuesday, February 2, 2010
Slime Mold vs Traffic Planners
Freakonomics article
MSNBC article
A very elegant experimental setup:
The scientists placed food deposits (oat flakes) in a pattern that mimicked the distribution of population in the greater Tokyo area. They also discouraged mold growth in areas corresponding to obstacles like ocean and mountains by placing light sources (mold’s sworn enemy) in these spots. The researchers then introduced a single deposit of the mold on their mock central Tokyo and let the slime do its thing.
The result? The mold formed a network that closely mimicked the actual Tokyo railway map. In terms of efficiency and fault tolerance, the mold performed about the same as the real Tokyo system, and it did so at a slightly lower cost.
Conclusion?
Organic development can complement the planning efforts of a central intelligence. Planners see the big picture, but may have limited information about the small details. Organic planning accumulates the collective wisdom of myriad individuals who each know only a very small part of the picture, but know their part very well.
An interesting thought experiment: think about how this describes how a company functions. Are there any lessons that companies (or any other group effort) can learn from slime molds?
MSNBC article
A very elegant experimental setup:
The scientists placed food deposits (oat flakes) in a pattern that mimicked the distribution of population in the greater Tokyo area. They also discouraged mold growth in areas corresponding to obstacles like ocean and mountains by placing light sources (mold’s sworn enemy) in these spots. The researchers then introduced a single deposit of the mold on their mock central Tokyo and let the slime do its thing.
The result? The mold formed a network that closely mimicked the actual Tokyo railway map. In terms of efficiency and fault tolerance, the mold performed about the same as the real Tokyo system, and it did so at a slightly lower cost.
Conclusion?
Organic development can complement the planning efforts of a central intelligence. Planners see the big picture, but may have limited information about the small details. Organic planning accumulates the collective wisdom of myriad individuals who each know only a very small part of the picture, but know their part very well.
An interesting thought experiment: think about how this describes how a company functions. Are there any lessons that companies (or any other group effort) can learn from slime molds?
Monday, January 25, 2010
Sociology of cooperative video games
Et tu, Mario?
I thought the Contra example was hilarious - I totally remember that feeling as well.
Most cooperative games lie in a vast middle ground, however, a no man's land between altruism and gaming Darwinism that offers up a host of ways to misbehave.
Part of the problem (and the joy) of playing games is that such behavior isn't explicitly condoned or condemned. Looting and friendly fire aren't forbidden by most games, which leaves us to figure out our own rules. This is the right decision: Good game designers allow players to be whoever they want and trust they'll come to their own consensus about what constitutes "fair play." That's why the New Super Mario Bros. Wii was more enjoyable when I played it as God intended—with a good friend and copious amounts of beer. There was no back-stabbing, and no one's feelings were hurt.
I thought the Contra example was hilarious - I totally remember that feeling as well.
Most cooperative games lie in a vast middle ground, however, a no man's land between altruism and gaming Darwinism that offers up a host of ways to misbehave.
Part of the problem (and the joy) of playing games is that such behavior isn't explicitly condoned or condemned. Looting and friendly fire aren't forbidden by most games, which leaves us to figure out our own rules. This is the right decision: Good game designers allow players to be whoever they want and trust they'll come to their own consensus about what constitutes "fair play." That's why the New Super Mario Bros. Wii was more enjoyable when I played it as God intended—with a good friend and copious amounts of beer. There was no back-stabbing, and no one's feelings were hurt.
Monday, September 14, 2009
"We are the glue holding ourselves together."
Wired.com article
I wouldn't say that the article itself is particularly well-written, but certainly the insights revealed by the study of the Framingham papers are quite interesting.
By studying Framingham as an interconnected network rather than a mass of individuals, Christakis and Fowler made a remarkable discovery: Obesity spread like a virus. Weight gain had a stunning infection rate. If one person became obese, the likelihood that his friend would follow suit increased by 171 percent. (This means that the network is far more predictive of obesity than the presence of genes associated with the condition.)
It has long been recognized, for instance, that the human capacity for close friendship is remarkably consistent. People from cultures throughout the world report between four and seven bosom buddies. "The properties of our social networks are byproducts of evolution," Christakis says. "The assumption has been that our mind can handle only so many other people."
After analyzing thousands of photos, the scientists found that, on average, each student had 6.6 close friends in their online network. In other words, nothing has really changed; even the most fervent Facebook users still maintain only a limited circle of intimates.
Because networks transmit the stuff of life—from happiness to HIV—evolution has generated a diversity of personality traits, which take advantage of different positions within the group. There are wallflowers and Wilt Chamberlains, shy geeks and "super-connectors." According to Christakis and Fowler, there is no single solution to the problem of other people. Individual variation is a crucial element of every stable community, from the Aborigines of Australia to the avatars of Second Life.
Wednesday, July 22, 2009
Happiness
Eric Wiener, on Rick Steve's show, as blogged by GetRichSlowly
There have been studies that show that people are materialistic — irrespective of how much money they actually have — people who are materialistic tend to be less happy than people who are not.
Close relationships are a better predictor of happiness than monetary wealth. “Happiness is other people,” Weiner says. “Our happiness is determined in large part by our quality and quantity of relationships with others.”
Let’s talk about Denmark, for instance, because Denmark ranks consistently in the top three for happiest countries in the world. The Danes have low expectations. In survey after survey, they’re asked about expectations, and they have relatively low expectations. We Americans have very very high expectations. And I think that partly explains the discrepancy.
I think if you have low or moderate expectations, you’re less likely to be disappointed. You’re more likely to be satisfied or content. You’re more likely to be happy.
There have been studies that show that people are materialistic — irrespective of how much money they actually have — people who are materialistic tend to be less happy than people who are not.
Close relationships are a better predictor of happiness than monetary wealth. “Happiness is other people,” Weiner says. “Our happiness is determined in large part by our quality and quantity of relationships with others.”
Let’s talk about Denmark, for instance, because Denmark ranks consistently in the top three for happiest countries in the world. The Danes have low expectations. In survey after survey, they’re asked about expectations, and they have relatively low expectations. We Americans have very very high expectations. And I think that partly explains the discrepancy.
I think if you have low or moderate expectations, you’re less likely to be disappointed. You’re more likely to be satisfied or content. You’re more likely to be happy.
Tuesday, June 2, 2009
Just Don't Look
Do I want to hear about the latest sob story on the nightly news?
Do I want to know what J. Lo or Brad Pitt or [insert celebrity of choice] is up to?
Do I care about the latest weight-loss approach? (Seriously, just burn more calories than you eat, or eat less calories than you burn - how hard is that?)
This post at Coding Horror, which references this post at kottke.org, describes an approach to dealing with things/people that run on attention - Just Don't Look.
That's how you change the world. Not by arguing with people. Certainly not by screaming at them. You do it by ignoring them.
Do I want to know what J. Lo or Brad Pitt or [insert celebrity of choice] is up to?
Do I care about the latest weight-loss approach? (Seriously, just burn more calories than you eat, or eat less calories than you burn - how hard is that?)
This post at Coding Horror, which references this post at kottke.org, describes an approach to dealing with things/people that run on attention - Just Don't Look.
That's how you change the world. Not by arguing with people. Certainly not by screaming at them. You do it by ignoring them.
Monday, June 1, 2009
How to defuse a "Screw-Me" moment? (Hint - "Spin" is the wrong answer)
Blog post at Rands in Repose
Manage the room. Questions aren’t Screw-mes. You can clarify and stay on track. You know that Amanda is going to ask about hard data, right? Don’t let her take over the conversation. Say, “I’ve got your data in the appendix, but let me get through this first, ok?” Yeah, you just shut down a Senior VP. Nicely done. No way you can do that without serious confidence in your preparation. Yes, Tim?
Tim’s got the Screw-Me and you didn’t see it coming. Total left field. Completely valid strategic observation and you don’t have a clue how to answer. Shit.
You will recognize the Screw-Me by the complete silence that fills both the room and your head. That’s the realization everyone is having that you’re Screwed. First, let’s not make it worse…
Tim: “Rands, what about THIS?”
I’m a poker player and an experienced meeting surfer, so the room will not immediately know from the look on my face that This has Screwed me, but what I choose to do next will define my ongoing relationship with the room.
There are two options when you are cornered by This. Your animal brain, when cornered, will try to find a way out. You can taste this approach even before you begin. I am going to spin. I am going to talk quickly and confidently about This and I am going to hope that in my furious verbal scurrying they are going to believe I’ve got This handled.
That’s not what they’re seeing or hearing.
This is not your staff meeting where a little verbal soft shoe is going to entertain and delight. These are the execs and no matter how many meetings you’ve surfed, they see straight through spin, they know this dance, and the longer you sit there spinning, the longer you give your boss an opportunity to step in, try to make the diving save, and make you look like a blithering fool.
It takes a little practice to make the correct move when you feel the spin coming. You are going to do three things:
Manage the room. Questions aren’t Screw-mes. You can clarify and stay on track. You know that Amanda is going to ask about hard data, right? Don’t let her take over the conversation. Say, “I’ve got your data in the appendix, but let me get through this first, ok?” Yeah, you just shut down a Senior VP. Nicely done. No way you can do that without serious confidence in your preparation. Yes, Tim?
Tim’s got the Screw-Me and you didn’t see it coming. Total left field. Completely valid strategic observation and you don’t have a clue how to answer. Shit.
You will recognize the Screw-Me by the complete silence that fills both the room and your head. That’s the realization everyone is having that you’re Screwed. First, let’s not make it worse…
Tim: “Rands, what about THIS?”
I’m a poker player and an experienced meeting surfer, so the room will not immediately know from the look on my face that This has Screwed me, but what I choose to do next will define my ongoing relationship with the room.
There are two options when you are cornered by This. Your animal brain, when cornered, will try to find a way out. You can taste this approach even before you begin. I am going to spin. I am going to talk quickly and confidently about This and I am going to hope that in my furious verbal scurrying they are going to believe I’ve got This handled.
That’s not what they’re seeing or hearing.
This is not your staff meeting where a little verbal soft shoe is going to entertain and delight. These are the execs and no matter how many meetings you’ve surfed, they see straight through spin, they know this dance, and the longer you sit there spinning, the longer you give your boss an opportunity to step in, try to make the diving save, and make you look like a blithering fool.
It takes a little practice to make the correct move when you feel the spin coming. You are going to do three things:
- Acknowledge the Screw-Me.
- Admit “I don’t know.”
- Concretely explain the steps you’re going to take to find out and give yourself a deadline.
Wednesday, May 27, 2009
3 Skills of Sexy Data Geeks
Blog post at Dataspora
Skill #1: Statistics (Studying). Statistics is perhaps the most important skill and the hardest to learn. It’s a deep and rigorous discipline, and one that is actively progressing (the widely used method of Least Angle Regression was only recently developed in 2004).
Skill #2: Data Munging (Suffering). The second critical skill mentioned above is “data munging.” Among data geek circles, this refers to the painful process of cleaning, parsing, and proofing one’s data before it’s suitable for analysis. Real world data is messy. At best it’s inconsistently delimited or packed into an unnecessarily complex XML schema. At worst, it’s a series of scraped HTML pages or a thoroughly undocumented fixed-width format.
Related to munging but certainly far less painful is the ability to retrieve, slice, and dice well-structured data from persistent data stores, using a combination of SQL, scripting languages (especially Python and its SciPy and NumPy libraries), and even several oldie-but-goodie Unix utilities (cut, join).
And when data sets grow too large to manage on a single desktop, the samurai of data geeks are capable of parallelizing storage and computation with tools like 96-nodes of Postgres, snow and RMPI, Hadoop and Mapreduce, and on Amazon EC2 to boot.
Skill #3: Visualization (Storytelling). This third and last skill that Professor Varian refers to is the easiest to believe one has. Most of us have had exposure to basic chart-making widgets of Excel. But a little knowledge is a dangerous thing: these software tools are often insufficient when faced with the visualization of large, multivariate data sets.
Here it’s worth making a distinction between two breeds of data visualizations, which differ in their audience and their goals. The first are exploratory data visualizations (as named by John Tukey), intended to faciliate a data analyst’s understanding of the data. These may consist of scatter plot matrices and histograms, where labels and colors are minimally set by default. Their goal is to help develop a hypothesis about the data, and their audience typically numbers one.
A second kind of data visualization are those intended to communicate to a wider audience, whose goal is to visually advocate for a hypothesis. While most data geeks are facile with exploratory graphics, the ability to create this second kind of visualization, these visual narratives, is again a separate skill — with separate tools.
The ability to visualize and communicate data is critical, because even with good data and rigorous statistical techniques, if the results of an analysis are poorly visualized, they will not convince: whether it’s an academic discovery or a business proposal.
Skill #1: Statistics (Studying). Statistics is perhaps the most important skill and the hardest to learn. It’s a deep and rigorous discipline, and one that is actively progressing (the widely used method of Least Angle Regression was only recently developed in 2004).
Skill #2: Data Munging (Suffering). The second critical skill mentioned above is “data munging.” Among data geek circles, this refers to the painful process of cleaning, parsing, and proofing one’s data before it’s suitable for analysis. Real world data is messy. At best it’s inconsistently delimited or packed into an unnecessarily complex XML schema. At worst, it’s a series of scraped HTML pages or a thoroughly undocumented fixed-width format.
Related to munging but certainly far less painful is the ability to retrieve, slice, and dice well-structured data from persistent data stores, using a combination of SQL, scripting languages (especially Python and its SciPy and NumPy libraries), and even several oldie-but-goodie Unix utilities (cut, join).
And when data sets grow too large to manage on a single desktop, the samurai of data geeks are capable of parallelizing storage and computation with tools like 96-nodes of Postgres, snow and RMPI, Hadoop and Mapreduce, and on Amazon EC2 to boot.
Skill #3: Visualization (Storytelling). This third and last skill that Professor Varian refers to is the easiest to believe one has. Most of us have had exposure to basic chart-making widgets of Excel. But a little knowledge is a dangerous thing: these software tools are often insufficient when faced with the visualization of large, multivariate data sets.
Here it’s worth making a distinction between two breeds of data visualizations, which differ in their audience and their goals. The first are exploratory data visualizations (as named by John Tukey), intended to faciliate a data analyst’s understanding of the data. These may consist of scatter plot matrices and histograms, where labels and colors are minimally set by default. Their goal is to help develop a hypothesis about the data, and their audience typically numbers one.
A second kind of data visualization are those intended to communicate to a wider audience, whose goal is to visually advocate for a hypothesis. While most data geeks are facile with exploratory graphics, the ability to create this second kind of visualization, these visual narratives, is again a separate skill — with separate tools.
The ability to visualize and communicate data is critical, because even with good data and rigorous statistical techniques, if the results of an analysis are poorly visualized, they will not convince: whether it’s an academic discovery or a business proposal.
Tuesday, May 26, 2009
Rise of the Startups
My gut feeling has been that history is cyclical, generally speaking, and that true revolutionary change is highly unlikely in the long term. Little did I know that there is an incredible amount of information on this topic just a Google search away.
Wikipedia article on "Social cycle theory"
Anyways, the following excerpt from this Wired.com article about the "new new economy" caught my attention:
Wikipedia article on "Social cycle theory"
Anyways, the following excerpt from this Wired.com article about the "new new economy" caught my attention:
As venture capitalist Paul Graham put it, "It turns out the rule 'large and disciplined organizations win' needs to have a qualification appended: 'at games that change slowly.' No one knew till change reached a sufficient speed."The article points out the recent decline of large corporations and points out their disadvantages in our current economy, while highlighting the strengths of small, nimble startups. The article also links to an article by Paul Graham, expanding on the whole "old versus new" debate.
The result is that the next new economy, the one rising from the ashes of this latest meltdown, will favor the small.
But in the late twentieth century something changed. It turned out that economies of scale were not the only force at work. Particularly in technology, the increase in speed one could get from smaller groups started to trump the advantages of size.Reading that article led to another one, which talks about the declining importance of credentials and how startups are much more meritocratic - nobody cares where you went to school or who your parents are, all that matters is your performance.
Large organizations will start to do worse now, though, because for the first time in history they're no longer getting the best people. An ambitious kid graduating from college now doesn't want to work for a big company. They want to work for the hot startup that's rapidly growing into one. If they're really ambitious, they want to start it.
History suggests that, all other things being equal, a society prospers in proportion to its ability to prevent parents from influencing their children's success directly. It's a fine thing for parents to help their children indirectly—for example, by helping them to become smarter or more disciplined, which then makes them more successful. The problem comes when parents use direct methods: when they are able to use their own wealth or power as a substitute for their children's qualities.
Large organizations can't do this. But a bunch of small organizations in a market can come close. A market takes every organization and keeps just the good ones. As organizations get smaller, this approaches taking every person and keeping just the good ones. So all other things being equal, a society consisting of more, smaller organizations will care less about credentials.
Googlenomics and Auctions
Wired.com article about Hal Varian, Google's Chief Economist
Varian believes that a new era is dawning for what you might call the datarati—and it's all about harnessing supply and demand. "What's ubiquitous and cheap?" Varian asks. "Data." And what is scarce? The analytic ability to utilize that data. As a result, he believes that the kind of technical person who once would have wound up working for a hedge fund on Wall Street will now work at a firm whose business hinges on making smart, daring choices—decisions based on surprising results gleaned from algorithmic spelunking and executed with the confidence that comes from really doing the math.
This is an example of a disruptive innovation - a gutsy move for Google, but one that ultimately paid off.
The problem with an all-at-once auction, however, was that advertisers might be inclined to lowball their bids to avoid the sucker's trap of paying a huge amount more than the guy just below them on the page. So the Googlers decided that the winner of each auction would pay the amount (plus a penny) of the bid from the advertiser with the next-highest offer. (If Joe bids $10, Alice bids $9, and Sue bids $6, Joe gets the top slot and pays $9.01. Alice gets the next slot for $6.01, and so on.) Since competitors didn't have to worry about costly overbidding errors, the paradoxical result was that it encouraged higher bids.
By turning over its sales process entirely to an auction-based system, the company could similarly upend the world of advertising, removing human guesswork from the equation.
The move was risky. Going ahead with the phaseout—nicknamed Premium Sunset—meant giving up campaigns that were selling for hundreds of thousands of dollars, for the unproven possibility that the auction process would generate even bigger sums. "We were going to erase a huge part of the company's revenue," says Tim Armstrong, then head of direct sales in the US. (This March, Armstrong left Google to become AOL's new chair and CEO.) "Ninety-nine percent of companies would have said, 'Hold on, don't make that change.' But we had Larry, Sergey, and Eric saying, 'Let's go for it.'"
The article asks if we can imagine using auctions in our everyday lives? Does this make our free market economy much more agile, responsive, and transparent? Take game consoles, for example. An auction-based system would very quickly determine the value of consoles, creating a true free market economy, rather than our current system of retail price management (RPM) agreements (sometimes called vertical price-fixing).
Google even uses auctions for internal operations, like allocating servers among its various business units. Since moving a product's storage and computation to a new data center is disruptive, engineers often put it off. "I suggested we run an auction similar to what the airlines do when they oversell a flight. They keep offering bigger vouchers until enough customers give up their seats," Varian says. "In our case, we offer more machines in exchange for moving to new servers. One group might do it for 50 new ones, another for 100, and another won't move unless we give them 300. So we give them to the lowest bidder—they get their extra capacity, and we get computation shifted to the new data center."
Varian believes that a new era is dawning for what you might call the datarati—and it's all about harnessing supply and demand. "What's ubiquitous and cheap?" Varian asks. "Data." And what is scarce? The analytic ability to utilize that data. As a result, he believes that the kind of technical person who once would have wound up working for a hedge fund on Wall Street will now work at a firm whose business hinges on making smart, daring choices—decisions based on surprising results gleaned from algorithmic spelunking and executed with the confidence that comes from really doing the math.
This is an example of a disruptive innovation - a gutsy move for Google, but one that ultimately paid off.
The problem with an all-at-once auction, however, was that advertisers might be inclined to lowball their bids to avoid the sucker's trap of paying a huge amount more than the guy just below them on the page. So the Googlers decided that the winner of each auction would pay the amount (plus a penny) of the bid from the advertiser with the next-highest offer. (If Joe bids $10, Alice bids $9, and Sue bids $6, Joe gets the top slot and pays $9.01. Alice gets the next slot for $6.01, and so on.) Since competitors didn't have to worry about costly overbidding errors, the paradoxical result was that it encouraged higher bids.
By turning over its sales process entirely to an auction-based system, the company could similarly upend the world of advertising, removing human guesswork from the equation.
The move was risky. Going ahead with the phaseout—nicknamed Premium Sunset—meant giving up campaigns that were selling for hundreds of thousands of dollars, for the unproven possibility that the auction process would generate even bigger sums. "We were going to erase a huge part of the company's revenue," says Tim Armstrong, then head of direct sales in the US. (This March, Armstrong left Google to become AOL's new chair and CEO.) "Ninety-nine percent of companies would have said, 'Hold on, don't make that change.' But we had Larry, Sergey, and Eric saying, 'Let's go for it.'"
The article asks if we can imagine using auctions in our everyday lives? Does this make our free market economy much more agile, responsive, and transparent? Take game consoles, for example. An auction-based system would very quickly determine the value of consoles, creating a true free market economy, rather than our current system of retail price management (RPM) agreements (sometimes called vertical price-fixing).
Google even uses auctions for internal operations, like allocating servers among its various business units. Since moving a product's storage and computation to a new data center is disruptive, engineers often put it off. "I suggested we run an auction similar to what the airlines do when they oversell a flight. They keep offering bigger vouchers until enough customers give up their seats," Varian says. "In our case, we offer more machines in exchange for moving to new servers. One group might do it for 50 new ones, another for 100, and another won't move unless we give them 300. So we give them to the lowest bidder—they get their extra capacity, and we get computation shifted to the new data center."
Let the Little Guys Drive (Disruptive Innovation)
Article at Wired.com
If a domestic auto industry is to survive, it will have to incorporate and encourage breakthroughs from outsiders like Transonic. Automakers will need to transition from a vertical, proprietary, hierarchical model to an open, modular, collaborative one, becoming central nodes in an entrepreneurial ecosystem. In other words, the industry will need to undergo much the same wrenching transformation that the US computer business did some three decades ago, when the minicomputer gave way to the personal computer. Whereas minicomputers were restricted to using mainly software and hardware from their makers, PCs used interchangeable elements that could be designed, manufactured, and installed by third parties. Opening the gates to outsiders unleashed a flood of innovation that gave rise to firms like Microsoft, Dell, and Oracle. It destroyed many of the old computer giants—but guaranteed a generation of American leadership in a critical sector of the world economy. It is late in the day, but the same could still happen in the car industry; it just has to harness our national entrepreneurial spirit to develop the next wave of auto breakthroughs.
By seeking to match the likes of Toyota, Detroit has been trying to come from behind in a game where its adversaries set the rules. To Klepper, the Carnegie Mellon economist, the Big Three today resemble the American television-receiver industry in the 1970s and 1980s, pioneered by US corporations that, after decades of domination, were suddenly confronted by foreign innovation. Companies like RCA and Zenith were slow to incorporate new technologies until it was too late; all exited or sold out to foreign firms. "Every time American companies catch up to the competition," Klepper says, "the competition already has moved on and instituted new things. In that situation, it's extremely difficult to get ahead."
The only escape from this conundrum is to pursue what Harvard Business School professor Clayton Christensen has called disruptive innovation—the kind of change that alters the trajectory of an industry. As Christensen argued in his 1997 book, The Innovator's Dilemma, successful companies in mature industries rarely embrace disruptive innovation because, by definition, it threatens their business models. Loath to revamp factories at high cost to make products that will compete with their own goods, companies drag their feet; perversely, financial markets often reward them for their shortsightedness. Good as they are, the European and Japanese automakers are established companies. At this point, they are as unlikely to pursue disruptive innovation as Detroit has been. That gives the US auto industry an opening. To take that opportunity, it will have to behave differently—it will have to step far outside the walls of the Rouge.
I very strongly believe in the idea of disruptive innovation. Other innovations that I consider disruptive are "netbooks" and casual gaming (i.e. PopCap Games). Getting sucked into a never-ending cycle of competition between established companies encourages incremental improvements, is reactionary, and ultimately drags down all players involved. Much better to break free and create a new paradigm/product.
If a domestic auto industry is to survive, it will have to incorporate and encourage breakthroughs from outsiders like Transonic. Automakers will need to transition from a vertical, proprietary, hierarchical model to an open, modular, collaborative one, becoming central nodes in an entrepreneurial ecosystem. In other words, the industry will need to undergo much the same wrenching transformation that the US computer business did some three decades ago, when the minicomputer gave way to the personal computer. Whereas minicomputers were restricted to using mainly software and hardware from their makers, PCs used interchangeable elements that could be designed, manufactured, and installed by third parties. Opening the gates to outsiders unleashed a flood of innovation that gave rise to firms like Microsoft, Dell, and Oracle. It destroyed many of the old computer giants—but guaranteed a generation of American leadership in a critical sector of the world economy. It is late in the day, but the same could still happen in the car industry; it just has to harness our national entrepreneurial spirit to develop the next wave of auto breakthroughs.
By seeking to match the likes of Toyota, Detroit has been trying to come from behind in a game where its adversaries set the rules. To Klepper, the Carnegie Mellon economist, the Big Three today resemble the American television-receiver industry in the 1970s and 1980s, pioneered by US corporations that, after decades of domination, were suddenly confronted by foreign innovation. Companies like RCA and Zenith were slow to incorporate new technologies until it was too late; all exited or sold out to foreign firms. "Every time American companies catch up to the competition," Klepper says, "the competition already has moved on and instituted new things. In that situation, it's extremely difficult to get ahead."
The only escape from this conundrum is to pursue what Harvard Business School professor Clayton Christensen has called disruptive innovation—the kind of change that alters the trajectory of an industry. As Christensen argued in his 1997 book, The Innovator's Dilemma, successful companies in mature industries rarely embrace disruptive innovation because, by definition, it threatens their business models. Loath to revamp factories at high cost to make products that will compete with their own goods, companies drag their feet; perversely, financial markets often reward them for their shortsightedness. Good as they are, the European and Japanese automakers are established companies. At this point, they are as unlikely to pursue disruptive innovation as Detroit has been. That gives the US auto industry an opening. To take that opportunity, it will have to behave differently—it will have to step far outside the walls of the Rouge.
I very strongly believe in the idea of disruptive innovation. Other innovations that I consider disruptive are "netbooks" and casual gaming (i.e. PopCap Games). Getting sucked into a never-ending cycle of competition between established companies encourages incremental improvements, is reactionary, and ultimately drags down all players involved. Much better to break free and create a new paradigm/product.
An interview with Tina Seelig
Posted by Guy Kawasaki at the OPEN Forum
Some excerpts I liked:
Question: What is the secret to successful negotiation?
Answer: Make sure that you understand the other person’s point of view. If you make assumptions, you will very likely be wrong. When I bought a car for my son. I assumed that the salesperson wanted us to pay the highest price. That wasn’t the case! After asking a bunch of questions, I learned that his commission wasn’t based on the price of the car—it was based on the scores he got on the customer evaluation form we filled out afterward. Of course, I was happy to give him a great score in return for a great price. This is how win-win negotiations come about.
Question: How does one balance work and “life”?
Answer: You copied a quote from my book into one of your recent blogs. That quote, attributed to the Chinese philosopher Lao-Tzu, is very powerful.
Some excerpts I liked:
Question: What is the secret to successful negotiation?
Answer: Make sure that you understand the other person’s point of view. If you make assumptions, you will very likely be wrong. When I bought a car for my son. I assumed that the salesperson wanted us to pay the highest price. That wasn’t the case! After asking a bunch of questions, I learned that his commission wasn’t based on the price of the car—it was based on the scores he got on the customer evaluation form we filled out afterward. Of course, I was happy to give him a great score in return for a great price. This is how win-win negotiations come about.
Question: How does one balance work and “life”?
Answer: You copied a quote from my book into one of your recent blogs. That quote, attributed to the Chinese philosopher Lao-Tzu, is very powerful.
“The master of the art of living makes little distinction between his work and his play, his labor and his leisure, his mind and his body, his education and his recreation, his love and his religion. He simply pursues his vision of excellence in whatever he does, leaving others to decide whether he is working or playing. To him, he is always doing both.”
Thursday, May 21, 2009
DinTaiFung
via Seth Godin's solicitation for updates to Purple Cow
Two things are guaranteed at the remarkable DinTaiFung restaurant in Taipei: the extremely long line outside and the size/weight of their world famous steamed juicy pork dumplings. Each dumpling uses only the freshest ingredients, weighs a precise 0.74 oz, and has exactly 18 folds. In 1993, NY Times named DinTaiFung as one of the top 10 restaurants in the world. Even with many outlets worldwide today, thousands of tourists still visit Taipei every year just to eat at its original location. One of the stories told about the restaurant owner is that he takes the tour buses to hear what people say about his restaurant. One day, he found that bus stopped before reaching its destination and tourists were encouraged to use the restrooms so that they can avoid using the ones at his restaurant. He went back and installed the most advanced toilets available in the restrooms and made sure that they were cleaned every 15 minutes. Since then, the restrooms at DinTaiFung also became one of the most talked about topics for tourists.
Two things are guaranteed at the remarkable DinTaiFung restaurant in Taipei: the extremely long line outside and the size/weight of their world famous steamed juicy pork dumplings. Each dumpling uses only the freshest ingredients, weighs a precise 0.74 oz, and has exactly 18 folds. In 1993, NY Times named DinTaiFung as one of the top 10 restaurants in the world. Even with many outlets worldwide today, thousands of tourists still visit Taipei every year just to eat at its original location. One of the stories told about the restaurant owner is that he takes the tour buses to hear what people say about his restaurant. One day, he found that bus stopped before reaching its destination and tourists were encouraged to use the restrooms so that they can avoid using the ones at his restaurant. He went back and installed the most advanced toilets available in the restrooms and made sure that they were cleaned every 15 minutes. Since then, the restrooms at DinTaiFung also became one of the most talked about topics for tourists.
Subscribe to:
Posts (Atom)
