The most famous Machine Learning MOOC of our time

If you haven't taken the Stanford's Machine Learning MOOC by Prof Andrew Ng on Coursera, you are less likely to be taken seriously in the AI community. Or so they say.

Somewhere in 2008 Andrew Ng started the Stanford Engineering Everywhere (SEE) program that placed a number of Stanford courses online, for free. Andrew himself was responsible for teaching one of these courses, Machine Learning, which consisted of video lectures by him, along with the student materials used in the Stanford CS229 class.

The "applied" version of the Stanford class (CS229a) was hosted on ml-class.org and started in October 2011, with over 100,000 students registered for its first iteration; and became one of the first successful MOOCs made by Stanford professors.

Andrew Ng and Daphne Koller left Stanford to co-found Coursera in 2012. The Machine Learning course was one of the key offerings on the platform. And it continues to be #1 (check here, here, or comprehensively here). 

Read More

Bubble-Game Theory

YOU CAN CONSIDER GOOGLE your friend only if the two of you play games with each other -- especially with Google the search box. I call our little game Bubble-game. The rule is simple. You need to come up with a vaguely familiar term that you know from somewhere -- desirably from within the Google Apps ecosystem that you personally use on various gadgets. Again, the only rule is that the term should be only vaguely familiar, if at all. It is not necessary to know the precise spelling.

So then, you turn to Google.com and ask. From within the context of your 'relationship' with Google, the algorithm would suggest to you the possible answers in the form of search results. And depending on how extensively you use Google --or, to put it more socially-- depending on how well Google 'knows' you, you should find traces in the search results that may indicate where you might have encountered the term for the first time and the subsequent info-branches it created thereafter: cached data, search queries, location information and frequently visited places, bookmarks and favourites, frequently visited sites, email and social circles, interactions and conversations you have had -- to mention a few. (For the complete list, you may want to review details in the public domain for project PRISM.)

If you have noticed, Google Now does something very similar albeit behind the scenes. Which in turn defines the bubble that you live and operate within inside a given app ecosystem. These informed results are algorithmically cultivated to "inform" you better. However, in the process, the algorithm assigns weights to certain information snippets to bump them up over others, and in doing so, it alters the reality for you.

It is my theory that over a period of time, pretty much like a chewing bubble gum one can effectively change the shape and size of this bubble. Since it was defined by your own habitual patterns in the first place, it can be redefined also. It would primarily involve controlling and altering one's digital information usage patterns around the given bubble. Typically, a bubble shrinks over time, making your behavior patterns more predictable. As you add milestones to your life such as acquiring a new degree, getting married, adding a newborn to the family, relocating to a new place, changing jobs, etc. would add additional dimensions and info-branching to the existing bubble. A significant effort may allow you to restrict the bubble from affecting your information consumption. However, there seems to be no way to burst the bubble unless the complete dataset is lost or disassociated with your digital identity.

Getting back to the Bubble-game, the term that Google and I played with today is "Rosenthal" (try here) -- a vaguely familiar term that randomly popped up in my head, most probably by unconsciously noticing Umberto Eco's book "The Name Of the Rose" on the bookshelf in the passing. The bubble involves a host of url's, bookmarks, comments, that I happened to capture a couple of years ago.

(PS: Eli Pariser demonstrated the bubble effect in his 2011 TED talk with striking examples. His ongoing research effort is updated on his personal blog - The Filter Bubble.)

Read More

George Sugihara On Early Warning Signs

Earlier this month SEED magazine published this very interesting article by George Sugihara, theoretical biologist at Scripps Institution of Oceanography, on how deep mathematical models tie the events of climat change, epileptic seizure, fishery collapses, and risk management surrounding the global financial crisis. Excerpts:

[...] Economics is not typically thought of as a global systems problem. Indeed, investment banks are famous for a brand of tunnel vision that focuses risk management at the individual firm level and ignores the difficult and costlier, albeit less frequent, systemic or financial-web problem. Monitoring the ecosystem-like network of firms with interlocking balance sheets is not in the risk manager’s job description.

A parallel situation exists in fisheries, where stocks are traditionally managed one species at a time. Alarm over collapsing fish stocks, however, is helping to create the current push for ecosystem-based ocean management. This is a step in the right direction, but the current ecosystem simulation models remain incapable of reproducing realistic population crashes. And the same is true of most climate simulation models: Though the geological record tells us that global temperatures can change very quickly, the models consistently underestimate that possibility. This is related to the next property, the nonlinear, non-equilibrium nature of systems.

Most engineered devices, consisting of mechanical springs, transistors, and the like, are built to be stable. That is, if stressed from rest, or equilibrium, they spring back. Many simple ecological models, physiological models, and even climate and economic models are built by assuming the same principle: a globally stable equilibrium. A related simplification is to see the world as consisting of separate parts that can be studied in a linear way, one piece at a time. These pieces can then be summed independently to make the whole. Researchers have developed a very large tool kit of analytical methods and statistics based on this linear idea, and it has proven invaluable for studying simple engineered devices. But even when many of the complex systems that interest us are not linear, we persist with these tools and models. It is a case of looking under the lamppost because the light is better even though we know the lost keys are in the shadows. Linear systems produce nice stationary statistics—constant risk metrics, for example. Because they assume that a process does not vary through time, one can subsample it to get an idea of what the larger universe of possibilities looks like. This characteristic of linear systems appeals to our normal heuristic thinking.

Nonlinear systems, however, are not so well behaved. They can appear stationary for a long while, then without anything changing, they exhibit jumps in variability—so-called “heteroscedasticity.” For example, if one looks at the range of economic variables over the past decade (daily market movements, GDP changes, etc.), one might guess that variability and the universe of possibilities are very modest. This was the modus operandi of normal risk management. As a consequence, the likelihood of some of the large moves we saw in 2008, which happened over so many consecutive days, should have been less than once in the age of the universe.

Our problem is that the scientific desire to simplify has taken over, something that Einstein warned against when he paraphrased Occam: “Everything should be made as simple as possible, but not simpler.” Thinking of natural and economic systems as essentially stable and decomposable into parts is a good initial hypothesis, current observations and measurements do not support that hypothesis—hence our continual surprise. Just as we like the idea of constancy, we are stubborn to change. The 19th century American humorist Josh Billings, perhaps, put it best: “It ain’t what we don’t know that gives us trouble, it’s what we know that just ain’t so.”

Among these principles is the idea that there might be universal early warning signs for critical transitions, diagnostic signals that appear near unstable tipping points of rapid change. The recent argument for early warning signs is based on the following: 1) that both simple and more realistic, complex nonlinear models show these behaviors, and 2) that there is a growing weight of empirical evidence for these common precursors in varied systems.

A key phenomenon known for decades is so-called “critical slowing” as a threshold approaches. That is, a system’s dynamic response to external perturbations becomes more sluggish near tipping points. Mathematically, this property gives rise to increased inertia in the ups and downs of things like temperature or population numbers—we call this inertia “autocorrelation”—which in turn can result in larger swings, or more volatility. Another related early signaling behavior is an increase in “spatial resonance”: Pulses occurring in neighboring parts of the web become synchronized. Nearby brain cells fire in unison minutes to hours prior to an epileptic seizure, for example.

The global financial meltdown illustrates the phenomenon of critical slowing and spatial resonance. Leading up to the crash, there was a marked increase in homogeneity among institutions, both in their revenue-generating strategies as well as in their risk-management strategies, thus increasing correlation among funds and across countries—an early warning. Indeed, with regard to risk management through diversification, it is ironic that diversification became so extreme that diversification was lost: Everyone owning part of everything creates complete homogeneity. Reducing risk by increasing portfolio diversity makes sense for each individual institution, but if everyone does it, it creates huge group or system-wide risk. Mathematically, such homogeneity leads to increased connectivity in the financial system, and the number and strength of these linkages grow as homogeneity increases. Thus, the consequence of increasing connectivity is to destabilize a generic complex system: Each institution becomes more affected by the balance sheets of neighboring institutions than by its own. [...]

Try here for the full article. The article was originally published on Dec 10, 2010.

Read More