Today marks an important milestone for Wolfram|Alpha, and for computational knowledge in general: for the first time, Wolfram|Alpha is now on average giving complete, successful responses to more than 90% of the queries entered on its website (and with “nearby” interpretations included, the fraction is closer to 95%).

I consider this an impressive achievement—the hard-won result of many years of progressively filling out the knowledge and linguistic capabilities of the system.

The picture below shows how the fraction of successful queries (in green) has increased relative to unsuccessful ones (red) since Wolfram|Alpha was launched in 2009. And from the log scale in the right-hand panel, we can see that there’s been a roughly exponential decrease in the failure rate, with a half-life of around 18 months. It seems to be a kind of Moore’s law for computational knowledge: the net effect of innumerable individual engineering achievements and new ideas is to give exponential improvement.

Continue reading

One day I’m sure everyone will routinely collect all sorts of data about themselves. But because I’ve been interested in data for a very long time, I started doing this long ago. I actually assumed lots of other people were doing it too, but apparently they were not. And so now I have what is probably one of the world’s largest collections of personal data.

Every day—in an effort at “self awareness”—I have automated systems send me a few emails about the day before. But even though I’ve been accumulating data for years—and always meant to analyze it—I’ve never actually gotten around to doing it. But with Mathematica and the automated data analysis capabilities we just released in Wolfram|Alpha Pro, I thought now would be a good time to finally try taking a look—and to use myself as an experimental subject for studying what one might call “personal analytics”.

Let’s start off talking about email. I have a complete archive of all my email going back to 1989—a year after Mathematica was released, and two years after I founded Wolfram Research. Here’s a plot with a dot showing the time of each of the third of a million emails I’ve sent since 1989:

Continue reading

It’s a sad but true fact that most data that’s generated or collected—even with considerable effort—never gets any kind of serious analysis. But in a sense that’s not surprising. Because doing data science has always been hard. And even expert data scientists usually have to spend lots of time wrangling code and data to do any particular analysis.

I myself have been using computers to work with data for more than a third of a century. And over that time my tools and methods have gradually evolved. But this week—with the release of Wolfram|Alpha Pro—something dramatic has happened, that will forever change the way I approach data.

The key idea is automation. The concept in Wolfram|Alpha Pro is that I should just be able to take my data in whatever raw form it arrives, and throw it into Wolfram|Alpha Pro. And then Wolfram|Alpha Pro should automatically do a whole bunch of analysis, and then give me a well-organized report about my data. And if my data isn’t too large, this should all happen in a few seconds.

And what’s amazing to me is that it actually works. I’ve got all kinds of data lying around: measurements, business reports, personal analytics, whatever. And I’ve been feeding it into Wolfram|Alpha Pro. And Wolfram|Alpha Pro has been showing me visualizations and coming up with analyses that tell me all kinds of useful things about the data.

Continue reading

Today I’m excited to be able to announce the launch of Wolfram|Alpha Pro—the biggest single step in the development of Wolfram|Alpha since its original introduction.

Over the two and a half years since we first launched, Wolfram|Alpha has been growing rapidly in content and capabilities. But today’s introduction of Wolfram|Alpha Pro in effect adds a whole new model for interacting with Wolfram|Alpha—and brings all sorts of fundamentally new and remarkable capabilities.

Starting today, everyone has access to Wolfram|Alpha Pro at wolframalpha.com. Unlike the “tourist” version of Wolfram|Alpha, though, you have to log in, and, yes, to get full capabilities there’s a subscription ($4.99/month, or $2.99/month for students). (Right now, you can try it for free with a trial subscription.)

So, what does Wolfram|Alpha Pro do?

Continue reading

There’s been very little change in top-level internet domains (like .com, .org, .us, etc.) for a long time. But a number of years ago I started thinking about the possibility of having a new .data top-level domain (TLD). And starting this week, there’ll finally be a period when it’s possible to apply to create such a thing.

It’s not at all clear what’s going to happen with new TLDs—or how people will end up feeling about them. Presumably there’ll be TLDs for places and communities and professions and categories of goods and events. A .data TLD would be a slightly different kind of thing. But along with some other interested parties, I’ve been exploring the possibility of creating such a thing. Continue reading

Last weekend I decided to have a short break from all the exciting technological things we’re doing… and to give a talk at the Singularity Summit in New York City about the implications of A New Kind of Science for the future, technological and otherwise. Here’s the transcript:

Well, what I wanted to do here was to have some fun—and talk about the future.

That’s something that’s kind of recreational for me. Because what I normally do is work in the trenches just trying to actually build the future… kind of one brick at a time—or at least one big project at a time.

I’ve been doing this now for a bit more than 30 years, and I guess I’ve built a fairly tall tower. From which it’s possible to do and see some pretty interesting things. Continue reading

Opening keynote of the Wolfram Mathematica Virtual Conference 2011, September 26–27.

Hi. I’m pleased to be with you. I’ve been asked to talk a bit about the background and vision of Mathematica.

You know, this month it’s exactly 25 years since I started building Mathematica. It’s been really exciting seeing Mathematica grow and prosper over the years. But I have to say that in just the last few years, something truly remarkable seems to have been happening. It feels like Mathematica is really coming of age. It’s in just the right place at the right time. And it’s making possible some fundamentally new and profoundly powerful things. Like Wolfram|Alpha, and CDF, and yet other things that we’ll have coming over the next year.

But let me start off at the beginning. How did Mathematica come to be in the first place? Here’s the personal story. Continue reading

I’m so sad this evening—as millions are—to hear of Steve Jobs’s death. Scattered over the last quarter century, I learned much from Steve Jobs, and was proud to consider him a friend. And indeed, he contributed in various ways to all three of my major life projects so far: Mathematica, A New Kind of Science and Wolfram|Alpha.

I first met Steve Jobs in 1987, when he was quietly building his first NeXT computer, and I was quietly building the first version of Mathematica. A mutual friend had made the introduction, and Steve Jobs wasted no time in saying that he was planning to make the definitive computer for higher education, and he wanted Mathematica to be part of it. I don’t now remember the details of our first meeting, but at the end of it, Steve gave me his business card, which tonight I found duly still sitting in my files:

Continue reading

Things with Wolfram|Alpha are going well. Really well. So well that I’m now incredibly keen to scale them up dramatically.

When I started the Wolfram|Alpha project, I was not even sure anything like it would be possible. But over the last two years we’ve proved that, yes, with the tower of technology we’ve created, one can in fact take large swaths of knowledge, make them computable, and deliver them for everyone to use.

From the outside, it’s easy to see that there’s been steady growth in the domains of knowledge that Wolfram|Alpha covers. And over the next few months there’ll be some big additions, notably in everyday and consumer areas. But to me what’s most dramatic is what’s happened on the inside. Because what we’ve done is to build a giant system of technology and management processes that allows us systematically to make any area of knowledge computable.

The catch is that it always takes effort. We rely on a huge tower of automation. But in every new area we tackle there are new issues, new opportunities—and new ways that resources and human effort have to be used.

I’m very pleased with how broad and deep the coverage we have already achieved is. But we have an immense to-do list, assembled not least from all the feedback we’ve received from users of Wolfram|Alpha. And the good news is that at this point it’s a straight shot: given enough effort, we can complete the to-do list. We have all the systems we need to scale the knowledge in Wolfram|Alpha up all the way. Continue reading

The precursors of what we’re trying to do with computable data in Wolfram|Alpha in many ways stretch back to the very dawn of human history—and in fact their development has been fascinatingly tied to the whole progress of civilization.

Last year we invited the leaders of today’s great data repositories to our Wolfram Data Summit—and as a conversation piece we assembled a timeline of the historical development of systematic data and computable knowledge.

This year, as we approach the Wolfram Data Summit 2011, we’ve taken the comments and suggestions we got, and we’re making available a five-feet-long (1.5 meters) printed poster of the timeline—as well as having the basic content on the web.

The story the timeline tells is a fascinating one: of how, in a multitude of steps, our civilization has systematized more and more areas of knowledge—collected the data associated with them, and gradually made them amenable to automation. Continue reading