We’ve got an incredible amount of new technology coming out this summer. Two weeks ago we launched Wolfram Programming Cloud. Today I’m pleased to announce the release of a major new version of Mathematica: Mathematica 10.
We released Mathematica 1 just over 26 years ago—on June 23, 1988. And ever since we’ve been systematically making Mathematica ever bigger, stronger, broader and deeper. But Mathematica 10—released today—represents the single biggest jump in new functionality in the entire history of Mathematica.
At a personal level it is very satisfying to see after all these years how successful the principles that I defined at the very beginning of the Mathematica project have proven to be. And it is also satisfying to see how far we’ve gotten with all the talent and hard work that has been poured into Mathematica over nearly three decades.
We’ll probably never know whether our commitment to R&D over all these years makes sense at a purely commercial level. But it has always made sense to me—and the success of Mathematica and our company has allowed us to take a very long-term view, continually investing in building layer upon layer of long-term technology.
One of the recent outgrowths—from combining Mathematica, Wolfram|Alpha and more—has been the creation of the Wolfram Language. And in effect Mathematica is now an application of the Wolfram Language.
But Mathematica still very much has its own identity too—as our longtime flagship product, and the system that has continually redefined technical computing for more than a quarter of a century.
And today, with Mathematica 10, more is new than in any single previous version of Mathematica. It is satisfying to see such a long curve of accelerating development—and to realize that there are more new functions being added with Mathematica 10 than there were functions altogether in Mathematica 1.
So what is the new functionality in Mathematica 10? It’s a mixture of completely new areas and directions (like geometric computation, machine learning and geographic computation)—together with extensive strengthening, polishing and expanding of existing areas. It’s also a mixture of things I’ve long planned for us to do—but which had to wait for us to develop the necessary technology—together with things I’ve only fairly recently realized we’re in a position to tackle.
When you first launch Mathematica 10 there are some things you’ll notice right away. One is that Mathematica 10 is set up to connect immediately to the Wolfram Cloud. Unlike Wolfram Programming Cloud—or the upcoming Mathematica Online—Mathematica 10 doesn’t run its interface or computations in the cloud. Instead, it maintains all the advantages of running these natively on your local computer—but connects to the Wolfram Cloud so it can have cloud-based files and other forms of cloud-mediated sharing, as well as the ability to access cloud-based parts of the Wolfram Knowledgebase.
If you’re an existing Mathematica user, you’ll notice some changes when you start using notebooks in Mathematica 10. Like there’s now autocompletion everywhere—for option values, strings, wherever. And there’s also a hovering help box that lets you immediately get function templates or documentation. And there’s also—as much requested by the user community—computation-aware multiple undo. It’s horribly difficult to know how and when you can validly undo Mathematica computations—but in Mathematica 10 we’ve finally managed to solve this to the point of having a practical multiple undo.
Another very obvious change in Mathematica 10 is that plots and graphics have a fresh new default look (you can get the old look with an option setting, of course):
And as in lots of other areas, that’s just the tip of the iceberg. Underneath, there’s actually a whole powerful new mechanism of “plot themes”—where instead of setting lots of individual options, you can for example now just specify an overall theme for a plot—like “web” or “minimal” or “scientific”.
But what about more algorithmic areas? There’s an amazing amount there that’s new in Mathematica 10. Lots of new algorithms—including many that we invented in-house. Like the algorithm that lets Mathematica 10 routinely solve systems of numerical polynomial equations that have 100,000+ solutions. Or the cluster of algorithms we invented that for the first time give exact symbolic solutions to all sorts of hybrid differential equations or differential delay equations—making such equations now as accessible as standard ordinary differential equations.
Of course, when it comes to developing algorithms, we’re in a spectacular position these days. Because our multi-decade investment in coherent system design now means that in any new algorithm we develop, it’s easy for us to bring together algorithmic capabilities from all over our system. If we’re developing a numerical algorithm, for example, it’s easy for us to do sophisticated algebraic preprocessing, or use combinatorial optimization or graph theory or whatever. And we get to make new kinds of algorithms that mix all sorts of different fields and approaches in ways that were never possible before.
From the very beginning, one of our central principles has been to automate as much as possible—and to create not just algorithms, but complete meta-algorithms that automate the whole process of going from a computational goal to a specific computation done with a specific algorithm. And it’s been this kind of automation that’s allowed us over the years to “consumerize” more and more areas of computation—and to take them from being accessible only to experts, to being usable by anyone as routine building blocks.
And in Mathematica 10 one important area where this is happening is machine learning. Inside the system there are all kinds of core algorithms familiar to experts—logistic regression, random forests, SVMs, etc. And all kinds of preprocessing and scoring schemes. But to the user there are just two highly automated functions: Classify and Predict. And with these functions, it’s now easy to call on machine learning whenever one wants.
There are huge new algorithmic capabilities in Mathematica 10 in graph theory, image processing, control theory and lots of other areas. Sometimes one’s not surprised that it’s at least possible to have such-and-such a function—even though it’s really nice to have it be as clean as it is in Mathematica 10. But in other cases it at first seems somehow impossible that the function could work.
There are all kinds of issues. Maybe the general problem is undecidable, or theoretically intractable. Or it’s ill conditioned. Or it involves too many cases. Or it needs too much data. What’s remarkable is how often—by being algorithmically sophisticated, and by leveraging what we’ve built in Mathematica and the Wolfram Language—it’s possible to work around these issues, and to build a function that covers the vast majority of important practical cases.
Another important issue is just how much we can represent and do computation on. Expanding this is a big emphasis in the Wolfram Language—and Mathematica 10 has access to everything that’s been developed there. And so, for example, in Mathematica 10 there’s an immediate symbolic representation for dates, times and time series—as well as for geolocations and geographic data.
The Wolfram Language has ways to represent a very broad range of things in the real world. But what about data on those things? Much of that resides in the Wolfram Knowledgebase in the cloud. Soon we’re going to be launching the Wolfram Discovery Platform, which is built to allow large-scale access to data from the cloud. But since that’s not the typical use of Mathematica, basic versions of Mathematica 10 are just set up for small-scale data access—and need explicit Wolfram Cloud Credits to get more.
Still, within Mathematica 10 there are plenty of spectacular new things that will be possible by using just small amounts of data from the Wolfram Knowledgebase.
A little while ago I found a to-do list for Mathematica that I wrote in 1991. Some of the entries on it were done in just a few years. But most required the development of towers of technology that took many years to build. And at least one has been a holdout all these years—until now.
On the to-do it was just “PDEs”. But behind those four letters are centuries of mathematics, and a remarkably complex tower of algorithmic requirements. Yes, Mathematica has been able to handle various kinds of PDEs (partial differential equations) for 20 years. But in Mathematica we always seek as much generality and robustness as possible, and that’s where the challenge has been. Because we’ve wanted to be able to handle PDEs in any kind of geometry. And while there are standard methods—like finite element analysis—for solving PDEs in different geometries, there’s been no good way to describe the underlying geometry in enough generality.
Over the years, we’ve put immense effort into the design of Mathematica and what’s now the Wolfram Language. And part of that design has involved developing broad computational representations for what have traditionally been thought of as mathematical concepts. It’s difficult—if fascinating—intellectual work, in effect getting “underneath” the math to create new, often more general, computational representations.
A few years ago we did it for probability theory and the whole cluster of things around it, like statistical distributions and random processes. Now in Mathematica 10 we’ve done it for another area: geometry.
What we’ve got is really a fundamental extension to the domain of what can be represented computationally, and it’s going to be an important building block for many things going forward. And in Mathematica 10 it delivers some very powerful new functionality—including PDEs and finite elements.
So, what’s hard about representing geometry computationally? The problem is not in handling specific kinds of cases—there are a variety of methods for doing that—but rather in getting something that’s truly general, and extensible, while still being easy to use in specific cases. We’ve been thinking about how to do this for well over a decade, and it’s exciting to now have a solution.
It turns out that math in a sense gets us part of the way there—because it recognizes that there are various kinds of geometrical objects, from points to lines to surfaces to volumes, that are just distinguished by their dimensions. In computer systems, though, these objects are typically represented rather differently. 3D graphics systems, for example, typically handle points, lines and surfaces, but don’t really have a notion of volumes or solids. CAD systems, on the other hand, handle volumes and solids, but typically don’t handle points, lines and surfaces. GIS systems do handle both boundaries and interiors of regions—but only in 2D.
So why can’t we just “use the math”? The problem is that specific mathematical theories—and representations—tend once again to handle, or at least be convenient in, only specific kinds of cases. So, for example, one can describe geometry in terms of equations and inequalities—in effect using real algebraic geometry—but this is only convenient for simple “math-related” shapes. One can use combinatorial topology, which is essentially based on mesh regions, and which is quite general, but difficult to use directly—and doesn’t readily cover things like non-bounded regions. Or one could try using differential geometry—which may be good for manifolds, but doesn’t readily cover geometries with mixed dimensions, and isn’t closed under Boolean operations.
What we’ve built in effect operates “underneath the math”: it’s a general symbolic representation of geometry, which makes it convenient to apply any of these different mathematical or computational approaches. And so instead of having all sorts of separate “point in polygon”, “point in mesh”, “point on line” etc. functions, everything is based on a single general RegionMember function. And similarly Area, Volume, ArcLength and all their generalizations are just based on a single RegionMeasure function.
The result is a remarkably smooth and powerful way of doing geometry, which conveniently spans from middle-school triangle math to being able to describe the most complex geometrical forms for engineering and physics. What’s also important—and typical of our approach to everything—is that all this geometry is deeply integrated with the rest of the system. So, for example, one can immediately find equation solutions within a geometric region, or compute a maximum in it, or integrate over it—or, for that matter, solve a partial differential equation in it, with all the various kinds of possible boundary conditions conveniently being described.
The geometry language we have is very clean. But underneath it is a giant tower of algorithmic functionality—that relies on a fair fraction of the areas that we’ve been developing for the past quarter century. To the typical user there are few indications of this complexity—although perhaps the multi-hundred-page treatise on the details of going beyond automatic settings for finite elements in Mathematica 10 provides a clue.
Geometry is just one new area. The drive for generality continues elsewhere too. Like in image processing, where we’re now supporting most image processing operations not only in 2D but also in 3D images. Or in graph computation, where everything works seamlessly with directed graphs, undirected graphs, mixed graphs, multigraphs and weighted graphs. As usual, it’s taken developing all sorts of new algorithms and methods to deal with cases that in a sense cross disciplines, and so haven’t been studied before, even though it’s obvious they can appear in practice.
As I’ve mentioned, there are some things in Mathematica 10 that we’ve been able to do essentially because our technology stack has now reached the point where they’re possible. There are others, though, that in effect have taken solving a problem, and often a problem that we’ve been thinking about for a decade or two. An example of this is the system for handling formal math operators in Mathematica 10.
In a sense what we’re doing is to take the idea of symbolic representation one more step. In math, we’ve always allowed a variable like x to be symbolic, so it can correspond to any possible value. And we’ve allowed functions like f to be symbolic too. But what about mathematical operators like derivative? In the past, these have always been explicit—so for example they actually take derivatives if they can. But now we have a new notion of “inactive” functions and operators, which gives us a general way to handle mathematical operators purely symbolically, so that we can transform and manipulate expressions formally, while still maintaining the meaning of these operators.
This makes possible all sorts of new things—from conveniently representing complicated vector analysis expressions, to doing symbolic transformations not only on math but also on programs, to being able to formally manipulate objects like integrals, with built-in implementations of all the various generalizations of things like Leibniz’s rule.
In building Mathematica 10, we’ve continued to push forward into uncharted computational—and mathematical—territory. But we’ve also worked to make Mathematica 10 even more convenient for areas like elementary math. Sometimes it’s a challenge to fit concepts from elementary math with the overall generality that we want to maintain. And often it requires quite a bit of sophistication to make it work. But the result is a wonderfully seamless transition from the elementary to the advanced. And in Mathematica 10, we’ve once again achieved this for things like curve computations and function domains and ranges.
The development of the Wolfram Language has had many implications for Mathematica—first visible now in Mathematica 10. In addition to all sorts of interaction with real-world data and with external systems, there are some fundamental new constructs in the system itself. An example is key-value associations, which in effect introduce “named parts” throughout the system. Another example is the general templating system, important for programmatically constructing strings, files or web pages.
With the Wolfram Language there are vast new areas of functionality—supporting new kinds of programming, new structures and new kinds of data, new forms of deployment, and new ways to integrate with other systems. And with all this development—and all the new products it’s making possible—one might worry that the traditional core directions of Mathematica would be left behind. But nothing is further from the truth. And in fact all the new Wolfram Language development has made possible even more energetic efforts in traditional Mathematica areas.
Partly that is the result of new software capabilities. Partly it is the result of new understanding that we’ve developed about how to push forward the design of a very large knowledge-based system. And partly it’s the result of continued strengthening and streamlining of our internal R&D processes.
We’re still a fairly small company (about 700 people), but we’ve been steadily ramping up our R&D output. And it’s amazing to see what we’ve been able to build for Mathematica 10. In the 19 months (588 days) since Mathematica 9 was released, we’ve finished more than 700 new functions that are now in Mathematica 10—and we’ve added countless enhancements to existing functions.
I think the fact that this is possible is a great tribute to the tools, team and organization we’ve built—and the strength of the principles under which we’ve been operating all these years.
To most people in the software business, if they knew the amount of R&D that’s gone into Mathematica 10, it would seem crazy. Most would assume that a 26-year-old product would be in a maintenance mode, with tiny updates being made every few years. But that’s not the story with Mathematica at all. Instead, 26 years after its initial release, its rate of growth is still accelerating. There’s going to be even more to come.
But today I’m pleased to announce that the fruits of a “crazy” amount of R&D are here: Mathematica 10.
To comment, please visit the copy of this post at the Wolfram Blog »
