It’s not easy to make a big software system that really fits together. It’s incredibly important, though. Because it’s what makes the whole system more than just the sum of its parts. It’s what gives the system limitless possibilities—rather than just a bunch of specific features.

But it’s hard to achieve. It requires maintaining consistency and coherence across every area, over the course of many years. But I think it’s something we’ve been very successful at doing with Mathematica. And I think it’s actually one of the most crucial assets for the long-term future of Mathematica.

It’s also a part of things that I personally am deeply involved in.

Ever since we started developing it more than 21 years ago, I’ve been the chief architect and chief designer of Mathematica‘s core functionality. And particularly for Mathematica 6, there was a huge amount of design to do. Actually, I think much more even than for Mathematica 1.

In fact, I just realized that over the course of the decade during which were developing Mathematica 6—and accelerating greatly towards the end—I spent altogether about 10,000 hours doing what we call “design reviews” for Mathematica 6, trying to make all those new functions and pieces of functionality in Mathematica 6 be as clean and simple as possible, and all fit together.

At least the way I do it, doing software design is a lot like doing fundamental science.

In fundamental science, one starts from a bunch of phenomena, and then one tries to drill down to find out what’s underneath them—to try to find the root causes, the ultimate primitives, of what’s going on.

Well, in software design, one starts from a bunch of functionality, and then one needs to drill down to find out just what ultimate primitives one needs to support them.

In science, if one does a good job at finding the primitives, then one can have a very broad theory that covers not just the phenomena one started from, but lots of others too.

And in software design, it’s the same kind of thing.

If one does a good job at finding the primitives, then one can build a very broad system that gives one not just the functionality one was first thinking about, but lots more too.

Over the years, we’ve developed a pretty good process for doing design reviews.

We start with some particular new area of functionality. Then we get a rough description of the functions—or whatever—that we think we’ll need to cover it. Then we get down to the hard job of design analysis. Of trying to work out just what the correct fundamental primitives to cover the area are. The clean, simple functions that represent the essence of what’s going on—and that fit together with each other, and with the rest of Mathematica, to cover what’s needed.

Long ago I used to do design analysis pretty much solo.

But nowadays our company is full of talented people who help. The focal point is our Design Analysis group, which works with our experts in particular areas to start the process of refining possible designs.

At some point, though, I always get involved. So that anything that’s a core function of Mathematica is always something that I’ve personally design reviewed.

I sometimes wonder whether it’s crazy for me to do this. But I think having one person ultimately review everything is a good way to make sure that there really is coherence and consistency across the system. Of course, when the system is as big as Mathematica 6, doing all those design reviews to my level of perfection takes a long time—about 10,000 hours, in fact.

Design reviews are usually meetings with somewhere between two and twenty people. (Almost always they’re done with web conferencing, not in person.)

The majority of the time, there’s a preliminary implementation of whatever it is that we’re reviewing. Sometimes the people who are in the design review meeting will say “we think we have this mostly figured out”. Sometimes they’ll say “we can’t see how to set this up; we need your help”. Either way, what usually happens is that I start off trying out what’s been built, and asking lots and lots of questions about the whole area that’s involved.

It’s sometimes a little weird. One hour I’ll be intensely thinking about the higher mathematics of number theory functions. And the next hour I’ll be intensely focused on how we should handle data about cities around the world. Or how we should set up the most general possible interfaces to external control devices.

But although the subject matter is very varied, the principles are at some level the same.

I want to understand things at the most fundamental level—to see what the essential primitives should be. Then I want to make sure those primitives are built so that they fit in as well as possible to the whole existing structure of Mathematica—and so they are as easy as possible for people to understand, and work with.

It’s often a very grueling process. Progressively polishing things until they are as clean and simple as possible.

Sometimes we’ll start a meeting with things looking pretty complicated. A dozen functions that use some strange new construct, and have all sorts of weird arguments and options.

It’s usually pretty obvious that we have to do better. But figuring out how is often really hard.

There’ll usually be a whole series of incremental ideas. And then a few big shifts—which usually come from getting a clearer understanding of what the true core functionality has to be.

Often we’ll be talking quite a bit about precedents elsewhere in Mathematica. Because the more we can make what we’re designing now be like something we’ve done before in Mathematica, the better.

For several reasons. First, because it means we’re using approaches that we’ve tested somewhere else before.

Second, because it means that what we’re doing now will fit in better to what already exists.

And third, because it means that people who are already familiar with other things Mathematica does will have an easier time understanding the new things we’re adding.

But some of the most difficult design decisions have to do with when to break away from precedent. When is what we’re doing now really different from anything else that we’ve done before? When is it something sufficiently new—and big—that it makes sense to create some major new structure for it?

At least when we’re doing design reviews for Mathematica kernel functions, we always have a very definite final objective for our meetings: we want to actually write the reference documentation—the “function pages”—for what we’ve been talking about.

Because that documentation is what’s going to provide the specification for the final implementation—as well as the final definition of the function.

It always works pretty much the same way: I’ll be typing at my computer, and everyone else will be watching my screen via screen-sharing. And I’ll actually be writing the reference documentation for what each function does. And I’ll be asking every sentence or so: “Is that really correct? Is that actually what it should do?” And people will be pointing out this or that problem with what we’re saying.

It’s a good process, that I think does well at concentrating and capturing what we do in design analysis.

One of the things that happens in design reviews is that we finalize the names for functions.

Naming is a quintessential design review process. It involves drilling down to understand with as much as clarity as possible what a function really does, and is really about. And then finding the perfect word or two that captures the essence of it.

The name has to be something that’s familiar enough to people who should be using the function that they’ll immediately have an idea of what the function does. But that’s general enough that it won’t restrict what people will think of doing with the function.

Somehow the very texture of the name also has to communicate something about how broad the function is supposed to be. If it’s fairly specialized, it should have a specialized-sounding name. If it’s very broad, then it can have a much simpler name—often a much more common English word.

I always have a test for candidate names. If I imagine making up a sentence that explains what the function does, will the proposed name be something that fits into that sentence? Or will one end up always saying that the function with name X does something that is described as Y?

Sometimes it takes us days to come up with the right name for a function. But usually one knows when it’s right. It somehow just fits. And one can immediately remember it.

In Mathematica 6, a typical case of function naming was Manipulate.

It took quite a while to come up with that name.

We created this great function. But what should it be called? Interface? Activate? Dynamic? Live?
What?

Interface might seem good, because, after all, it creates an interface. But it’s a particular kind of interface, not a generic one.

Activate might be good, because it makes things active. But again it’s too generic.

Dynamic: again it sounds too general, and also a bit too technical. And anyway we wanted to use that name for something else.

Live… that’s a very confusing word. It’s even hard to parse when one reads it. Does it say “make it alive”, or “here’s something that is alive”, or what?

Well, after a while one realizes that one has to understand with more clarity just what it is that this great new function is doing.

Yes, it’s creating an interface. Yes, it’s making things active, dynamic, alive. But really, first and foremost, what it’s doing is to provide a way to control something. It’s attaching knobs and switches and so on to let one control almost anything.

So what about a word like Control? Again, very hard to understand. Is the thing itself a control? Or is it exerting control?

Handle? Again, too hard to understand.

Harness? A little better. But again, some ambiguity. And definitely too much of a “horse” motif.

Yoke? That one survived for several days. But finally the oxen jokes overwhelmed it.

And then came Manipulate.

At first, it was, “Oh, that’s too long a word for such a great and important function.”

But in my experience it often “feels right” to have a fairly long word for a function that does so much. Of course there were jokes about it sounding “manipulative”.

But as we went on talking about the function, we started just calling it Manipulate among ourselves. And everyone who joined the conversation just knew what it meant. And as we went on developing all its detailed capabilities, it still seemed to fit. It gave the right sense of controlling something, and making something happen.

So that’s how Manipulate got its name. It’s worked well.

Still, in developing Mathematica 6, we had to name nearly 1000 functions. And each name has to last—just as the names in Mathematica 1 have lasted.

Occasionally it was fairly obvious what a function should be called.

Perhaps it had some standard name, say in mathematics or computing, such as Norm or StringSplit.

Perhaps it fit into some existing family of names, like ContourPlot3D.

But most of the time, each name took lots and lots of work to invent. Each one is sort of a minimal expression of a concept that a primitive in Mathematica implements.

Unlike human languages that grow and mutate over time, Mathematica has to be defined once and for all. So that it can be implemented, and so that both the computers and the people who use it can know what everything in it means.

As the Mathematica system has grown, it’s in some ways become more and more difficult to do the design. Because every new thing that’s added has to fit in with more and more that’s already there.

But in some ways it’s also become easier. Because there are more precedents to draw on. But most importantly, because we’ve gotten (and I think I personally have gotten) better and better at doing the design.

It’s not so much that the quality of the results has changed. It’s more that we’ve gotten faster and faster at solving design problems.

There are problems that come up today that I can solve in a few minutes—yet I remember twenty years ago it taking hours to solve similar problems.

Over the years, there’ve been quite a few “old chestnuts”: design problems that we just couldn’t crack. Places where we just couldn’t see a clean way to add some particular kind of functionality to Mathematica.

But as we’ve gotten better and better at design, we’ve been solving more and more of these. Dynamic interactivity was one big example. And in fact Mathematica 6 has a remarkable number of them solved.

Doing design reviews and nailing down the functional design of Mathematica is a most satisfying intellectual activity. It’s incredibly diverse in subject matter. And in a sense always very pure.

It’s about a huge range of fundamental ideas—and working out how to fit them all together to create a coherent system that all makes sense.

It’s certainly as hard as anything I know about in science. But in many ways it’s more creative. One’s not trying to decode what exists in the world. One’s trying to create something from scratch—to build a world that one can then work within.

I use Mathematica 6 every day. And every day I use countless design ideas that make all the pieces fit smoothly together.

And I realize that, yes, those 10,000 hours of design reviews were worth spending. Even just for me, what we did in them will save me countless hours in being able to do so much more with Mathematica, so much more easily.

And now I’m looking forward to all the design reviews we’re starting to do for Mathematica 7, and Mathematica 8….

As of 2019, we’ve reached Mathematica Version 12 (which is also Wolfram Language Version 12)—with more than 6000 built-in functions.