Talking to Himself, November 3 to November 4

Andrew November 3: So, explain this to me. What’s your, I mean our, vision for the near-term future of intelligence? I mean, in the next 5–10 years.

Andrew November 4: Whoa… that’s a big question, and maybe worth a 30-minute brainstorm and 2 cups of tea. Unfortunately I, I mean we, don’t have the answer to that one quite yet. But I can tell you something else I learned today. I think I’m getting a clearer understanding of how human and machine intelligence might complement each other in solving problems. And in particular, 2 types of human intelligence.

AN3: Darn, you got my hopes up with all that talk about the singularity… but I’ll hear you out. 2 types of human intelligence… what do you mean?

AN4: First of all, by intelligence I mean the ability to achieve one’s goals in a wide variety of environments. I won’t define it further; I’ll leave you with people’s usual connotations of intelligence. Now the 2 types I’m referring to are expert intelligence and layman intelligence. Layman intelligence is the intelligence the average human gets as part of the current “standard package” of being born as a human. That includes incredible association-forming mechanisms and visual and spatial recognition. Expert intelligence, as I’m defining it here, is any intelligence beyond this standard package. Chess grandmasters, professional basketball players, and scientists trained to do physics all demonstrate expert intelligence.

AN3: OK, I see. And what’s the purpose of distinguishing between these 2 types of human intelligence?

AN4: Perhaps an example will help here. Recall our initial goal is to understand how these 3 types of intelligence (2 human, 1 machine) might complement each other in solving problems. I’m going to steal a couple examples from Michael Nielsen’s Reinventing Discovery: The New Era of Networked Science. Suppose we were trying to solve the problem of mapping out the entities in the universe, where by “mapping out entities” I mean knowing what entities are useful to talk about, some characteristics of those entities (e.g. the electromagnetic spectra and thus composition of those entities, or the shapes of galaxies – spiral, elliptical, etc.), including their locations, and the important relations between those entities (e.g. one type of relation is “belonging”: planets belong to stars, which belong to galaxies; another type is gravitation). Given the state of today’s world and technology, what are the subproblems we’d have to solve in order to accomplish this?

AN3: Well, I imagine we’d first have to image as much of the galaxy, i.e. as much of the sky, as we could – I’m envisioning panning a super high-resolution telescope across the night sky from an observatory somewhere in Chile. Then we could in theory lay out all the printed images on the ground in the same positions that we imaged them, and look at each one in turn. That might be a lot of images though… we might want to understand both the macro-structure of larger universal entities like superclusters as well as micro-structures like individual stars and black holes. So we might have to zoom in and out to figure out which entities are useful to talk about. And then we could go and study their characteristics like their elemental composition via spectroscopy, we could classify them, and we could…

AN4: Yes, I think that will be enough detail to illustrate the point. You’re always planning about how to solve problems, aren’t you?

AN3: I thought we liked that about ourself.

AN4: I guess we do… Anyway, let’s take the first subproblem you mentioned – imaging the night sky. As you mentioned, this is clearly a job for a precise machine like a high-resolution telescope. Plus, I’m guessing that we’ve built telescopes to record electromagnetic radiation outside the range of visible light as well (not sure about this though). You couldn’t imagine just looking out at the night sky and recording the positions of all the stars in a notebook, could you? Well, I guess Kepler did that, but today we can get much more resolution than we could just by looking… Kepler never would have been able to differentiate a nearby, less bright star from a brighter star further away, for instance.

AN3: Yes, that’s all reasonable… What’s your point?

AN4: Here we have one conclusion: machines can measure the things that humans have designed them to measure more precisely and reliably than humans can. We’re going to be making generalizations like this to answer our original question of how human and machine intelligences might complement each other.

AN3: Ignoring my distaste for generalizations based on single examples, I like the direction in which this is going – we’re trying to characterize the comparative advantages that machine and the 2 human types of intelligence have with respect to each other. Then we can plot out how we might solve the “mapping out the universe” problem!

AN4: Exactly!

AN3: Let me try making a characterization now! Let’s take the second subproblem, that of taking the images output by the telescope and figuring out, I guess, what’s in the images.

AN4: I would suggest that you break that down into more subproblems.

AN3: What do you mean?

AN4: Here, I’ll suggest one strategy for “figuring out what’s in the images.” Step one: identify all the universal entities currently known to astronomy (e.g. galaxies, stars) and what they’re documented to look like. Step two: look at a sample of the images and figure out probably multiple resolutions at which to display those images so that we can actually visualize the entities and their relationships, as well as catch unforeseen entities and relationships. Then, to map all the known entities, follow step three: for each level of resolution, search through all the images for things that look like what the various known entities would look like at that resolution. To discover unforeseen entities and relationships…

AN3: OK, I get your point. So for step one, I would leverage the second type of human intelligence you mentioned, expert intelligence (in case, the astronomer’s intelligence) to know the latest entities known to astronomy, as well as their various types and what they look like. For step two, perhaps this is the job of the astronomer as well, because the astronomer is likely to know what kind of visualization characteristically depicts an entity and depicts an entity relationship. For step three… obviously the astronomer can’t look at all the images herself! Could the astronomer possibly get the aid of a computer vision expert and write computer vision algorithms to detect the entities?

AN4: Such algorithms as they are today could get you part of the way there; for example, they could screen out obvious non-entities. But without a mass of labeled training data, they probably couldn’t tell you whether a galaxy was spiral or elliptical.

AN3: Dang… but it seems impractical to have the astronomer, or even her whole research lab, do this labeling of spiral vs. elliptical galaxies.

AN4: You’re forgetting about the third type of intelligence we mentioned.

AN3: Layman intelligence? Is classifying a galaxy as spiral or elliptical easily teachable to a layman?

AN4: It’s surprisingly accessible; you can go to and check out how accessible it is yourself. Classifying galaxy shapes is one place where our human “standard package,” which includes spatial recognition, gets a lot of leverage.

AN3: Oh, wow – I’m getting the hang of this! It’s even kind of fun =) … Sorry, back to serious talk.

AN4: In fact, the solutions we’ve proposed here for “mapping out the universe” are a simplified version of what the Sloan Digital Sky Survey and scientists Kevin Schawinski and Chris Lintott (who built Galaxy Zoo) actually did!

AN3: Wow, that wasn’t as hard as I thought it’d be =)

AN4: Well, let’s think about the characterizations we can make about the three types of intelligence. So far we have that machines can measure precise and reliably. What else?

AN3: It seems like two expert intelligences were important here: the astronomer’s, for knowing the universal entities and their relationships, as well as what they looked like, and the computer vision programmer’s, for knowing what’s possible with computer vision and for designing the algorithm to filter out obvious non-entities. I’m not sure how to generalize the expert intelligence here except to say that the expert intelligences were good for having the relevant “domain knowledge.” And then the layman intelligences were good at the spatial recognition to classify spiral and elliptical galaxies, as well as just having a lot more time between the thousands of citizens than the two astronomers.

AN4: I think I like those general characterizations… And, as you can see, of the three types of intelligence, the expert intelligence has currently the most case-by-case comparative advantage, in the sense that it’s hard to make general statements about when expert intelligence beats out the other two types except to say that, perhaps, expert intelligence is good at having the “domain knowledge and ways of thinking” in areas where there exist human experts.

AN3: Oh, I see, as opposed to layman intelligence, where you can point to very specific comparative advantages like spatial recognition, natural language processing, association-forming mechanisms, etc. And the same thing for machines: at least right now, you can point to specific comparative advantages like speed of computation, accuracy of measurement and computation, reliability of measurement and computation, freedom from boredom, etc.

AN4: Precisely. Also, we did miss one comparative advantage above that also belongs to the astronomer’s expert intelligence. I would say that “knowing the right questions to ask” about the map – in this case, focusing attention on the distinction between spiral and elliptical galaxies among all possible questions to be asked about the map – is the expert’s comparative advantage.

AN3: Very interesting! I do have one question here though. As you know, I’ve been reading Ray Kurzweil’s The Singularity is Near and the book makes compelling arguments that machine intelligence is growing at an accelerating pace and human biological intelligence, at least the way it exists now, will become only a small part of our human-machine joint intelligences in the future. Will that make the analysis we’ve done here obsolete?

AN4: Although I would take Ray Kurzweil’s predictions with a grain of salt, I would agree that machine intelligence will continue to beat human intelligence in more and more areas that we currently consider the comparative advantage of expert or layman intelligences. As an example, Schawinski and Lintott have actually trained a classification algorithm based on the Zooites’ classifications of spiral vs. elliptical galaxies to automatically do the classification without layman input, and the algorithm has achieved a 90 percent AUC! It does seem to be the tendency that machine intelligence will replace human intelligence in areas like spatial recognition and NLP, as well as other areas; the only highly uncertain question is when this will happen, but even Kurzweil’s estimates suggest that such advances are still at least five years out. In the meantime, it may be worthwhile to think about how we might leverage these three types of intelligence as they exist today.