We should ask, when will AI make a discovery on its own? For instance, computers should be able to understand numbers, and run analysis on numbers. Computers have complete access to every fact that humans know about numbers. So numbers should be the first place that we should expect to see genuine innovation from AI. This is a simple test for the moment that AI is able to make original contributions to our society: when can AI come up with a new thesis about numbers, and then build an original proof, something that can be published in the major, peer-reviewed math journals.

Until AI can do that, we have to admit that it's not really aware or sentient or any of the other more ambitious things that have recently been claimed for it.

Can AI teach us anything new about the pattern of prime numbers?

Can AI develop an original proof for the shape of shadows in high dimensional spaces?

Can AI creatively prove a new limit to mathematics?

There are 2 researchers in AI who deserve more attention: Kenneth O. Stanley and Joel Lehman. They wrote a great book: Why Greatness Cannot Be Planned. They look at the limits of utility functions and explain the importance of novelty. As an antidote to some of the hype around AI, I strongly recommend this book:

https://www.amazon.com/Why-Greatness-Cannot-Planned-Objectiv...

I am surprised this book is not mentioned. Talks about how optimization doesn't lead to optimal goals in complex environments (e.g. life).

https://www.amazon.com/Why-Greatness-Cannot-Planned-Objectiv...

This is a good book on the subject: "Why Greatness Cannot Be Planned: The Myth of the Objective" https://www.amazon.com/Why-Greatness-Cannot-Planned-Objectiv...

There's an interesting book on the topic discussed in these emails, entitled "Why Greatness Cannot Be Planned":

http://amzn.to/2CtIrRR

There's a YouTube talk by the author on the subject here:

https://youtu.be/dXQPL9GooyI

The rough idea is this:

- creativity often arises by stumbling around in a problem space, or in operating "randomly" under an artificially-imposed constraint

- modern life is obsessed with metrics and goal-setting, and this has extended into creative pursuits including science, research, and business

- sometimes, short-term focus on the goal defeats the goal-given aims (see e.g. shareholder value focus)

- the authors point out that when they were researching artificial intelligence, they discovered that systems that focused too much on an explicitly-coded "objective" would end up producing lackluster results, but systems that did more "playful" exploration within a problem space produced more creative results

- using this backdrop, the authors suggest perhaps innovation is not driven by narrowly focused heroic effort and is instead driven by serendipitous discovery and playful creativity

I found the ideas compelling, as I do find Kay's description of the "art" behind research.

The kindle version on Amazon [0] is slightly cheaper at $16.19.

[0] http://www.amazon.com/Why-Greatness-Cannot-Planned-Objective...