BGonline.org Forums

Explainability

Posted By: David Levy
Date: Friday, 17 November 2017, at 1:22 a.m.

In Response To: Explainability (Ray Kershaw)

There is a middle layer to the neural net. The backgammon position is encoded into some number of input "neurons", some of them potentially proprietary. Each of those neurons has a weighted connection to some number of neurons in a middle, hidden layer, and each of them has a weighted connection to each of the output neurons which represent win and gammon percentages.

This backgammon meaning of the hidden layer is likely impossible to know. So the weights are likely unconnected with things that backgammon players would understand.

I think what Fowler is suggesting is that there is an entirely separate AI process of extracting explanations from a net.

Messages In This Thread

• Explainability
  David Levy -- Wednesday, 15 November 2017, at 4:57 a.m.
  • Explainability
    Ray Kershaw -- Thursday, 16 November 2017, at 10:37 p.m.
    • Explainability
      David Levy -- Friday, 17 November 2017, at 1:22 a.m.
      • Explainability
        Maik Stiebler -- Friday, 17 November 2017, at 10:39 a.m.