COMPUTER SCIENCE TRIPOS Part IB – 2020 – Paper 6
Artificial Intelligence (sbh11)
A neural network takes input vectors x ∈ R
n
, has a single layer of hidden nodes hi
where i = 1, . . . , p, and a single output node o.
z
h1
h2
hp
o
x1
x2
xn
.
.
.
.
.
.
All nodes compute the function z = σ(a) where
a =
Xm
i=1
wizi + w0.
Here z and zi denote the inputs and outputs of the node, and each node has its
own set of weights w0, w1, . . . , wm. Examples take the form (x, y) and the error the
network makes for an example is E(x, y, w), where w is the collection of all the
weights in the network.
(a) An example has been applied to the network and we know the quantity
δ = ∂E/∂a for the output node o. Explain how this knowledge can be used
to compute the partial derivative of E with respect to the weights for some
hidden node hi
. [8 marks]
(b) The hidden nodes are now replaced with a different type of node, computing
zi = hi(x) = φ(||x − ci
||2
)
where φ is some new function, the ci ∈ R
n are the parameters for the new nodes,
and
||v||2 = v
T v.
Give a detailed derivation of a training algorithm for this network. [12 marks]