Torch Demo

Posted on | In

We have 5 steps to do in training a torch neural network.

*(1) Load and normalize data; *
*(2) Define Neural Network; *
*(3) Define Loss function; *
*(4) Train network on training data; *
*(5) Test network on test data. *

Code

1
2
3
4
5
require 'paths'
require 'nn'
require 'torch'
require 'cunn'
require 'cutorch'

Load and normalize data

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
if (not paths.filep("cifar10torchsmall.zip")) then
    os.execute('wget -c https://s3.amazonaws.com/torch7/data/cifar10torchsmall.zip')
    os.execute('unzip cifar10torchsmall.zip')
end

trainset = torch.load('cifar10-train.t7')
testset = torch.load('cifar10-test.t7')
classes = {'airplane', 'automobile', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck'}

setmetatable(trainset,
    {__index = function(t, i)
                   return {t.data[i], t.label[i]}
               end}
);

function trainset:size()
    return self.data:size(1)
end
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
trainset.data = trainset.data:double()
testset.data = testset.data:double()

mean = {}
stdv  = {}
for i=1,3 do
    mean[i] = trainset.data[{ {}, {i}, {}, {}  }]:mean()
    --print('Channel ' .. i .. ', Mean: ' .. mean[i])
    trainset.data[{ {}, {i}, {}, {}  }]:add(-mean[i])   
    stdv[i] = trainset.data[{ {}, {i}, {}, {}  }]:std()
    --print('Channel ' .. i .. ', Standard Deviation: ' .. stdv[i])
    trainset.data[{ {}, {i}, {}, {}  }]:div(stdv[i])
end
for i=1,3 do
    testset.data[{ {}, {i}, {}, {} }]:add(-mean[i])
    testset.data[{ {}, {i}, {}, {} }]:div(stdv[i])
end

Define neural network

1
2
3
4
5
6
7
8
9
10
11
12
13
14
net = nn.Sequential()
net:add(nn.SpatialConvolution(3, 6, 5, 5)) -- 3 input image channels, 6 output channels, 5x5 convolution kernel
net:add(nn.ReLU())                         -- non-linearity
net:add(nn.SpatialMaxPooling(2,2,2,2))     -- A max-pooling operation that looks at 2x2 windows and finds the max.
net:add(nn.SpatialConvolution(6, 16, 5, 5))
net:add(nn.ReLU())                         -- non-linearity
net:add(nn.SpatialMaxPooling(2,2,2,2))
net:add(nn.View(16*5*5))                   -- reshapes from a 3D tensor of 16x5x5 into 1D tensor of 16*5*5
net:add(nn.Linear(16*5*5, 120))            -- fully connected layer (matrix multiplication between input and weights)
net:add(nn.ReLU())                         -- non-linearity
net:add(nn.Linear(120, 84))
net:add(nn.ReLU())                         -- non-linearity
net:add(nn.Linear(84, 10))                 -- 10 is the number of outputs of the network (in this case, 10 digits)
net:add(nn.LogSoftMax())                   -- converts the output to a log-probability. Useful for classification problems

Define the Loss function

1
criterion = nn.ClassNLLCriterion()

Train the neural network

1
2
3
4
trainer = nn.StochasticGradient(net, criterion)
trainer.learningRate = 0.001
trainer.maxIteration = 5
trainer:train(trainset)
1
2
3
4
5
6
7
-- train on GPU
net = net:cuda()
criterion = criterion:cuda()
trainset.data = trainset.data:cuda()
trainset.label = trainset.label:cuda()
testset.data = testset.data:cuda()
testset.label = testset.label:cuda()

Test the network, print accuracy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
correct = 0
class_performance = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
for i=1,10000 do
    local groundtruth = testset.label[i]
    local prediction = net:forward(testset.data[i])
    local confidences, indices = torch.sort(prediction, true)  -- true means sort in descending order
    if groundtruth == indices[1] then
        correct = correct + 1
        class_performance[groundtruth] = class_performance[groundtruth] + 1
    end
end

print(correct, 100*correct/10000 .. " % ")
for i=1,#classes do
    print(classes[i], 100*class_performance[i]/1000 .. ' %')
end

Result

It’ll take around 30 minutes to get the result on CPU. However, it’ll cost less than 10 minutes on GPU.

1
th filename.lua

On CPU

cpu

On GPU

gpu