即日起在codingBlog上分享您的技术经验即可获得积分,积分可兑换现金哦。

caffe+unbuntu14.04+GPU运行srcnn配置详解

编程语言 m0_37407756 187℃ 0评论
本文目录
[隐藏]

本人使用caffe+unbuntu14.04+GPU运行srcnn

因为最近看了一篇论文,SRCNN-Image Super-Resolution Using Deep Convolutional Networks

想实现一下,把其中的过程记录一下,希望能对后者有用,因本人也是初学者,整理的内容都是自己遇到的实际错误,这期间难免会有差错,还请批评指正。

1. 准备工作

在配置好的caffe文件夹中,进入examples目录,创建SRCNN文件夹,并进入文件夹,再创建modelTest
Train
四个文件夹。

model:用来保存驯良好的模型

Test:保存要测试的图片

Train:保存要训练的图片

2. 产生训练和测试数据

2.1. 在CNN文件夹下面(以下均是在此文件夹下)创建两个路径文件。

$vim train.txt

输入内容:

examples/SRCNN/train.h5

:wq保存文档。

$vim test.txt

输入内容:

examples/SRCNN/test.h5

:wq保存文档

2.2. 产生训练数据

$vim generate_train.m

输入以下的内容:

clear;closeall;

%%settings

folder= ‘Train’;

savepath= ‘train.h5’;

size_input= 33;

size_label= 21;

scale= 3;

stride= 14;

%%initialization

data= zeros(size_input, size_input, 1, 1);

label= zeros(size_label, size_label, 1, 1);

padding= abs(size_input – size_label)/2;

count= 0;

%%generate data

filepaths= dir(fullfile(folder,’*.bmp’));

fori = 1 : length(filepaths)

image= imread(fullfile(folder,filepaths(i).name));

image= rgb2ycbcr(image);

image= im2double(image(:, :, 1));

im_label= modcrop(image, scale);

[hei,wid]= size(im_label);

im_input=imresize(imresize(im_label,1/scale,’bicubic’),[hei,wid],’bicubic’);

forx = 1 : stride : hei-size_input+1

fory = 1 :stride : wid-size_input+1

subim_input= im_input(x : x+size_input-1, y : y+size_input-1);

subim_label= im_label(x+padding : x+padding+size_label-1, y+padding :y+padding+size_label-1);

count=count+1;

data(:,:, 1, count) = subim_input;

label(:,:, 1, count) = subim_label;

end

end

end

order= randperm(count);

data= data(:, :, 1, order);

label= label(:, :, 1, order);

%%writing to HDF5

chunksz= 128;

created_flag= false;

totalct= 0;

forbatchno = 1:floor(count/chunksz)

last_read=(batchno-1)*chunksz;

batchdata= data(:,:,1,last_read+1:last_read+chunksz);

batchlabs= label(:,:,1,last_read+1:last_read+chunksz);

startloc= struct(‘dat’,[1,1,1,totalct+1], ‘lab’, [1,1,1,totalct+1]);

curr_dat_sz= store2hdf5(savepath, batchdata, batchlabs, ~created_flag, startloc,chunksz);

created_flag= true;

totalct= curr_dat_sz(end);

end

h5disp(savepath);

终端下输入:

$matlab -nodesktop -nosplash -logfile generate_train.log -rgenerate_train

产生训练数据train.h5


2.3. 产生测试数据

$vim generate_test.m

输入以下内容:

clear;closeall;

%%settings

folder= ‘Test/Set5’;

savepath= ‘test.h5’;

size_input= 33;

size_label= 21;

scale= 3;

stride= 21 ;

%%initialization

data= zeros(size_input, size_input, 1, 1);

label= zeros(size_label, size_label, 1, 1);

padding= abs(size_input – size_label)/2;

count= 0;

%%generate data

filepaths= dir(fullfile(folder,’*.bmp’));

fori = 1 : length(filepaths)

image= imread(fullfile(folder,filepaths(i).name));

image= rgb2ycbcr(image);

image= im2double(image(:, :, 1));

im_label= modcrop(image, scale);

[hei,wid]= size(im_label);

im_input=imresize(imresize(im_label,1/scale,’bicubic’),[hei,wid],’bicubic’);

forx = 1 : stride : hei-size_input+1

fory = 1 :stride : wid-size_input+1

subim_input= im_input(x : x+size_input-1, y : y+size_input-1);

subim_label= im_label(x+padding : x+padding+size_label-1,
y+padding :y+padding+size_label-1);

count=count+1;

data(:,:, 1, count) = subim_input;

label(:,:, 1, count) = subim_label;

end

end

end

order= randperm(count);

data= data(:, :, 1, order);

label= label(:, :, 1, order);

%%writing to HDF5

chunksz= 2;

created_flag= false;

totalct= 0;

forbatchno = 1:floor(count/chunksz)

last_read=(batchno-1)*chunksz;

batchdata= data(:,:,1,last_read+1:last_read+chunksz);

batchlabs= label(:,:,1,last_read+1:last_read+chunksz);

startloc= struct(‘dat’,[1,1,1,totalct+1], ‘lab’, [1,1,1,totalct+1]);

curr_dat_sz= store2hdf5(savepath, batchdata, batchlabs, ~created_flag,
startloc,chunksz);

created_flag= true;

totalct= curr_dat_sz(end);

end

h5disp(savepath);

终端下输入:

$matlab -nodesktop -nosplash -logfile generate_test.log -rgenerate_test

产生测试数据test.h5。仅仅用于判断训练到达什么地步。

3. 建立训练文件

3.1. 建立solver文件

$vim SRCNN_solver.prototxt

输入内容:

#The train/test net protocol buffer definition

net:”examples/SRCNN/SRCNN_net.prototxt”

test_iter:556

#Carry out testing every 500 training iterations.

test_interval:500

#The base learning rate, momentum and the weight decay of
the network.

base_lr:0.0001

momentum:0.9

weight_decay:0

#The learning rate policy

lr_policy:”fixed”

#Display every 100 iterations

display:100

#The maximum number of iterations

max_iter:1500000

#snapshot intermediate results

snapshot:500

snapshot_prefix:”examples/SRCNN/SRCNN”

#solver mode: CPU or GPU

solver_mode:GPU

:wq保存退出。

3.2. 建立net文件

$vim SRCNN_net.prototxt

在此输入:

name:”SRCNN”

layer{

name:”data”

type:”HDF5Data”

top:”data”

top:”label”

hdf5_data_param{

source:”examples/SRCNN/train.txt”

batch_size:128

}

include:{ phase: TRAIN }

}

layer{

name:”data”

type:”HDF5Data”

top:”data”

top:”label”

hdf5_data_param{

source:”examples/SRCNN/test.txt”

batch_size:2

}

include:{ phase: TEST }

}

layer{

name:”conv1″

type:”Convolution”

bottom:”data”

top:”conv1″

param{

lr_mult:1

}

param{

lr_mult:0.1

}

convolution_param{

num_output:64

kernel_size:9

stride:1

pad:0

weight_filler{

type:”gaussian”

std:0.001

}

bias_filler{

type:”constant”

value:0

}

}

}

layer{

name:”relu1″

type:”ReLU”

bottom:”conv1″

top:”conv1″

}

layer{

name:”conv2″

type:”Convolution”

bottom:”conv1″

top:”conv2″

param{

lr_mult:1

}

param{

lr_mult:0.1

}

convolution_param{

num_output:32

kernel_size:1

stride:1

pad:0

weight_filler{

type:”gaussian”

std:0.001

}

bias_filler{

type:”constant”

value:0

}

}

}

layer{

name:”relu2″

type:”ReLU”

bottom:”conv2″

top:”conv2″

}

layer{

name:”conv3″

type:”Convolution”

bottom:”conv2″

top:”conv3″

param{

lr_mult:0.1

}

param{

lr_mult:0.1

}

convolution_param{

num_output:1

kernel_size:5

stride:1

pad:0

weight_filler{

type:”gaussian”

std:0.001

}

bias_filler{

type:”constant”

value:0

}

}

}

layer{

name:”loss”

type:”EuclideanLoss”

bottom:”conv3″

bottom:”label”

top:”loss”

}

:wq保存退出。

3.3. SRCNN训练

$vim train.sh

输入:

#!/usr/bin/envsh

/home/abc/caffe-master/build/tools/caffetrain–solver=/home/abc/caffe-master/examples/SRCNN/SRCNN_solver.prototxt#\

#–weights=models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel\

#–gpu0

增加运行权限:

$chmod +x train.sh

运行脚本文件:

$./train.sh

4. 保存滤波器

4.1. 创建mat文件

$cp SRCNN_net.prototxt SRCNN_mat.prototxt

输入以下内容:

name:”SRCNN”

input:”data”

input_dim:1

input_dim:1

input_dim:33

input_dim:33

input:”label”

input_dim:1

input_dim:1

input_dim:21

input_dim:21

layer{

name:”conv1″

type:”Convolution”

bottom:”data”

top:”conv1″

param{

lr_mult:1

}

param{

lr_mult:0.1

}

convolution_param{

num_output:64

kernel_size:9

stride:1

pad:0

weight_filler{

type:”gaussian”

std:0.001

}

bias_filler{

type:”constant”

value:0

}

}

}

layer{

name:”relu1″

type:”ReLU”

bottom:”conv1″

top:”conv1″

}

layer{

name:”conv2″

type:”Convolution”

bottom:”conv1″

top:”conv2″

param{

lr_mult:1

}

param{

lr_mult:0.1

}

convolution_param{

num_output:32

kernel_size:1

stride:1

pad:0

weight_filler{

type:”gaussian”

std:0.001

}

bias_filler{

type:”constant”

value:0

}

}

}

layer{

name:”relu2″

type:”ReLU”

bottom:”conv2″

top:”conv2″

}

layer{

name:”conv3″

type:”Convolution”

bottom:”conv2″

top:”conv3″

param{

lr_mult:0.1

}

param{

lr_mult:0.1

}

convolution_param{

num_output:1

kernel_size:5

stride:1

pad:0

weight_filler{

type:”gaussian”

std:0.001

}

bias_filler{

type:”constant”

value:0

}

}

}

layer{

name:”loss”

type:”EuclideanLoss”

bottom:”conv3″

bottom:”label”

top:”loss”

}

:wq保存即可。

4.2. 创建M文件,将weights和bias转为.mat格式

$vim saveFilters.m

%layers中的parameter(1)表示权重参数,parameter(2)表示偏置参数;

caffe.reset_all();%清除已经创建的所有solver和独立net;

clear;close all;

%%settings

folder= ‘examples/SRCNN/’;

model= [folder ‘SRCNN_mat.prototxt’];

weights= [folder ‘SRCNN_iter_500.caffemodel’];

savepath= [folder ‘x3.mat’];

layers= 3;

%%load model using mat_caffe

net= caffe.Net(model,weights,’test’);%创建一个网络load权重weights

%%reshap parameters

weights_conv= cell(layers,1);

foridx = 1 : layers

conv_filters= net.layers([‘conv’
num2str(idx)]).params(1).get_data();

[~,fsize,channel,fnum]= size(conv_filters);

ifchannel == 1

weights= single(ones(fsize^2,
fnum));

else

weights= single(ones(channel,
fsize^2, fnum));

end

fori = 1 : channel

forj = 1 : fnum

temp= conv_filters(:,:,i,j);

ifchannel == 1

weights(:,j)= temp(:);

else

weights(i,:,j)= temp(:);

end

end

end

weights_conv{idx}= weights;

end

%%save parameters

weights_conv1= weights_conv{1};

weights_conv2= weights_conv{2};

weights_conv3= weights_conv{3};

biases_conv1= net.layers(‘conv1’).params(2).get_data();

biases_conv2= net.layers(‘conv2’).params(2).get_data();

biases_conv3= net.layers(‘conv3’).params(2).get_data();

save(savepath,’weights_conv1′,’biases_conv1′,’weights_conv2′,’biases_conv2′,’weights_conv3′,’biases_conv3′);

:wq保存。

4.3. 运行M文件

$ matlab -nodesktop -nosplash -logfile saveFilters.log -r saveFilters

此时,在model中会生成x.mat文件。

5. 重构

已经知道了x.mat文件中,有三层卷积层的卷积核矩阵weights_conv*和偏置向量biases_conv*。

编写一个demo_net.m文件,使用这些参数构建卷积网络结构,对输入图像(矩阵)进行处理,即可得到结果。

转载请注明:CodingBlog » caffe+unbuntu14.04+GPU运行srcnn配置详解

喜欢 (0)or分享 (0)
发表我的评论
取消评论

*

表情