last day (28 days later) » 

8:01 AM
0
A: Python real time image classification problems with Neural Networks

Dale SongTry this c++ solution. It uses threads for the I/O overhead in your task, I tested it using bvlc_alexnet.caffemodel to do image classification and didn't see obvious slowing down of the main thread(webcam stream) when caffe running(on GPU): #include <stdio.h> #include <string> #include <boost/th...

 
This looks interesting. I will try it out and report back. Just one question, how do I pass a cv::Mat as an input to a caffe network in C++. Also when I call the pretrained network, are there any parameters for raw_scale and channel_swap like there are in python? I've never used C++ caffe before.
 
@user3543300 The interface DataTransformer<Dtype>::Transform(const cv::Mat& cv_img, Blob<Dtype>* transformed_blob) in data_transformer.cpp will convert the cv::Mat to a caffe::Blob object which will be taken as input to a caffe network by calling Net::Forward( const vector<Blob<Dtype>*> & bottom, Dtype* loss). DataTransformer::Transform() will automatically perform the channel_swap predure within it, but if to normalize image data from [0,255] to [0,1], you should explicitly set a scale using member function set_scale(float value) in caffe::DataTransformer.
 
I'm a bit confused, but in python I do this: net = caffe.Classifier(net_model_file,net_pretrained, mean=mean, channel_swap=(2,1,0), raw_scale=255, image_dims=(256, 256)) Are you saying that's all done automatically?
 
@user3543300 channel_swap=(2,1,0), raw_scale=255 is not needed. You only need do a little work similar to mean=mean, image_dims=(256, 256) such as trans_para.set_mean_file("/path/to/imagenet_mean.binaryproto‌​") and cv::resize(*frame, resized_image, re_size, 0, 0, CV_INTER_LINEAR); in my answer.
 
I ran the code and my fps reduced to around 15 again. Not sure what is going on. I have a Nvidia GeForce 940MX GPU and Intel® Core™ i7-6500U CPU @ 2.50GHz × 4
 
8:01 AM
@user3543300 I tried it on a PC with a Intel® Core™ i5-4440 cpu, NVIDIA GeForce GT 630 gpu, 8G memory. It's hardware that matters? Now I'm also not much sure.
@user3543300 Is it GPU memory bandwidth that matters?
 
Not sure. I have 2 GB of GPU memory. Were you able to get 30 fps on the main thread?
 
I had about 20 fps on the main thread. And I meant maybe it is GPU memory bandwidth that matters, not GPU memory capacity.
 
I see. I'm wondering why I can't get 30 fps, since the prediction and the webcam stream aren't exactly related.
 
I guess it's GPU memory bandwidth limitation that slows down the main thread. But I'm not much sure about it.
 
That is unfortunate. I didn't realize the webcam stream required that much from the GPU
 
8:16 AM
No that. I think I express it not properly.
I means that maybe it's the I/O operations, e.g. copyiing data from CPU to GPU, retrieving images from webcam, that slows down the webcam stream, with the hardware that has possibly limited memory bandwidth.
Maybe you cab try it on different hardwares to find out the truth.
 
I see. I'll try and test a separate application that uses the webcam, while running predictions on the net in another application, and see if the webcam app is still being slown down.
 
Or, run your code on a different PC/laptop to see whether there is difference.
 
The only other laptop I have with an NVIDIA GPU has a webcam limitation of 15 fps. I could try it on a machine with a more powerful CPU (without a GPU), but I suspect a lack of GPU will affect the performance a lot.
 
 
1 hour later…
9:55 AM
Maybe not, because there will not be I/O operations that transfer data from CPU to GPU.
 
10:06 AM
Maybe it will be a CPU bound problem. I guess.
 

  last day (28 days later) »