Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

problems when finetuning #86

Open
Ten000hours opened this issue Apr 7, 2018 · 0 comments
Open

problems when finetuning #86

Ten000hours opened this issue Apr 7, 2018 · 0 comments

Comments

@Ten000hours
Copy link

hi, when i run the code
GLOG_logtostderr=1 ../../build/tools/finetune_net.bin pascal_finetune_solver.prototxt ~/rcnn/data/caffe_nets/ilsvrc_2012_train_iter_310k 2>&1 | tee log.txt

the caffe net initailizition seems stopped suddenly,the log.txt shows like this
I0406 16:51:33.727171 26542 finetune_net.cpp:25] Starting Optimization I0406 16:51:33.727485 26542 solver.cpp:41] Creating training net. I0406 16:51:33.728703 26542 net.cpp:75] Creating Layer data I0406 16:51:33.728741 26542 net.cpp:111] data -> data I0406 16:51:33.728757 26542 net.cpp:111] data -> label I0406 16:51:33.728803 26542 window_data_layer.cpp:279] Window data layer: foreground (object) overlap threshold: 0.5 background (non-object) overlap threshold: 0.5 foreground sampling fraction: 0.25 I0406 16:51:33.737037 26542 window_data_layer.cpp:345] num: 0 /home/xuxiangrui/Pascal_voc/celeba/VOCdevkit/VOC2007/JPEGImages/000002.jpg 3 594 423 windows to process: 2762 I0406 16:51:33.914831 26542 window_data_layer.cpp:345] num: 100 /home/xuxiangrui/Pascal_voc/celeba/VOCdevkit/VOC2007/JPEGImages/000189.jpg 3 400 496 windows to process: 2304 I0406 16:51:34.066776 26542 window_data_layer.cpp:345] num: 200 /home/xuxiangrui/Pascal_voc/celeba/VOCdevkit/VOC2007/JPEGImages/000393.jpg 3 640 408 windows to process: 2301 I0406 16:51:34.247323 26542 window_data_layer.cpp:345] num: 300 /home/xuxiangrui/Pascal_voc/celeba/VOCdevkit/VOC2007/JPEGImages/000595.jpg 3 360 640 windows to process: 1185 I0406 16:51:34.380002 26542 window_data_layer.cpp:345] num: 400 /home/xuxiangrui/Pascal_voc/celeba/VOCdevkit/VOC2007/JPEGImages/000808.jpg 3 337 500 windows to process: 2281 I0406 16:51:34.591991 26542 window_data_layer.cpp:354] Number of images: 500 I0406 16:51:34.592015 26542 window_data_layer.cpp:358] class 0 has 1261791 samples I0406 16:51:34.592025 26542 window_data_layer.cpp:362] Amount of context padding: 16 I0406 16:51:34.592027 26542 window_data_layer.cpp:365] Crop mode: warp I0406 16:51:34.592031 26542 window_data_layer.cpp:376] output data size: 64,3,227,227 I0406 16:51:34.592034 26542 window_data_layer.cpp:388] Loading mean file from../../data/ilsvrc12/imagenet_mean.binaryproto I0406 16:51:36.851869 26542 net.cpp:126] Top shape: 64 3 227 227 (9893568) I0406 16:51:36.851935 26542 net.cpp:126] Top shape: 64 1 1 1 (64) I0406 16:51:36.851958 26542 net.cpp:157] data does not need backward computation. I0406 16:51:36.851992 26542 net.cpp:75] Creating Layer conv1 I0406 16:51:36.852007 26542 net.cpp:85] conv1 <- data I0406 16:51:36.852041 26542 net.cpp:111] conv1 -> conv1 I0406 16:51:36.866598 26542 net.cpp:126] Top shape: 64 96 55 55 (18585600) I0406 16:51:36.866847 26542 net.cpp:152] conv1 needs backward computation. I0406 16:51:36.867000 26542 net.cpp:75] Creating Layer relu1 I0406 16:51:36.867081 26542 net.cpp:85] relu1 <- conv1 I0406 16:51:36.867144 26542 net.cpp:99] relu1 -> conv1 (in-place) I0406 16:51:36.867208 26542 net.cpp:126] Top shape: 64 96 55 55 (18585600) I0406 16:51:36.867270 26542 net.cpp:152] relu1 needs backward computation. I0406 16:51:36.867333 26542 net.cpp:75] Creating Layer pool1 I0406 16:51:36.867391 26542 net.cpp:85] pool1 <- conv1 I0406 16:51:36.867465 26542 net.cpp:111] pool1 -> pool1 I0406 16:51:36.867533 26542 net.cpp:126] Top shape: 64 96 27 27 (4478976) I0406 16:51:36.867555 26542 net.cpp:152] pool1 needs backward computation. I0406 16:51:36.867578 26542 net.cpp:75] Creating Layer norm1 I0406 16:51:36.867590 26542 net.cpp:85] norm1 <- pool1 I0406 16:51:36.867606 26542 net.cpp:111] norm1 -> norm1 I0406 16:51:36.867746 26542 net.cpp:126] Top shape: 64 96 27 27 (4478976) I0406 16:51:36.867774 26542 net.cpp:152] norm1 needs backward computation. I0406 16:51:36.867817 26542 net.cpp:75] Creating Layer conv2 I0406 16:51:36.867832 26542 net.cpp:85] conv2 <- norm1 I0406 16:51:36.867864 26542 net.cpp:111] conv2 -> conv2 I0406 16:51:36.885260 26542 net.cpp:126] Top shape: 64 256 27 27 (11943936) I0406 16:51:36.885288 26542 net.cpp:152] conv2 needs backward computation. I0406 16:51:36.885303 26542 net.cpp:75] Creating Layer relu2 I0406 16:51:36.885310 26542 net.cpp:85] relu2 <- conv2 I0406 16:51:36.885320 26542 net.cpp:99] relu2 -> conv2 (in-place) I0406 16:51:36.885331 26542 net.cpp:126] Top shape: 64 256 27 27 (11943936) I0406 16:51:36.885340 26542 net.cpp:152] relu2 needs backward computation. I0406 16:51:36.885351 26542 net.cpp:75] Creating Layer pool2 I0406 16:51:36.885360 26542 net.cpp:85] pool2 <- conv2 I0406 16:51:36.885370 26542 net.cpp:111] pool2 -> pool2 I0406 16:51:36.885381 26542 net.cpp:126] Top shape: 64 256 13 13 (2768896) I0406 16:51:36.885390 26542 net.cpp:152] pool2 needs backward computation. I0406 16:51:36.885404 26542 net.cpp:75] Creating Layer norm2 I0406 16:51:36.885413 26542 net.cpp:85] norm2 <- pool2 I0406 16:51:36.885422 26542 net.cpp:111] norm2 -> norm2 I0406 16:51:36.885435 26542 net.cpp:126] Top shape: 64 256 13 13 (2768896) I0406 16:51:36.885448 26542 net.cpp:152] norm2 needs backward computation. I0406 16:51:36.885460 26542 net.cpp:75] Creating Layer conv3 I0406 16:51:36.885469 26542 net.cpp:85] conv3 <- norm2 I0406 16:51:36.885479 26542 net.cpp:111] conv3 -> conv3 I0406 16:51:36.906862 26542 net.cpp:126] Top shape: 64 384 13 13 (4153344) I0406 16:51:36.906883 26542 net.cpp:152] conv3 needs backward computation. I0406 16:51:36.906889 26542 net.cpp:75] Creating Layer relu3 I0406 16:51:36.906893 26542 net.cpp:85] relu3 <- conv3 I0406 16:51:36.906898 26542 net.cpp:99] relu3 -> conv3 (in-place) I0406 16:51:36.906903 26542 net.cpp:126] Top shape: 64 384 13 13 (4153344) I0406 16:51:36.906905 26542 net.cpp:152] relu3 needs backward computation. I0406 16:51:36.906910 26542 net.cpp:75] Creating Layer conv4 I0406 16:51:36.906913 26542 net.cpp:85] conv4 <- conv3 I0406 16:51:36.906916 26542 net.cpp:111] conv4 -> conv4 I0406 16:51:36.917457 26542 net.cpp:126] Top shape: 64 384 13 13 (4153344) I0406 16:51:36.917470 26542 net.cpp:152] conv4 needs backward computation. I0406 16:51:36.917475 26542 net.cpp:75] Creating Layer relu4 I0406 16:51:36.917479 26542 net.cpp:85] relu4 <- conv4 I0406 16:51:36.917482 26542 net.cpp:99] relu4 -> conv4 (in-place) I0406 16:51:36.917486 26542 net.cpp:126] Top shape: 64 384 13 13 (4153344) I0406 16:51:36.917490 26542 net.cpp:152] relu4 needs backward computation. I0406 16:51:36.917495 26542 net.cpp:75] Creating Layer conv5 I0406 16:51:36.917497 26542 net.cpp:85] conv5 <- conv4 I0406 16:51:36.917503 26542 net.cpp:111] conv5 -> conv5 I0406 16:51:36.927706 26542 net.cpp:126] Top shape: 64 256 13 13 (2768896) I0406 16:51:36.927819 26542 net.cpp:152] conv5 needs backward computation. I0406 16:51:36.927845 26542 net.cpp:75] Creating Layer relu5 I0406 16:51:36.927855 26542 net.cpp:85] relu5 <- conv5 I0406 16:51:36.927867 26542 net.cpp:99] relu5 -> conv5 (in-place) I0406 16:51:36.927880 26542 net.cpp:126] Top shape: 64 256 13 13 (2768896) I0406 16:51:36.927886 26542 net.cpp:152] relu5 needs backward computation. I0406 16:51:36.927898 26542 net.cpp:75] Creating Layer pool5 I0406 16:51:36.927903 26542 net.cpp:85] pool5 <- conv5 I0406 16:51:36.927911 26542 net.cpp:111] pool5 -> pool5 I0406 16:51:36.927924 26542 net.cpp:126] Top shape: 64 256 6 6 (589824) I0406 16:51:36.927928 26542 net.cpp:152] pool5 needs backward computation. I0406 16:51:36.927949 26542 net.cpp:75] Creating Layer fc6 I0406 16:51:36.927958 26542 net.cpp:85] fc6 <- pool5 I0406 16:51:36.927968 26542 net.cpp:111] fc6 -> fc6

Can anyone help me with this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant