Home

Welcome! This homepage describes the source code for the implementation of OPFsumm.

Introduction

OPFsumm is a video summarization approach based on the Optimum-Path Forest (OPF) classifier [1,2]. Basically, it aims at automatic generating static summaries from video sequences. The first work that employed OPF for static video summarization used the prototypes selected by the OPF clustering algorithm [3] as the keyframes to compose the video summary [4]. Later on, the supervised OPF was used for automatic genre classification in videos, achieving promising results [5]. Castelo-Fernández and Calderón-Ruiz also proposed an OPF-based approach for video summarization, in which scenes, shots and frames are identified at different levels of clustering processes [6].

In 2016, Martins et al. [7] proposed a new OPF-based video summarization approach that makes use of temporal information to improve the quality of the video summaries. In short, the main idea remains using the prototypes as the keyframes, but now we can split the video into different frame sets respecting their temporal information. Then, we can apply OPF clustering on each set and obtain a summary of that specific part of the video. Finally, we can merge all partial summaries (chronological order) to obtain the final summary.

In this document, we explain how to use OPFsumm for video summarization purposes. However, if you are looking for more information about the OPF classifier, please consult http://www.ic.unicamp.br/~afalcao/libopf or the [LibOPF](https://github.com/jppbsi/LibOPF) Github homepage.

File format

Since OPFsumm employs unsupervised OPF, one is required to use the OPF file format for unlabeled datasets, as follows:

<# of samples> <# of labels> <# of features> 
<0> <label> <feature 1 from element 0> <feature 2 from element 0> ...
<1> <label> <feature 1 from element 1> <feature 2 from element 1> ...
.
.
<i> <label> <feature 1 from element i> <feature 2 from element i> ...
<i+1> <label> <feature 1 from element i+1> <feature 2 from element i+1> ...
.
.
<n-1> <label> <feature 1 from element n-1> <feature 2 from element n-1> ... 

The first number of each line is a sample identifier (i.e., the frame number) numbered from 0 to n-1, where n stands for the number of frames. Although we have no labels, we shall set the number of labels to 1 in the header, but we can use 0 in the second column for the further lines.

Example: Suppose you have a dataset with 5 frames, being each one represented by a feature vector of size 4. Therefore, the OPF file format should looks like as follows:

	5	1	4
	0	0	0.21	0.45	0.40	0.13
	1	0	0.22	0.43	0.36	0.10
	2	0	0.67	1.12	0.43	0.11
	3	0	0.60	1.11	0.97	0.56
	4	0	0.79	0.04	0.09	0.25

Comment #1: Note that, the file must be binary with no blank spaces. This ASCII representation is just for illustration.

Comment #2: The first line of the file, 5 1 4, contains, respectively, the dataset size, the number of labels (classes) and the number of features in the feature vectors. The remaining lines contain the sample identifier (integer from 0 to n-1, in which n is the dataset size), its label and the feature values for each sample.

Comment #3: The values corresponding to <# of samples> must be the VIDEO FRAME NUMBER.

Comment #4: To correctly generate a dataset opf file, you must use 'txt2opf' in order to convert the formated text file (ASCII) to binary opf file. Usage: txt2opf [1][2]

[1]: input OPF file name in the ASCII format [2]: output OPF file name in the binary format

• • References**

[1] J. P. Papa, A. X. Falcão, and Celso T. N. Suzuki. Supervised pattern classification based on optimum-path forest. International Journal of Imaging Systems and Technology, 19(2):120-131, 2009.
[2] J. P. Papa, A. X. Falcão, V. H. C. Albuquerque and J. M. R. da Silva Tavares. Efficient supervised optimum-path forest classification for large datasets. Pattern Recognition, 45(1):512-520, 2012.
[3] L.M. Rocha, F.A.M. Cappabianco, and A.X. Falcão. Data clustering as an optimum-path forest problem with applications in image analysis. International Journal of Imaging Systems and Technology, 19(2):50-68, 2009.
[4] G.B. Martins, L.C.S. Afonso, D. Osaku, J.G. Almeida and J.P. Papa. Static Video Summarization through Optimum-Path Forest Clustering. In: 19th Iberoamerican Congress on Pattern Recognition, Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, v. 8827, 893-900, 2014.
[5] G.B. Martins, J.G. Almeida and J.P. Papa. Supervised Video Genre Classification Using Optimum-Path Forest. In: XX Iberoamerican Congress on Pattern Recognition. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, v. 9423, 735-742, 2015.
[6] C. Castelo-Fernández and G. Calderón-Ruiz. Automatic Video Summarization Using the Optimum-Path Forest Unsupervised Classifier. In: XX Iberoamerican Congress on Pattern Recognition. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, v. 9423, 760-767, 2015.
[7] G.B. Martins, J.P. Papa and J.G. Almeida. Temporal-and Spatial-Driven Video Summarization Using Optimum-Path Forest. In: 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images, 335-339, 2016.