Feature point labels are saved as default even if labels are provided

[neelan29]

I annotate image with feature points and provide different names for them (eg. feature point 0 was renamed as left eye, feature point 1 was renamed as right eye, etc). However, when I save the XML files and open them, the feature point labels are still in their default names as "0", "1", "2".

Is there no other way to change the default labels?

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[amitguptagwl]

It is quite difficult to understand the issue with the information you've provided. Can you please share the browser detail? If you see any error on browser console? What is the issue you're facing with point tool? Are you using shortcuts? Or any other detail that you can share.

…

On Sat, 26 Jan 2019 03:46 neelan29, ***@***.***> wrote: I am unable to annotate images using the "point" feature. However, other shapes work for me. Can you please help me with this? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#146>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHVgKE7UmsN9MQSZq0oB_0TVrwnCMRgjks5vG4I2gaJpZM4aTv-a> .

[amitguptagwl]

@neelan29 In my understanding, dlib don't support custom name for feature point, hence they're saved with their index position as name. But if you save data as project file ie nimn, so the same data can be retrieved.

[gitclem]

@amitguptagwl, I believe your understanding is wrong. I posted this question to Davis King:

I posted May 8, 2019:

I've looked at the XML output of the part labels (from file examples/faces/training_with_face_landmarks.xml) and it seems to label parts with just numbers instead of names of parts:

<images>
<image file='2007_007763.jpg'>
<box top='90' left='194' width='37' height='37'>
<part name='00' x='201' y='107'/>
<part name='01' x='201' y='110'/>
<part name='02' x='201' y='113'/>
<part name='03' x='202' y='117'/>

...

I presume this means you have to be consist with the ordering?

And what if you are trying to do a multi-nominal image matcher, for things that have different types of interior parts? Then they would have the same numbers.

Davis King replied on May 9, 2019

The labels can be anything, they don't have to be numbers. But you have to be consistent about their meaning. Like you can't just randomly shuffle the labels on each training sample, obviously.

Things with different types of parts need different models.

I think it would be better if you could put the named labels instead of numbers. (I suppose to make it backwards compatible, maybe add a checkbox for named labels instead of number and have it off by default.)

I'm not sure why you didn't understand @neelan29's comment. I'll give a try... I have a:

• Mac OSX 10.14.4 64 bit
• Chrome Version 73.0.3683.103

Head of git log says:

commit 62eaeba4e41bc6f84e1113b8dc75053bbd692e18
Author: Amit Gupta <[email protected]>
Date:   Thu Nov 29 13:32:43 2018 +0530

    Update prompt.js

I spent a bit of time labeling the interior points but when I export/save as dlib XML file, all those labels that I spent time on are now not in the XML file. In addition, I was going to try to use http://blog.dlib.net/2017/09/fast-multiclass-object-detection-in.html which means I have two (or more) object types which will have different types of interior points, so having different types of interior points all labeled as "0", etc. won't work for me.

So TYPE_ONE has four interior points from TYPE_TWO which has ten interior points (with different names from TYPE_ONE.)

(Also the "name" is wrong: dlib face detection dataset generated by ImgLab

<?xml version='1.0' encoding='ISO-8859-1'?>
<?xml-stylesheet type='text/xsl' href='image_metadata_stylesheet.xsl'?>
<dataset>
<name>dlib face detection dataset generated by ImgLab</name>
<comment>
    This dataset is manually crafted or adjusted using ImgLab web tool
    Check more detail on https://github.com/NaturalIntelligence/imglab
</comment>
...
	<image file='xxxxxx-1308696.png'>
		<box top='4' left='99' width='393' height='561'>
			<label>TYPE_ONE</label>
			<part name='0' x='141' y='104'/>
			<part name='1' x='142' y='429'/>
			<part name='2' x='454' y='107'/>
			<part name='3' x='453' y='431'/>
		</box>
	</image>
	<image file='299913.jpg'>
		<box top='144' left='11' width='572' height='316'>
			<label>TYPE_TWO</label>
			<part name='0' x='63' y='193'/>
			<part name='1' x='393' y='196'/>
			<part name='2' x='70' y='411'/>
			<part name='3' x='392' y='416'/>
			<part name='4' x='427' y='220'/>
			<part name='5' x='427' y='381'/>
			<part name='6' x='479' y='195'/>
			<part name='7' x='566' y='198'/>
			<part name='8' x='482' y='413'/>
			<part name='9' x='563' y='410'/>
		</box>
	</image>
...

I hope that clarifies what's going on.

[amitguptagwl]

Let's give it a try. You manually modify the file for correct labels or rather expected labels. Process it with dlib. If you find everything is ok. Share the file and dlib version you're trying here.

[gitclem]

Ok, but it'll be awhile. I'm still learning the library.... Thanks for looking at this!

[gitclem]

From the dlib library, I looked more closely at the imglab program's code in dlib/tools/imglab/src/metadata_editor.cpp

It's well documented if you read it carefully (which I missed the first time I tried it!) but it's a bit clumsy to use and not so obvious. It seems to me that your imglab program is trying to emulate it but you missed part of what it can do. However, your program is a lot easier to use.

The important text from metadata_editor.cpp says:

"It is also possible to label object parts by selecting a rectangle and "
"then right clicking.  A popup menu will appear and you can select a part label. "
"Note that you must define the allowable part labels by giving --parts on the "
"command line.  An example would be '--parts \"leye reye nose mouth\"'. "
"Alternatively, if you don't give --parts you can simply select a rectangle and shift+left "
"click to add parts. Parts added this way will be labeled with integer labels starting from 0. "
"You can only use this simpler part adding mode if all the parts in a rectangle are already "
    "labeled with integer labels or the rectangle has no parts at all."

Using the all important --parts "part1 part2 part3 part4" command line argument will result in something like:

...
  <image file='image-train/20501301-2T.jpg'>
    <box top='96' left='79' width='110' height='159'>
      <label>obj1</label>
      <part name='part1' x='98' y='144'/>
      <part name='part2' x='81' y='144'/>
      <part name='part3='118' y='101'/>
      <part name='part4' x='171' y='127'/>
    </box>
 </image>
...

The command line --parts option is important because it creates a popup menu that so you can't accidentally misspell a label and create a bogus point type.

Error prevention is very important because one is potentially creating thousands of training set examples!

[amitguptagwl]

Hmm. agree. As I remember few years back when I was trying this library I faced the issue with string labels. Hence, I want someone to test it before we do the changes in this library. Changes are not big but we need to ensure they're not impacting other users

[gitclem]

The way I'd frame it, is since the dlib claims it allows named labels and if they don't work, then it's on the dlib people to fix any problem.

I'm more worried about your existing users that might get blindsided if you start supporting named labels and it breaks their existing work.

I have another area of concern I have of potential inconsistency between your code and dlib's imglab. It seems to me that the path to the image includes a relative path to the image file and your doesn't. I'm not clear how you do it but dlib's command line interface clearly requires a directory of the images.

Anyway, I'm going to fuss around with the dlib and see how it works with names vs numbers.

[amitguptagwl]

Since this is the web application, it has a restriction to know the path of the file.

[gitclem]

Since this is the web application, it has a restriction to know the path of the file.

This is an issue for me. The dlib library is expecting the XML file to give the relative path name for each image files. If one is working with thousands of images, it makes sense to organize the files by directories.

I have put together a XML file with the named labels, so my next step is to try to get dlib to train using XML and image files but it's not working because of this directory problem.

I see three ways out of this problem:

1. Create another program that patches the XML files by prefixing the directories to the file name of the image.

2. Pass your imglab program file which contains a list of the path. Then have imglab program insert the path prefix.

3. (I don't know if this is possible), violate the restrictions built into the web based apps.

I think I might go with option 1, as it wouldn't require me to figure out node.js, etc.

[amitguptagwl]

You can go ahead with option 1. However, I see one more possibility. We can give a text box in the label section (right side panel) where you can specify the relative path. While generating the output file, it can be considered. Another option can be to generate the path based on the image category set by the user.