feat(Examples): Add new faceID demos for MAX78000, MAX78002

[oguzhanbsolak]

feat(Examples): Add new faceID demos for MAX78000, MAX78002

Description

Deletes following old faceID-related examples:

• Examples/MAX78000/CNN/faceid
• Examples/MAX78000/CNN/faceid_demo
• Examples/MAX78000/CNN/faceid_evkit-riscv
• Examples/MAX78000/CNN/faceid_evkit
• Examples/MAX78002/CNN/faceid
• Examples/MAX78002/CNN/faceid_evkit

Updates following example:

• Examples/MAX78000/CNN/facial_recognition

Adds following new faceID examples:

• Examples/MAX78000/CNN/faceid_112
• Examples/MAX78002/CNN/mobilefacenet-112
• Examples/MAX78002/CNN/facial_recognition

Checklist Before Requesting Review

• PR Title follows correct guidelines.
• Description of changes and all other relevant information.
• (Optional) Link any related GitHub issues using a keyword
• (Optional) Provide info on any relevant functional testing/validation. For API changes or significant features, this is not optional.

[Jake-Carter]

Hi @oguzhanbsolak, the projects were failing to build with font errors. FYI as of #792 we removed the local copies of the fonts due to copyright issues. I updated the projects to use the new open-source fonts (ec60383)

I also ran the latest MSDKGen to update the project files. (0ed5dc3)

I noticed that the MAX78002 Eclipse project settings have some additional plugin settings that we don't usually include. Do you use Eclipse for your development?

I tested the demos, and the MAX78002 demo works fantastic. Even after just two images it immediately recognizes my embeddings. We may want to switch the orientation of the TFT display so that it's more convenient to type on, though.

I also see that we're saving the embeddings database to the last page of flash. I think we should create a custom linkerfile for this application. It will help customers avoid difficult issues in their own development.

The MAX78000FTHR demo did not work for me. I don't have an Adafruit TFT display, so I attempted to run it over the serial port. I noticed some issues with the source code that needed some fixes, but once I got it running I couldn't get it to recognize my face at all.

I will do some additional clean-up for the MAX78002 project. In the meantime, can you take a look at the MAX78000FTHR project running with the TFT disabled and see if it works?

[oguzhanbsolak]

Hi @oguzhanbsolak, the projects were failing to build with font errors. FYI as of #792 we removed the local copies of the fonts due to copyright issues. I updated the projects to use the new open-source fonts (ec60383)

I also ran the latest MSDKGen to update the project files. (0ed5dc3)

I noticed that the MAX78002 Eclipse project settings have some additional plugin settings that we don't usually include. Do you use Eclipse for your development?

I tested the demos, and the MAX78002 demo works fantastic. Even after just two images it immediately recognizes my embeddings. We may want to switch the orientation of the TFT display so that it's more convenient to type on, though.

I also see that we're saving the embeddings database to the last page of flash. I think we should create a custom linkerfile for this application. It will help customers avoid difficult issues in their own development.

The MAX78000FTHR demo did not work for me. I don't have an Adafruit TFT display, so I attempted to run it over the serial port. I noticed some issues with the source code that needed some fixes, but once I got it running I couldn't get it to recognize my face at all.

I will do some additional clean-up for the MAX78002 project. In the meantime, can you take a look at the MAX78000FTHR project running with the TFT disabled and see if it works?

Hi @Jake-Carter , thank you for your feedback. I fixed the TFT_disabled case, now you should be able to see "Unknown" or the name of the recognized person at the serial port.

I don't use Eclipse, I believe I copied the configuration from some old project by mistake. I don't have a preference for the configuration.

I also added a small fix for preventing redefinition warnings for MAXREFDES178 LEDs.

I believe the custom linker file may be useful, I used the last page because MAX78002 has a large flash, and simply there is plenty of space.

[Jake-Carter]

Thanks @oguzhanbsolak. Unfortunately I still can't get the MAX78000FTHR facial_recognition to work. I had to remove the camera sleep code (7b163a4), and added some additional indicators that the demo was running. The model is running but never recognizes my face. It may be the usual issues due to lighting conditions.

Can you run the latest changes on your board to test?

[oguzhanbsolak]

Thanks @oguzhanbsolak. Unfortunately I still can't get the MAX78000FTHR facial_recognition to work. I had to remove the camera sleep code (7b163a4), and added some additional indicators that the demo was running. The model is running but never recognizes my face. It may be the usual issues due to lighting conditions.

Can you run the latest changes on your board to test?

Hi @Jake-Carter , in my case demo works fine with the camera sleep code, and when I remove it, the facedetection is not working properly. I think ,we should add some delay to wait for the camera input, if it's needed to remove it. However, as the sleep code is working fine with my featherboard, I believe it might be an HW-related issue on your side. If it's possible, could you please try it with another featherboard? I also noticed that performance degrades in dark environments as you stated, but when lightning is good the recognition performance seems decent.

My last commit includes adding back the sleep code, and some small updates about formatting, timing etc.

[Jake-Carter]

Thanks @oguzhanbsolak, I re-tested again today and it works on multiple boards including the old one. I'm not sure why the sleep code was hanging previously... Perhaps I was mis-interpreting the debug info after it drops connection during sleep.

I fixed some of the timing inaccuracies and improved the formatting of the console output. The results show a total loop time of around 136ms when no face is detected, and 469ms when one is detected.

The reliability and accuracy is also much better than the old demo we ran of the FTHR board at CES last year.

I updated the READMEs and added a pre-compiled weights_2.bin back to the project, and I think this is ready to merge. Thanks for your work!

before:

I[facedetection: 155] Capture Time : 0ms
I[facedetection: 156] Total Time : 77ms
I[facedetection: 151] Process Time Total : 69ms
I[facedetection: 155] Capture Time : 1ms
I[facedetection: 156] Total Time : 77ms
I[facedetection: 151] Process Time Total : 73ms
I[facedetection: 155] Capture Time : 0ms
I[facedetection: 156] Total Time : 81ms
Unknown subject 
I[faceid    :  99] Process Time Total : 324ms
I[faceid    : 103] Capture Time : 0ms
I[faceid    : 104] Total Time : 331ms
I[facedetection: 151] Process Time Total : 73ms
I[facedetection: 155] Capture Time : 0ms
I[facedetection: 156] Total Time : 81ms

after:

I[main      : 187]      -----
I[facedetection: 155]   Image Capture Time : 48ms
I[facedetection: 156]   FaceDetect Process Time : 70ms
I[facedetection: 157]   Total FaceDetect Time : 118ms
I[main      : 199]      No face detected.
I[main      : 204]      ----- (Total loop time: 136ms)

I[main      : 187]      -----
I[facedetection: 155]   Image Capture Time : 48ms
I[facedetection: 156]   FaceDetect Process Time : 73ms
I[facedetection: 157]   Total FaceDetect Time : 122ms
I[main      : 194]      Face detected!
I[faceid    : 388]      FaceID result: Unknown subject
I[faceid    :  97]      FaceID Processing Time : 326ms
I[main      : 204]      ----- (Total loop time: 469ms)

[oguzhanbsolak]

Thanks @oguzhanbsolak, I re-tested again today and it works on multiple boards including the old one. I'm not sure why the sleep code was hanging previously... Perhaps I was mis-interpreting the debug info after it drops connection during sleep.

I fixed some of the timing inaccuracies and improved the formatting of the console output. The results show a total loop time of around 136ms when no face is detected, and 469ms when one is detected.

The reliability and accuracy is also much better than the old demo we ran of the FTHR board at CES last year.

I updated the READMEs and added a pre-compiled weights_2.bin back to the project, and I think this is ready to merge. Thanks for your work!

before:

I[facedetection: 155] Capture Time : 0ms
I[facedetection: 156] Total Time : 77ms
I[facedetection: 151] Process Time Total : 69ms
I[facedetection: 155] Capture Time : 1ms
I[facedetection: 156] Total Time : 77ms
I[facedetection: 151] Process Time Total : 73ms
I[facedetection: 155] Capture Time : 0ms
I[facedetection: 156] Total Time : 81ms
Unknown subject 
I[faceid    :  99] Process Time Total : 324ms
I[faceid    : 103] Capture Time : 0ms
I[faceid    : 104] Total Time : 331ms
I[facedetection: 151] Process Time Total : 73ms
I[facedetection: 155] Capture Time : 0ms
I[facedetection: 156] Total Time : 81ms

after:

I[main      : 187]      -----
I[facedetection: 155]   Image Capture Time : 48ms
I[facedetection: 156]   FaceDetect Process Time : 70ms
I[facedetection: 157]   Total FaceDetect Time : 118ms
I[main      : 199]      No face detected.
I[main      : 204]      ----- (Total loop time: 136ms)

I[main      : 187]      -----
I[facedetection: 155]   Image Capture Time : 48ms
I[facedetection: 156]   FaceDetect Process Time : 73ms
I[facedetection: 157]   Total FaceDetect Time : 122ms
I[main      : 194]      Face detected!
I[faceid    : 388]      FaceID result: Unknown subject
I[faceid    :  97]      FaceID Processing Time : 326ms
I[main      : 204]      ----- (Total loop time: 469ms)

Hi @Jake-Carter , thank you for the feedback and your improvements.

[sihyung-maxim]

Verify Register workflow is going to fail since the changes in the PR are out of sync to the main.

[oguzhanbsolak]

Verify Register workflow is going to fail since the changes in the PR are out of sync to the main.

Hi @sihyung-maxim, I merged the branch with the main with the last commit.

[oguzhanbsolak]

Hi @sihyung-maxim , clang format issues are solved with the last commit.