Skip to content

Commit

Permalink
Merge pull request #99 from zc277584121/main
Browse files Browse the repository at this point in the history
repair map mistake
  • Loading branch information
jaelgu authored Jul 7, 2022
2 parents ad0c727 + 0a2740b commit d154130
Showing 1 changed file with 39 additions and 39 deletions.
78 changes: 39 additions & 39 deletions video/video_deduplication/1_video_deduplication_engine.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "2a5baace",
"id": "e06275b8",
"metadata": {},
"source": [
"# How to Build a Video Deduplication System\n",
Expand All @@ -27,7 +27,7 @@
},
{
"cell_type": "markdown",
"id": "69dfd6cf",
"id": "9d953a08",
"metadata": {},
"source": [
"## Preparation\n",
Expand All @@ -48,7 +48,7 @@
{
"cell_type": "code",
"execution_count": 1,
"id": "0042fbfa",
"id": "469282e9",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -57,7 +57,7 @@
},
{
"cell_type": "markdown",
"id": "9738ca63",
"id": "e4f5a71c",
"metadata": {},
"source": [
"### Prepare the data\n",
Expand All @@ -75,7 +75,7 @@
{
"cell_type": "code",
"execution_count": 2,
"id": "5023c1df",
"id": "22543226",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -86,7 +86,7 @@
{
"cell_type": "code",
"execution_count": 3,
"id": "72f01cdf",
"id": "091912bc",
"metadata": {
"scrolled": true
},
Expand Down Expand Up @@ -285,7 +285,7 @@
},
{
"cell_type": "markdown",
"id": "4d042c4c",
"id": "ef4670d6",
"metadata": {},
"source": [
"Define some helper function to convert video to gif so that we can have a look at these videos. "
Expand All @@ -294,7 +294,7 @@
{
"cell_type": "code",
"execution_count": 4,
"id": "99abfd40",
"id": "3ea961d6",
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -334,7 +334,7 @@
},
{
"cell_type": "markdown",
"id": "619cae93",
"id": "97440442",
"metadata": {},
"source": [
"Positive denotes a video that is contain same content event in anchor video, while negative denotes an inconsistent."
Expand All @@ -343,7 +343,7 @@
{
"cell_type": "code",
"execution_count": 5,
"id": "4770d11c",
"id": "88519061",
"metadata": {
"scrolled": true
},
Expand Down Expand Up @@ -389,7 +389,7 @@
},
{
"cell_type": "markdown",
"id": "4999c2ed",
"id": "5c707302",
"metadata": {},
"source": [
"### Create a Milvus Collection\n",
Expand All @@ -400,7 +400,7 @@
{
"cell_type": "code",
"execution_count": 6,
"id": "208ab462",
"id": "df83801e",
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -432,7 +432,7 @@
{
"cell_type": "code",
"execution_count": 7,
"id": "e9c9e6d8",
"id": "ecd908ad",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -441,7 +441,7 @@
},
{
"cell_type": "markdown",
"id": "6d4281ad",
"id": "7acaf86b",
"metadata": {},
"source": [
"## Video Copy Detection\n",
Expand All @@ -453,7 +453,7 @@
},
{
"cell_type": "markdown",
"id": "5eac8eac",
"id": "5fdcd9b8",
"metadata": {},
"source": [
"### Load Video Embeddings into Milvus\n",
Expand All @@ -464,15 +464,15 @@
{
"cell_type": "code",
"execution_count": 8,
"id": "9f36c5df",
"id": "1dc07d2d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 26min 27s, sys: 15.9 s, total: 26min 42s\n",
"Wall time: 1min 56s\n"
"CPU times: user 26min 10s, sys: 14.1 s, total: 26min 24s\n",
"Wall time: 1min 55s\n"
]
}
],
Expand All @@ -496,7 +496,7 @@
},
{
"cell_type": "markdown",
"id": "e6c9fe9a",
"id": "f96e5c4b",
"metadata": {},
"source": [
"Here is detailed explanation for each line of the code:\n",
Expand All @@ -515,7 +515,7 @@
{
"cell_type": "code",
"execution_count": 9,
"id": "5102998a",
"id": "2738a2b3",
"metadata": {
"scrolled": true
},
Expand All @@ -540,7 +540,7 @@
{
"cell_type": "code",
"execution_count": 10,
"id": "441801e0",
"id": "97e3f3f7",
"metadata": {},
"outputs": [
{
Expand All @@ -557,7 +557,7 @@
},
{
"cell_type": "markdown",
"id": "d1697e68",
"id": "4af2a4e3",
"metadata": {},
"source": [
"## Evaluation\n",
Expand All @@ -571,15 +571,15 @@
{
"cell_type": "code",
"execution_count": 11,
"id": "b36f69ca",
"id": "81b8b5ed",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 5min 20s, sys: 2.28 s, total: 5min 23s\n",
"Wall time: 21.9 s\n"
"CPU times: user 4min 16s, sys: 3.15 s, total: 4min 19s\n",
"Wall time: 20.9 s\n"
]
}
],
Expand All @@ -600,7 +600,7 @@
{
"cell_type": "code",
"execution_count": 12,
"id": "eb79042a",
"id": "249ab799",
"metadata": {},
"outputs": [
{
Expand All @@ -623,7 +623,7 @@
{
"cell_type": "code",
"execution_count": 13,
"id": "b68adbdf",
"id": "a026d731",
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -652,16 +652,16 @@
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>recall_at_k</th>\n",
" <th>map_at_k</th>\n",
" <td>0.973977</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" mean_average_precision\n",
"recall_at_k 0.973977"
" mean_average_precision\n",
"map_at_k 0.973977"
]
},
"metadata": {},
Expand All @@ -671,22 +671,22 @@
"source": [
"benchmark = (\n",
" dc.with_metrics(['mean_average_precision',]) \\\n",
" .evaluate['ground_truth_event', 'topk_events'](name='recall_at_k') \\\n",
" .evaluate['ground_truth_event', 'topk_events'](name='map_at_k') \\\n",
" .report()\n",
")"
]
},
{
"cell_type": "markdown",
"id": "51de65b0",
"id": "a2003b8d",
"metadata": {},
"source": [
"We found that we achieved an excellent topk metric on this easy small dataset, which means that if we limit each event to have k duplicate videos, then they can all be almost recalled and they are almost true positive ."
]
},
{
"cell_type": "markdown",
"id": "6373bddd",
"id": "c1b44a40",
"metadata": {},
"source": [
"## Show query results\n",
Expand All @@ -697,7 +697,7 @@
{
"cell_type": "code",
"execution_count": 14,
"id": "8c96f4b1",
"id": "6d461033",
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -735,7 +735,7 @@
{
"cell_type": "code",
"execution_count": 15,
"id": "7cb4c0d7",
"id": "486b42fd",
"metadata": {},
"outputs": [
{
Expand All @@ -759,7 +759,7 @@
{
"cell_type": "code",
"execution_count": 16,
"id": "9ca15405",
"id": "b971fb56",
"metadata": {
"scrolled": true
},
Expand All @@ -785,7 +785,7 @@
{
"cell_type": "code",
"execution_count": 17,
"id": "439afc5d",
"id": "56e0526b",
"metadata": {},
"outputs": [
{
Expand All @@ -809,7 +809,7 @@
{
"cell_type": "code",
"execution_count": 18,
"id": "9aa5f6a1",
"id": "549fb90f",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -820,7 +820,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "433957bf",
"id": "d406323c",
"metadata": {},
"outputs": [],
"source": []
Expand Down

0 comments on commit d154130

Please sign in to comment.