diff --git a/README.md b/README.md
index 35c70529a..26b57d4a2 100644
--- a/README.md
+++ b/README.md
@@ -680,7 +680,7 @@ If you would like to try on your computer:
| | Model | Reference | Exported From | Supported Ailia Version | Blog |
|:-----------|------------:|:------------:|:------------:|:------------:|:------------:|
| [
](optical_flow_estimation/raft/) | [raft](/optical_flow_estimation/raft/) | [RAFT: Recurrent All Pairs Field Transforms for Optical Flow](https://github.com/princeton-vl/RAFT) | Pytorch | 1.2.6 and later | [EN](https://medium.com/axinc-ai/raft-a-machine-learning-model-for-estimating-optical-flow-6ab6d077e178) [JP](https://medium.com/axinc/raft-optical-flow%E3%82%92%E6%8E%A8%E5%AE%9A%E3%81%99%E3%82%8B%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92%E3%83%A2%E3%83%87%E3%83%AB-bf898965de05) |
-
+| [
](optical_flow_estimation/cotracker3/) | [cotracker3](/optical_flow_estimation/cotracker3/) | [ CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos](https://github.com/facebookresearch/co-tracker) | Pytorch | 2.4 and later | |
## Point segmentation
| | Model | Reference | Exported From | Supported Ailia Version | Blog |
diff --git a/optical_flow_estimation/cotracker3/LICENSE.md b/optical_flow_estimation/cotracker3/LICENSE.md
new file mode 100644
index 000000000..e395ca3e2
--- /dev/null
+++ b/optical_flow_estimation/cotracker3/LICENSE.md
@@ -0,0 +1,399 @@
+Attribution-NonCommercial 4.0 International
+
+=======================================================================
+
+Creative Commons Corporation ("Creative Commons") is not a law firm and
+does not provide legal services or legal advice. Distribution of
+Creative Commons public licenses does not create a lawyer-client or
+other relationship. Creative Commons makes its licenses and related
+information available on an "as-is" basis. Creative Commons gives no
+warranties regarding its licenses, any material licensed under their
+terms and conditions, or any related information. Creative Commons
+disclaims all liability for damages resulting from their use to the
+fullest extent possible.
+
+Using Creative Commons Public Licenses
+
+Creative Commons public licenses provide a standard set of terms and
+conditions that creators and other rights holders may use to share
+original works of authorship and other material subject to copyright
+and certain other rights specified in the public license below. The
+following considerations are for informational purposes only, are not
+exhaustive, and do not form part of our licenses.
+
+ Considerations for licensors: Our public licenses are
+ intended for use by those authorized to give the public
+ permission to use material in ways otherwise restricted by
+ copyright and certain other rights. Our licenses are
+ irrevocable. Licensors should read and understand the terms
+ and conditions of the license they choose before applying it.
+ Licensors should also secure all rights necessary before
+ applying our licenses so that the public can reuse the
+ material as expected. Licensors should clearly mark any
+ material not subject to the license. This includes other CC-
+ licensed material, or material used under an exception or
+ limitation to copyright. More considerations for licensors:
+ wiki.creativecommons.org/Considerations_for_licensors
+
+ Considerations for the public: By using one of our public
+ licenses, a licensor grants the public permission to use the
+ licensed material under specified terms and conditions. If
+ the licensor's permission is not necessary for any reason--for
+ example, because of any applicable exception or limitation to
+ copyright--then that use is not regulated by the license. Our
+ licenses grant only permissions under copyright and certain
+ other rights that a licensor has authority to grant. Use of
+ the licensed material may still be restricted for other
+ reasons, including because others have copyright or other
+ rights in the material. A licensor may make special requests,
+ such as asking that all changes be marked or described.
+ Although not required by our licenses, you are encouraged to
+ respect those requests where reasonable. More_considerations
+ for the public:
+ wiki.creativecommons.org/Considerations_for_licensees
+
+=======================================================================
+
+Creative Commons Attribution-NonCommercial 4.0 International Public
+License
+
+By exercising the Licensed Rights (defined below), You accept and agree
+to be bound by the terms and conditions of this Creative Commons
+Attribution-NonCommercial 4.0 International Public License ("Public
+License"). To the extent this Public License may be interpreted as a
+contract, You are granted the Licensed Rights in consideration of Your
+acceptance of these terms and conditions, and the Licensor grants You
+such rights in consideration of benefits the Licensor receives from
+making the Licensed Material available under these terms and
+conditions.
+
+Section 1 -- Definitions.
+
+ a. Adapted Material means material subject to Copyright and Similar
+ Rights that is derived from or based upon the Licensed Material
+ and in which the Licensed Material is translated, altered,
+ arranged, transformed, or otherwise modified in a manner requiring
+ permission under the Copyright and Similar Rights held by the
+ Licensor. For purposes of this Public License, where the Licensed
+ Material is a musical work, performance, or sound recording,
+ Adapted Material is always produced where the Licensed Material is
+ synched in timed relation with a moving image.
+
+ b. Adapter's License means the license You apply to Your Copyright
+ and Similar Rights in Your contributions to Adapted Material in
+ accordance with the terms and conditions of this Public License.
+
+ c. Copyright and Similar Rights means copyright and/or similar rights
+ closely related to copyright including, without limitation,
+ performance, broadcast, sound recording, and Sui Generis Database
+ Rights, without regard to how the rights are labeled or
+ categorized. For purposes of this Public License, the rights
+ specified in Section 2(b)(1)-(2) are not Copyright and Similar
+ Rights.
+ d. Effective Technological Measures means those measures that, in the
+ absence of proper authority, may not be circumvented under laws
+ fulfilling obligations under Article 11 of the WIPO Copyright
+ Treaty adopted on December 20, 1996, and/or similar international
+ agreements.
+
+ e. Exceptions and Limitations means fair use, fair dealing, and/or
+ any other exception or limitation to Copyright and Similar Rights
+ that applies to Your use of the Licensed Material.
+
+ f. Licensed Material means the artistic or literary work, database,
+ or other material to which the Licensor applied this Public
+ License.
+
+ g. Licensed Rights means the rights granted to You subject to the
+ terms and conditions of this Public License, which are limited to
+ all Copyright and Similar Rights that apply to Your use of the
+ Licensed Material and that the Licensor has authority to license.
+
+ h. Licensor means the individual(s) or entity(ies) granting rights
+ under this Public License.
+
+ i. NonCommercial means not primarily intended for or directed towards
+ commercial advantage or monetary compensation. For purposes of
+ this Public License, the exchange of the Licensed Material for
+ other material subject to Copyright and Similar Rights by digital
+ file-sharing or similar means is NonCommercial provided there is
+ no payment of monetary compensation in connection with the
+ exchange.
+
+ j. Share means to provide material to the public by any means or
+ process that requires permission under the Licensed Rights, such
+ as reproduction, public display, public performance, distribution,
+ dissemination, communication, or importation, and to make material
+ available to the public including in ways that members of the
+ public may access the material from a place and at a time
+ individually chosen by them.
+
+ k. Sui Generis Database Rights means rights other than copyright
+ resulting from Directive 96/9/EC of the European Parliament and of
+ the Council of 11 March 1996 on the legal protection of databases,
+ as amended and/or succeeded, as well as other essentially
+ equivalent rights anywhere in the world.
+
+ l. You means the individual or entity exercising the Licensed Rights
+ under this Public License. Your has a corresponding meaning.
+
+Section 2 -- Scope.
+
+ a. License grant.
+
+ 1. Subject to the terms and conditions of this Public License,
+ the Licensor hereby grants You a worldwide, royalty-free,
+ non-sublicensable, non-exclusive, irrevocable license to
+ exercise the Licensed Rights in the Licensed Material to:
+
+ a. reproduce and Share the Licensed Material, in whole or
+ in part, for NonCommercial purposes only; and
+
+ b. produce, reproduce, and Share Adapted Material for
+ NonCommercial purposes only.
+
+ 2. Exceptions and Limitations. For the avoidance of doubt, where
+ Exceptions and Limitations apply to Your use, this Public
+ License does not apply, and You do not need to comply with
+ its terms and conditions.
+
+ 3. Term. The term of this Public License is specified in Section
+ 6(a).
+
+ 4. Media and formats; technical modifications allowed. The
+ Licensor authorizes You to exercise the Licensed Rights in
+ all media and formats whether now known or hereafter created,
+ and to make technical modifications necessary to do so. The
+ Licensor waives and/or agrees not to assert any right or
+ authority to forbid You from making technical modifications
+ necessary to exercise the Licensed Rights, including
+ technical modifications necessary to circumvent Effective
+ Technological Measures. For purposes of this Public License,
+ simply making modifications authorized by this Section 2(a)
+ (4) never produces Adapted Material.
+
+ 5. Downstream recipients.
+
+ a. Offer from the Licensor -- Licensed Material. Every
+ recipient of the Licensed Material automatically
+ receives an offer from the Licensor to exercise the
+ Licensed Rights under the terms and conditions of this
+ Public License.
+
+ b. No downstream restrictions. You may not offer or impose
+ any additional or different terms or conditions on, or
+ apply any Effective Technological Measures to, the
+ Licensed Material if doing so restricts exercise of the
+ Licensed Rights by any recipient of the Licensed
+ Material.
+
+ 6. No endorsement. Nothing in this Public License constitutes or
+ may be construed as permission to assert or imply that You
+ are, or that Your use of the Licensed Material is, connected
+ with, or sponsored, endorsed, or granted official status by,
+ the Licensor or others designated to receive attribution as
+ provided in Section 3(a)(1)(A)(i).
+
+ b. Other rights.
+
+ 1. Moral rights, such as the right of integrity, are not
+ licensed under this Public License, nor are publicity,
+ privacy, and/or other similar personality rights; however, to
+ the extent possible, the Licensor waives and/or agrees not to
+ assert any such rights held by the Licensor to the limited
+ extent necessary to allow You to exercise the Licensed
+ Rights, but not otherwise.
+
+ 2. Patent and trademark rights are not licensed under this
+ Public License.
+
+ 3. To the extent possible, the Licensor waives any right to
+ collect royalties from You for the exercise of the Licensed
+ Rights, whether directly or through a collecting society
+ under any voluntary or waivable statutory or compulsory
+ licensing scheme. In all other cases the Licensor expressly
+ reserves any right to collect such royalties, including when
+ the Licensed Material is used other than for NonCommercial
+ purposes.
+
+Section 3 -- License Conditions.
+
+Your exercise of the Licensed Rights is expressly made subject to the
+following conditions.
+
+ a. Attribution.
+
+ 1. If You Share the Licensed Material (including in modified
+ form), You must:
+
+ a. retain the following if it is supplied by the Licensor
+ with the Licensed Material:
+
+ i. identification of the creator(s) of the Licensed
+ Material and any others designated to receive
+ attribution, in any reasonable manner requested by
+ the Licensor (including by pseudonym if
+ designated);
+
+ ii. a copyright notice;
+
+ iii. a notice that refers to this Public License;
+
+ iv. a notice that refers to the disclaimer of
+ warranties;
+
+ v. a URI or hyperlink to the Licensed Material to the
+ extent reasonably practicable;
+
+ b. indicate if You modified the Licensed Material and
+ retain an indication of any previous modifications; and
+
+ c. indicate the Licensed Material is licensed under this
+ Public License, and include the text of, or the URI or
+ hyperlink to, this Public License.
+
+ 2. You may satisfy the conditions in Section 3(a)(1) in any
+ reasonable manner based on the medium, means, and context in
+ which You Share the Licensed Material. For example, it may be
+ reasonable to satisfy the conditions by providing a URI or
+ hyperlink to a resource that includes the required
+ information.
+
+ 3. If requested by the Licensor, You must remove any of the
+ information required by Section 3(a)(1)(A) to the extent
+ reasonably practicable.
+
+ 4. If You Share Adapted Material You produce, the Adapter's
+ License You apply must not prevent recipients of the Adapted
+ Material from complying with this Public License.
+
+Section 4 -- Sui Generis Database Rights.
+
+Where the Licensed Rights include Sui Generis Database Rights that
+apply to Your use of the Licensed Material:
+
+ a. for the avoidance of doubt, Section 2(a)(1) grants You the right
+ to extract, reuse, reproduce, and Share all or a substantial
+ portion of the contents of the database for NonCommercial purposes
+ only;
+
+ b. if You include all or a substantial portion of the database
+ contents in a database in which You have Sui Generis Database
+ Rights, then the database in which You have Sui Generis Database
+ Rights (but not its individual contents) is Adapted Material; and
+
+ c. You must comply with the conditions in Section 3(a) if You Share
+ all or a substantial portion of the contents of the database.
+
+For the avoidance of doubt, this Section 4 supplements and does not
+replace Your obligations under this Public License where the Licensed
+Rights include other Copyright and Similar Rights.
+
+Section 5 -- Disclaimer of Warranties and Limitation of Liability.
+
+ a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
+ EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
+ AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
+ ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
+ IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
+ WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
+ PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
+ ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
+ KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
+ ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
+
+ b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
+ TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
+ NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
+ INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
+ COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
+ USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
+ ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
+ DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
+ IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
+
+ c. The disclaimer of warranties and limitation of liability provided
+ above shall be interpreted in a manner that, to the extent
+ possible, most closely approximates an absolute disclaimer and
+ waiver of all liability.
+
+Section 6 -- Term and Termination.
+
+ a. This Public License applies for the term of the Copyright and
+ Similar Rights licensed here. However, if You fail to comply with
+ this Public License, then Your rights under this Public License
+ terminate automatically.
+
+ b. Where Your right to use the Licensed Material has terminated under
+ Section 6(a), it reinstates:
+
+ 1. automatically as of the date the violation is cured, provided
+ it is cured within 30 days of Your discovery of the
+ violation; or
+
+ 2. upon express reinstatement by the Licensor.
+
+ For the avoidance of doubt, this Section 6(b) does not affect any
+ right the Licensor may have to seek remedies for Your violations
+ of this Public License.
+
+ c. For the avoidance of doubt, the Licensor may also offer the
+ Licensed Material under separate terms or conditions or stop
+ distributing the Licensed Material at any time; however, doing so
+ will not terminate this Public License.
+
+ d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
+ License.
+
+Section 7 -- Other Terms and Conditions.
+
+ a. The Licensor shall not be bound by any additional or different
+ terms or conditions communicated by You unless expressly agreed.
+
+ b. Any arrangements, understandings, or agreements regarding the
+ Licensed Material not stated herein are separate from and
+ independent of the terms and conditions of this Public License.
+
+Section 8 -- Interpretation.
+
+ a. For the avoidance of doubt, this Public License does not, and
+ shall not be interpreted to, reduce, limit, restrict, or impose
+ conditions on any use of the Licensed Material that could lawfully
+ be made without permission under this Public License.
+
+ b. To the extent possible, if any provision of this Public License is
+ deemed unenforceable, it shall be automatically reformed to the
+ minimum extent necessary to make it enforceable. If the provision
+ cannot be reformed, it shall be severed from this Public License
+ without affecting the enforceability of the remaining terms and
+ conditions.
+
+ c. No term or condition of this Public License will be waived and no
+ failure to comply consented to unless expressly agreed to by the
+ Licensor.
+
+ d. Nothing in this Public License constitutes or may be interpreted
+ as a limitation upon, or waiver of, any privileges and immunities
+ that apply to the Licensor or You, including from the legal
+ processes of any jurisdiction or authority.
+
+=======================================================================
+
+Creative Commons is not a party to its public
+licenses. Notwithstanding, Creative Commons may elect to apply one of
+its public licenses to material it publishes and in those instances
+will be considered the “Licensor.” The text of the Creative Commons
+public licenses is dedicated to the public domain under the CC0 Public
+Domain Dedication. Except for the limited purpose of indicating that
+material is shared under a Creative Commons public license or as
+otherwise permitted by the Creative Commons policies published at
+creativecommons.org/policies, Creative Commons does not authorize the
+use of the trademark "Creative Commons" or any other trademark or logo
+of Creative Commons without its prior written consent including,
+without limitation, in connection with any unauthorized modifications
+to any of its public licenses or any other arrangements,
+understandings, or agreements concerning use of licensed material. For
+the avoidance of doubt, this paragraph does not form part of the
+public licenses.
+
+Creative Commons may be contacted at creativecommons.org.
\ No newline at end of file
diff --git a/optical_flow_estimation/cotracker3/README.md b/optical_flow_estimation/cotracker3/README.md
new file mode 100644
index 000000000..ab6da444d
--- /dev/null
+++ b/optical_flow_estimation/cotracker3/README.md
@@ -0,0 +1,51 @@
+# CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos
+
+## Input
+
+
+
+(Image from https://github.com/facebookresearch/co-tracker/blob/main/gradio_demo/videos/bear.mp4)
+
+Shape : (1, 3, 854, 480)
+
+## Output
+
+
+
+
+### usage
+Automatically downloads the onnx and prototxt files on the first run.
+It is necessary to be connected to the Internet while downloading.
+
+For the sample video,
+``` bash
+$ python3 cotracker3.py
+```
+
+If you want to specify the input video, put the video path after the `--input` option.
+You can use `--savepath` option to change the name of the output file to save.
+
+```bash
+$ python3 cotracker3.py --input IMAGE_PATH --savepath SAVE_IMAGE_PATH
+```
+
+By default, the ailia SDK is used. If you want to use ONNX Runtime, use the --onnx option.
+```bash
+$ python3 cotracker3.py --onnx
+```
+
+## Reference
+
+- [CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos](https://github.com/facebookresearch/co-tracker)
+
+## Framework
+
+Pytorch 2.4
+
+## Model Format
+
+ONNX opset=20
+
+## Netron
+
+[cotracker3.onnx.prototxt](https://netron.app/?url=https://storage.googleapis.com/ailia-models/cotracker3/cotracker3.onnx.prototxt)
diff --git a/optical_flow_estimation/cotracker3/cotracker3.py b/optical_flow_estimation/cotracker3/cotracker3.py
new file mode 100644
index 000000000..fc9552594
--- /dev/null
+++ b/optical_flow_estimation/cotracker3/cotracker3.py
@@ -0,0 +1,147 @@
+import sys
+import cv2
+import time
+import numpy as np
+
+import ailia
+import onnxruntime as ort
+from vis import Visualizer
+
+# import original modules
+sys.path.append('../../util')
+from arg_utils import get_base_parser, update_parser # noqa: E402
+from model_utils import check_and_download_models # noqa: E402
+
+# logger
+from logging import getLogger # noqa: E402
+logger = getLogger(__name__)
+
+
+# ======================
+# Parameters
+# ======================
+VIDEO_PATH = 'input.mp4'
+SAVE_PATH = 'output.mp4'
+
+# ======================
+# Argument Parser Config
+# ======================
+parser = get_base_parser(
+ 'CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos',
+ VIDEO_PATH,
+ SAVE_PATH,
+)
+
+parser.add_argument("--grid_size", type=int, default=10, help="Regular grid size")
+parser.add_argument(
+ "--grid_query_frame",
+ type=int,
+ default=0,
+ help="Compute dense and grid tracks starting from this frame",
+)
+parser.add_argument(
+ "--backward_tracking",
+ action="store_true",
+ help="Compute tracks in both directions, not only forward",
+)
+
+parser.add_argument('--onnx', action='store_true', help='execute onnxruntime version.')
+
+args = update_parser(parser)
+
+# ==========================
+# MODEL AND OTHER PARAMETERS
+# ==========================
+WEIGHT_PATH = 'cotracker3.onnx'
+MODEL_PATH = 'cotracker3.onnx.prototxt'
+REMOTE_PATH = 'https://storage.googleapis.com/ailia-models/cotracker3/'
+
+def read_video_from_path(path):
+ try:
+ cap = cv2.VideoCapture(path)
+ except Exception as e:
+ print("Error opening video file: ", e)
+ return None
+ frames = []
+
+ while cap.isOpened():
+ ret, frame = cap.read()
+ if not ret:
+ break
+ frames.append(frame)
+ cap.release()
+
+ return np.stack(frames)
+
+
+def compute(net,video):
+ if not args.onnx:
+ result = net.run((video,np.array(args.grid_size ,dtype=np.int64),
+ np.array(args.grid_query_frame,dtype=np.int64)))
+ else:
+ input_name1 = net.get_inputs()[0].name
+ input_name2 = net.get_inputs()[1].name
+ input_name3 = net.get_inputs()[2].name
+ result= net.run([],{input_name1:video,
+ input_name2:np.array(args.grid_size ,dtype=np.int64),
+ input_name3:np.array(args.grid_query_frame,dtype=np.int64)})
+ return result
+
+# ======================
+# Main functions
+# ======================
+def recognize_from_video():
+ # net initialize
+ if not args.onnx:
+ memory_mode = ailia.get_memory_mode(
+ reduce_constant=True, ignore_input_with_initializer=True,
+ reduce_interstage=False, reuse_interstage=True)
+
+ net = ailia.Net(MODEL_PATH, WEIGHT_PATH, env_id=args.env_id,memory_mode=memory_mode)
+ else:
+ net = ort.InferenceSession(WEIGHT_PATH)
+
+ # load video
+ vis = Visualizer( pad_value=120, linewidth=3)
+
+ for path in args.input:
+ video = read_video_from_path(path)
+ np.transpose(video,(0, 3, 1, 2))
+ video = np.transpose(video,(0, 3, 1, 2))[np.newaxis, ...].astype(np.float32)
+
+
+ # calculate feature map
+ logger.info('Start calculating feature map...')
+ if args.benchmark:
+ logger.info('BENCHMARK mode')
+ for i in range(args.benchmark_count):
+ start = int(round(time.time() * 1000))
+ result = compute(net,video)
+ end = int(round(time.time() * 1000))
+ logger.info(f'\tailia processing time {end - start} ms')
+ else:
+ result = compute(net,video)
+
+ pred_tracks = np.array(result[0])
+ pred_visibility = np.array(result[1])
+
+ # save a video with predicted tracks
+ logger.info(f'saved at : {args.savepath}')
+ vis.visualize(
+ video,
+ pred_tracks,
+ pred_visibility,
+ args.savepath
+ )
+ logger.info('Script finished successfully.')
+
+
+def main():
+ # model files check and download
+ check_and_download_models(WEIGHT_PATH, MODEL_PATH, REMOTE_PATH)
+
+ recognize_from_video()
+
+
+if __name__ == '__main__':
+ main()
diff --git a/optical_flow_estimation/cotracker3/input.gif b/optical_flow_estimation/cotracker3/input.gif
new file mode 100644
index 000000000..2653a71cf
Binary files /dev/null and b/optical_flow_estimation/cotracker3/input.gif differ
diff --git a/optical_flow_estimation/cotracker3/input.mp4 b/optical_flow_estimation/cotracker3/input.mp4
new file mode 100644
index 000000000..451ad7a12
Binary files /dev/null and b/optical_flow_estimation/cotracker3/input.mp4 differ
diff --git a/optical_flow_estimation/cotracker3/output.gif b/optical_flow_estimation/cotracker3/output.gif
new file mode 100644
index 000000000..817795b10
Binary files /dev/null and b/optical_flow_estimation/cotracker3/output.gif differ
diff --git a/optical_flow_estimation/cotracker3/vis.py b/optical_flow_estimation/cotracker3/vis.py
new file mode 100644
index 000000000..116e905ac
--- /dev/null
+++ b/optical_flow_estimation/cotracker3/vis.py
@@ -0,0 +1,214 @@
+import os
+import cv2
+import numpy as np
+
+
+from matplotlib import cm
+import matplotlib.pyplot as plt
+
+def read_video_from_path(path):
+ try:
+ cap = cv2.VideoCapture(path)
+ except Exception as e:
+ print("Error opening video file: ", e)
+ return None
+ frames = []
+
+ while cap.isOpened():
+ ret, frame = cap.read()
+ if not ret:
+ break
+ frames.append(frame) # フレームをリストに追加
+ cap.release()
+
+ return np.stack(frames)
+
+
+class Visualizer:
+ def __init__(
+ self,
+ pad_value: int = 0,
+ linewidth: int = 2,
+ show_first_frame: int = 10,
+ tracks_leave_trace: int = 0, # -1 for infinite
+ ):
+
+ self.color_map = cm.get_cmap("gist_rainbow")
+
+ self.show_first_frame = show_first_frame
+ self.tracks_leave_trace = tracks_leave_trace
+ self.pad_value = pad_value
+ self.linewidth = linewidth
+
+ def visualize(
+ self,
+ video,
+ tracks,
+ visibility=None,
+
+ filename: str = "video",
+
+ query_frame=0,
+ opacity: float = 1.0,
+ ):
+
+ video = pad(video,self.pad_value,255)
+
+ color_alpha = int(opacity * 255)
+ tracks = tracks + self.pad_value
+
+ res_video = self.draw_tracks_on_video(
+ video=video,
+ tracks=tracks,
+ visibility=visibility,
+ query_frame=query_frame,
+ color_alpha=color_alpha,
+ )
+
+ self.save_video(res_video, filename=filename)
+ return res_video
+
+ def save_video(self, video, filename):
+
+
+ wide_list = [video[:,i,:,:,:] for i in range(video.shape[1])]
+
+ wide_list = [np.transpose(wide[0],(1, 2, 0)) for wide in wide_list]
+
+ # Prepare the video file path
+ save_path = filename
+
+ # Create a writer object
+ height, width, channels = wide_list[0].shape
+ fourcc = cv2.VideoWriter_fourcc(*'mp4v') # MP4コーデック
+ out = cv2.VideoWriter(save_path, fourcc, 30, (width, height))
+
+ # Write frames to the video file
+ for frame in wide_list[2:-1]:
+ out.write(frame)
+ out.release()
+
+ print(f"Video saved to {save_path}")
+
+ def draw_tracks_on_video(
+ self,
+ video,
+ tracks,
+ visibility = None,
+ query_frame=0,
+ color_alpha: int = 255,
+ ):
+
+
+ B, T, C, H, W = video.shape
+ _, _, N, D = tracks.shape
+
+ segm_mask = None
+
+ assert D == 2
+ assert C == 3
+
+ video = np.transpose(video[0],(0, 2, 3, 1)).astype(np.uint8) # S, H, W, C
+ tracks = tracks[0].astype(np.int64)
+
+
+ res_video = []
+
+ # process input video
+ for rgb in video:
+ res_video.append(rgb.copy())
+ vector_colors = np.zeros((T, N, 3))
+
+ #if segm_mask is None:
+ y_min, y_max = (
+ tracks[query_frame, :, 1].min(),
+ tracks[query_frame, :, 1].max(),
+ )
+ norm = plt.Normalize(y_min, y_max)
+ for n in range(N):
+ query_frame_ = query_frame
+
+ color = self.color_map(norm(tracks[query_frame_, n, 1]))
+ color = np.array(color[:3])[None] * 255
+ vector_colors[:, n] = np.repeat(color, T, axis=0)
+
+ # draw tracks
+ if self.tracks_leave_trace != 0:
+ for t in range(query_frame + 1, T):
+ first_ind = (
+ max(0, t - self.tracks_leave_trace)
+ if self.tracks_leave_trace >= 0
+ else 0
+ )
+ curr_tracks = tracks[first_ind : t + 1]
+ curr_colors = vector_colors[first_ind : t + 1]
+
+ res_video[t] = self._draw_pred_tracks(
+ res_video[t],
+ curr_tracks,
+ curr_colors,
+ )
+
+ # draw points
+ for t in range(T):
+ img = np.uint8(res_video[t])
+ for i in range(N):
+ coord = (tracks[t, i, 0], tracks[t, i, 1])
+ visibile = True
+ if visibility is not None:
+ visibile = visibility[0, t, i]
+ if coord[0] != 0 and coord[1] != 0:
+ img = draw_circle(
+ img,
+ coord=coord,
+ radius=int(self.linewidth * 2),
+ color=vector_colors[t, i].astype(int),
+ visible=visibile,
+ color_alpha=color_alpha,
+ )
+ res_video[t] = np.array(img)
+
+ # construct the final rgb sequence
+ if self.show_first_frame > 0:
+ res_video = [res_video[0]] * self.show_first_frame + res_video[1:]
+ return np.transpose(np.stack(res_video),(0, 3, 1, 2))[np.newaxis, ...].astype(np.uint8)
+
+def draw_ellipse(image, left_up_point, right_down_point, color, visible=True):
+ center = (
+ (left_up_point[0] + right_down_point[0]) // 2,
+ (left_up_point[1] + right_down_point[1]) // 2,
+ )
+ axes = (
+ abs(right_down_point[0] - left_up_point[0]) // 2,
+ abs(right_down_point[1] - left_up_point[1]) // 2,
+ )
+ thickness = -1 if visible else 2
+ color = tuple(map(int, color))
+ cv2.ellipse(image, center, axes, 0, 0, 360, color, thickness)
+ return image
+
+def draw_circle(rgb, coord, radius, color=(255, 0, 0), visible=True, color_alpha=None):
+ # Create a draw object
+ # Calculate the bounding box of the circle
+ left_up_point = (coord[0] - radius, coord[1] - radius)
+ right_down_point = (coord[0] + radius, coord[1] + radius)
+ # Draw the circle
+ color = tuple(list(color) + [color_alpha if color_alpha is not None else 255])
+
+ rgb = draw_ellipse(rgb,left_up_point,right_down_point,color,True)
+
+ return rgb
+
+
+
+def pad(video, pad_value, constant_value=255):
+ padding = ((0, 0),
+ (0, 0),
+ (0, 0),
+ (pad_value, pad_value),
+ (pad_value, pad_value))
+
+ padded_video = np.pad(video, pad_width=padding, mode='constant', constant_values=constant_value)
+ return padded_video
+
+
diff --git a/scripts/download_all_models.sh b/scripts/download_all_models.sh
index 05a5a8110..4493db60d 100755
--- a/scripts/download_all_models.sh
+++ b/scripts/download_all_models.sh
@@ -314,6 +314,7 @@ cd ../../object_tracking/qd-3dt; python3 qd-3dt.py ${OPTION}
cd ../../object_tracking/strong_sort; python3 strong_sort.py ${OPTION}
cd ../../object_tracking/deepsort_vehicle; python3 deepsort_vehicle.py ${OPTION}
cd ../../optical_flow_estimation/raft; python3 raft.py ${OPTION}
+cd ../../optical_flow_estimation/cotracker3; python3 cotracker3.py ${OPTION}
cd ../../point_segmentation/pointnet_pytorch; python3 pointnet_pytorch.py ${OPTION}
cd ../../pose_estimation/lightweight-human-pose-estimation; python3 lightweight-human-pose-estimation.py ${OPTION}
cd ../../pose_estimation/openpose; python3 openpose.py ${OPTION}