-
Notifications
You must be signed in to change notification settings - Fork 0
/
humanteleop.html
243 lines (231 loc) · 18.9 KB
/
humanteleop.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
<!DOCTYPE html>
<html>
<head>
<title>David Black Portfolio</title>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<link rel="stylesheet" href="w3.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Lato">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
<style>
body,h1,h2,h3,h4,h5,h6 {font-family: "Lato", sans-serif;}
body, html {
height: 100%;
color: #777;
line-height: 1.8;
}
/* Create a Parallax Effect */
.bgimg-1 {
background-attachment: fixed;
background-position: center;
background-repeat: no-repeat;
background-size: cover;
}
/* First image (Logo. Full height) */
.bgimg-1 {
background-image: url('images/work/humanTeleop.jpg');
min-height: 100%;
}
.w3-wide {letter-spacing: 10px;}
.w3-hover-opacity {cursor: pointer;}
/* Turn off parallax scrolling for tablets and phones */
@media only screen and (max-device-width: 1600px) {
.bgimg-1 {
background-attachment: scroll;
min-height: 400px;
}
}
</style>
</head>
<body>
<!-- Navbar (sit on top) -->
<!-- <div class="w3-top">-->
<!-- <div class="w3-bar" id="myNavbar">-->
<!-- <a class="w3-bar-item w3-button w3-hover-black w3-hide-medium w3-hide-large w3-right" href="javascript:void(0);" onclick="toggleFunction()" title="Toggle Navigation Menu">-->
<!-- <i class="fa fa-bars"></i>-->
<!-- </a>-->
<!-- <a href="index.html" class="w3-bar-item w3-button">HOME</a>-->
<!-- <a href="index.html#about" class="w3-bar-item w3-button w3-hide-small"><i class="fa fa-user"></i> ABOUT</a>-->
<!-- <a href="index.html#cv" class="w3-bar-item w3-button w3-hide-small"><i class="fa fa-th"></i> CV</a>-->
<!-- <a href="adventure.html" class="w3-bar-item w3-button w3-hide-small"><i class="fa fa-map-o"></i> ADVENTURE</a>-->
<!-- <a href="index.html#contact" class="w3-bar-item w3-button w3-hide-small"><i class="fa fa-envelope"></i> CONTACT</a>-->
<!-- <a href="#" class="w3-bar-item w3-button w3-hide-small w3-right w3-hover-red">-->
<!-- <i class="fa fa-search"></i>-->
<!-- </a>-->
<!-- </div>-->
<!-- <!– Navbar on small screens –>-->
<!-- <div id="navDemo" class="w3-bar-block w3-white w3-hide w3-hide-large w3-hide-medium">-->
<!-- <a href="index.html#about" class="w3-bar-item w3-button" onclick="toggleFunction()">ABOUT</a>-->
<!-- <a href="index.html#cv" class="w3-bar-item w3-button" onclick="toggleFunction()">CV</a>-->
<!-- <a href="adventure.html" class="w3-bar-item w3-button" onclick="toggleFunction()">ADVENTURE</a>-->
<!-- <a href="index.html#contact" class="w3-bar-item w3-button" onclick="toggleFunction()">CONTACT</a>-->
<!-- <a href="#" class="w3-bar-item w3-button">SEARCH</a>-->
<!-- </div>-->
<!-- </div>-->
<!-- First Parallax Image with Logo Text -->
<!-- <div class="bgimg-1 w3-display-container w3-opacity-min" id="home">-->
<!-- <div class="w3-display-middle" style="white-space:nowrap;">-->
<!-- <span class="w3-center w3-padding-large w3-black w3-xlarge w3-wide w3-animate-opacity">HUMAN <span class="w3-hide-small"></span> TELEOPERATION</span>-->
<!-- </div>-->
<!-- </div>-->
<!-- Container (Wiki Section) -->
<div class="w3-content w3-container w3-padding-64" id="contents">
<h2 id="human-teleop-wiki">Human Teleoperation Resources</h2>
<h5 id="version-date">Updated: October 29, 2024</h5>
<p>Human teleoperation is a framework for tightly coupled guidance of a novice follower person by an expert sonographer through a remote mixed reality (MR) system. By combining high-speed communication, MR, and haptics we can achieve intuitive, fast, precise teleoperation of an ultrasound procedure with almost robot-like performance but human flexibility and intelligence. This can enable greatly improved access to healthcare in small or remote communities.</p>
<p>This project has been featured by numerous news outlets, including: <a href="https://www.cbc.ca/player/play/video/9.6573580">CBC News</a>, <a href="https://globalnews.ca/news/10949371/bc-researchers-new-technology-remote-ultrasounds/">Global TV</a>, <a href="https://youtu.be/0LxiADm_ka4?si=iKS7GRrWhVgW9qmq">CTV News</a>, <a href="https://www.cbc.ca/listen/live-radio/1-110-daybreak-south/clip/16112225-theme-song-mining-songs-west-kootenay-drug-alert">CBC Radio 1</a> (starts around 30:45), <a href="https://thetyee.ca/News/2024/11/26/UBC-Student-Awarded-Remote-Ultrasound/">The Tyee</a>, <a href="https://www.mitacs.ca/news/mitacs-innovation-awards-celebrate-canadas-top-talent/">Mitacs</a>, <a href="https://www.healthing.ca/wellness/technology/ultrasound-anywhere-virtual-reality-tool-aims-to-change-the-face-of-rural-healthcare-for-canadians">Healthing.ca</a>, <a href="https://techcouver.com/2024/11/20/ubc-researcher-revolutionizes-remote-ultrasound-technology/">Techouver</a>, <a href="https://www.canhealth.com/2024/11/20/bc-researcher-wins-award-for-remote-ultrasound/">Canadian Healthcare Technology</a>, <a href="https://canadiansme.ca/prestigious-mitacs-innovation-awards-celebrate-canadas-top-talent-for-breakthroughs-in-health-food-industry-water-treatment-and-more/">CanadianSME</a>, <a href="https://www.bctechnology.com/news/2024/11/19/B.C.-Researcher-Wins-Prestigious-Mitacs-Innovation-Award-for-Remote-Ultrasound-Breakthrough.cfm">T-Net BC</a>, as well as <a href="https://research.ubc.ca/two-ubc-researchers-recognized-mitacs-outstanding-innovation-and-inclusive-innovator-year-awards">UBC Research</a> and <a href="https://apsc.ubc.ca/news/2024/ubc-engineering-researchers-david-black-and-dr-paul-onkundi-nyangaresi-receive-2024">UBC Applied Science</a>, <a href="https://www.vchri.ca/stories/2024/08/02/teleoperated-ultrasounds-bring-expert-care-rural-and-remote-communities">Vancouver Coastal Health</a>, the <a href="https://ece.ubc.ca/ece-student-perspectives-david-black-bmiai-fall-research-award-winner/?fbclid=IwAR1Xr2VrHbqTaCD4BnqhBOcMMts9JjDCCYInPcltPPRWH-0Gymz0uS8CPAQ">UBC ECE Department</a>, and the <a href="https://icics.ubc.ca/2022/07/05/ubc-rogers-collaborative-research-project-on-tele-ultrasound-wins-prestigious-best-paper-award/">UBC Institute for Computing, Information, and Cognitive Science</a>.</p>
<div class="w3-center">
<iframe src="//www.cbc.ca/i/phoenix/player/syndicate/?sourceId=9.6573580" width="640" height="360" frameborder="0" allowfullscreen></iframe>
</div>
<p>This page contains some of our work so far, listed below. For an overview of the system and a test deployment to Skidegate in Haida Gwaii, see the videos below.</p>
<p>A PowerPoint slide deck about the system is also available <a href="https://docs.google.com/presentation/d/1-ZvlZymLLH6Vg5rEtdbMZamNLHdzBbFd/edit?usp=drive_link&ouid=118015589757826872772&rtpof=true&sd=true">here</a>.</p>
<h5 id="contact">For more info, questions, or potential collaboration, please contact me at:</h5>
<i class="fa fa-linkedin fa-fw w3-hover-text-black w3-xlarge w3-margin-right"></i> <a href="https://www.linkedin.com/in/davidgblack/">LinkedIn</a> <br>
<i class="fa fa-google fa-fw w3-hover-text-black w3-xlarge w3-margin-right"></i> <a href="https://scholar.google.com/citations?user=As_yDKkAAAAJ&hl=en">Google Scholar</a> <br>
<i class="fa fa-envelope fa-fw w3-hover-text-black w3-xlarge w3-margin-right"></i> <a href="mailto:[email protected]">Email Me</a><br>
<h4 id="video">Video Overview from Mitacs Innovation Awards</h4>
<iframe width="560" height="315" src="https://www.youtube.com/embed/pDafyuoCzkY?si=aWE8gyaGXVEV-Jfc" title="Mitacs Innovation Award" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
<h4 id="demo">(slightly outdated) Demo and System Overview</h4>
<iframe width="560" height="315" src="https://www.youtube.com/embed/tbzmODsuRJ4?si=HPjMmxjs2zQNSxbR" title="Human Teleoperation Demo" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
<h4 id="haidagwaii">Haida Gwaii Tests</h4>
<iframe width="560" height="315" src="https://www.youtube.com/embed/Q85ykfIfcng?si=izUquIco_n5ww6-a" title="Haida Gwaii Deployment" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
<h4 id="haidagwaii">Global TV News Segment</h4>
<iframe width="560" height="315" src="https://globalnews.ca/video/embed/10949674/" title="Global TV Segment" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
<h4 id="patents">Patents</h4>
<ul>
<li><p>D. Black, S. Salcudean. <strong>Differential Magnetic Multi-axis Force Sensing,</strong> US Patent Office Provisional Patent. May 2023.</p></li>
<li><p>D. Black, Y. Oloumi, A.H. Hadi Hosseinabadi, S. Salcudean. <strong>System and Method for Low-Cost Remote Guidance of Procedures</strong>, US Patent Office Patent Application. March 2023.</p>
<ul>
<li><a href="https://patents.google.com/patent/US20230218270A1/en">US20230218270A1</a></li>
</ul>
</li>
<li><p>D. Black, Y. Oloumi, A.H. Hadi Hosseinabadi, S. Salcudean. <strong>System and Apparatus for Remote Interaction with an Object</strong>, US Patent Office Patent Application. July 2021.</p>
<ul>
<li><a href="https://patents.google.com/patent/US20240173018A1/en">US20240173018A1</a></li>
</ul>
</li>
</ul>
<h4 id="papers">Papers</h4>
<ul>
<li>
<p>D. Black, S. Salcudean. <strong>Stability and Transparency in Mixed Reality Bilateral Teleoperation</strong> submitted to IEEE Transactions on Robotics, October, 2024.</p>
<ul>
<li><a href="https://arxiv.org/pdf/2410.09679">PDF</a></li>
<li><a href="https://arxiv.org/abs/2410.09679">DOI: 10.48550/arXiv.2410.09679</a></li>
<p>Paper which models and analyzes the bilateral teleoperation system mathematically, following robotics and control literature and testing a variety of controllers for performance and robustness to time delays.</p>
</ul>
</li>
<li>
<p>R. Yeung, D. Black, P. Chen, V. Lessoway, J. Reid, S. Rangel-Suarez, S. Chang, S. Salcudean, <strong>Mixed Reality Tele-ultrasound over 750 km: a Clinical Study</strong>, submitted to IEEE Transactions on Biomedical Engineering</p>
<ul>
<li><a href="https://arxiv.org/pdf/2409.13058">PDF</a></li>
<li><a href="https://arxiv.org/abs/2409.13058">DOI: 10.48550/arXiv.2409.13058</a></li>
<p>The paper we have submitted about our tests in Skidegate in the Haida Gwaii islands, in which the expert was located over 750km away at our lab at UBC.</p>
</ul>
</li>
<li> <p>D. Black, S. Salcudean. <strong>Robust Object Pose Tracking for Augmented Reality Guidance and Teleoperation</strong>, in IEEE Transactions on Instrumentation and Measurement, May 2024.</p>
<ul>
<li><a href="/files/IEEE_TIM_2024.pdf">PDF</a></li>
<li><a href="https://doi.org/10.1109/TIM.2024.3398108">DOI: 10.1109/TIM.2024.3398108</a></li>
<li>Paper about HoloLens 2-based pose tracking of the ultrasound probe.</li>
</ul>
</li>
<li><p>D. Black, A.H. Hadi Hosseinabadi, M. Nogami, N. Rangga, S. Salcudean. <strong>Low-Profile 6-Axis Differential Magnetic Force/Torque Sensing</strong>, in IEEE Transactions on Medical Robotics and Bionics, May 2024.</p>
<ul>
<li><a href="/files/IEEE_TMRB_2024_FT.pdf">PDF</a></li>
<li><a href="https://doi.org/10.1109/TMRB.2024.3407350">DOI: 10.1109/TMRB.2024.3407350</a></li>
<li>Paper about our force sensor concept, optimal design, integration into an ultrasound probe, and neural network calibration.</li>
</ul>
</li>
<li><p>D. Black, D. Andjelic, S.E Salcudean. <strong>Evaluation of Communication and Human Response Latency for (Human) Teleoperation</strong>, in IEEE Transactions on Medical Robotics. January, 2024.</p>
<ul>
<li><a href="/files/IEEE_TMRB_2024.pdf">PDF</a></li>
<li><a href="https://doi.org/10.1109/TMRB.2024.3349612">DOI: 10.1109/TMRB.2024.3349612</a></li>
<li>A paper about the communication system and human response time.</li>
</ul>
</li>
<li> <p>D. Black, M. Nogami, S. Salcudean. <strong>Mixed Reality Human Teleoperation with Device-Agnostic Remote Ultrasound: Communication and HCI</strong>, in Computers and Graphics, January, 2024</p>
<ul>
<li><a href="/files/CAG_2024.pdf">PDF</a></li>
<li><a href="https://doi.org/10.1016/j.cag.2024.01.003">DOI: 10.1016/j.cag.2024.01.003</a></li>
<li>Paper in Computers and Graphics, which is an extended version of IEEE_VR_2023 with several added components, including device agnostic ultrasound probe streaming and control, and several HCI improvements enabled by the pose tracking (below).</li>
</ul>
</li>
<li> <p>D. Black, A.H. Hadi Hosseinabadi, M. Nogami, M. Pol, N. Rangga, S. Salcudean. <strong>Towards Differential Magnetic Force Sensing for Ultrasound Teleoperation</strong>, in IEEE World Haptics Conference, July 10-13, 2023, Delft, NL.</p>
<ul>
<li><a href="/files/WHC_2023.pdf">PDF</a></li>
<li><a href="https://ieeexplore.ieee.org/abstract/document/10224473">DOI: 10.1109/WHC56415.2023.10224473</a></li>
<li>Paper that was presented at the 2023 IEEE World Haptics Conference on our magnetic force sensor.</li>
</ul>
</li>
<li>
<p>D. Black, Y. Oloumi, A.H. Hadi Hosseinabadi, S. Salcudean. <strong>Human Teleoperation - a Haptically-Enabled Mixed Reality System for Teleultrasound</strong>, Human Computer Interaction. June, 2023 (submitted July 2021).</p>
<ul>
<li><a href="/files/HCI_2023.pdf">PDF</a></li>
<li><a href="https://doi.org/10.1080/07370024.2023.2218355">DOI: 10.1080/07370024.2023.2218355</a></li>
</ul>
</li>
<li><p>D. Black, H. Moradi, S.E Salcudean. <strong>Human-as-a-Robot Performance in Augmented Reality Teleultrasound</strong>, in International Journal of Computer Assisted Radiology and Surgery. April, 2023.</p>
<ul>
<li><a href="/files/IJCARS_2023.pdf">PDF</a></li>
<li><a href="https://doi.org/10.1007/s11548-023-02896-0">DOI: 10.1007/s11548-023-02896-0</a></li>
<li>A paper describing tests of human tracking ability and performance in mixed reality human teleoperation.</li>
</ul>
</li>
<li> <p>D. Black, S. Salcudean. <strong>Mixed Reality Human Teleoperation</strong>, in IEEE VR, March 25-29, 2023, Shanghai, China. Workshop Presentation & Paper.</p>
<ul>
<li><a href="/files/IEEE_VR_2023.pdf">PDF</a></li>
<li><a href="https://ieeexplore.ieee.org/abstract/document/10108864">DOI: 10.1109/VRW58643.2023.00083</a></li>
<li>A paper that was presented at the 2023 IEEE Conference on Virtual Reality, Extended Reality for Industrial and Occupational Support Workshop. This provides a good overview of the system.</li>
</ul>
</li>
<li><p>D. Black, S. Salcudean. <strong>A Mixed Reality System for Human Teleoperation in Tele-Ultrasound</strong>, in the Hamlyn Symposium for Medical Robotics, June 26-29, 2022, London, UK. Podium Presentation.</p>
<ul>
<li><a href="/files/HSMR_2022.pdf">PDF</a></li>
<li>A short paper showing some system upgrades, presented at the 2022 Hamlyn Symposium for Medical Robotics, Imperial College London, where we won Best Podium Presentation.</li>
</ul>
</li>
</ul>
</div>
<!-- Footer -->
<footer class="w3-center w3-black w3-padding-64 w3-opacity w3-hover-opacity-off">
<a href="#home" class="w3-button w3-light-grey"><i class="fa fa-arrow-up w3-margin-right"></i>To the top</a>
<div class="w3-xlarge w3-section">
<a href="https://github.com/dgblack"><i class="fa fa-github w3-hover-opacity"></i></a>
<a href="https://scholar.google.com/citations?user=As_yDKkAAAAJ&hl=en"><i class="fa fa-google w3-hover-opacity"></i></a>
<a href="https://twitter.com/BlackDavidG"><i class="fa fa-twitter w3-hover-opacity"></i></a>
<a href="https://linkedin.com/in/davidgblack"><i class="fa fa-linkedin w3-hover-opacity"></i></a>
</div>
<p>Powered by <a href="https://www.w3schools.com/w3css/default.asp" title="W3.CSS" target="_blank" class="w3-hover-text-green">w3.css</a></p>
</footer>
<script>
// Modal Image Gallery
function onClick(element) {
document.getElementById("img01").src = element.src;
document.getElementById("modal01").style.display = "block";
var captionText = document.getElementById("caption");
captionText.innerHTML = element.alt;
}
// Change style of navbar on scroll
window.onscroll = function () { myFunction() };
function myFunction() {
var navbar = document.getElementById("myNavbar");
if (document.body.scrollTop > 100 || document.documentElement.scrollTop > 100) {
navbar.className = "w3-bar" + " w3-card" + " w3-animate-top" + " w3-white";
} else {
navbar.className = navbar.className.replace(" w3-card w3-animate-top w3-white", "");
}
}
// Used to toggle the menu on small screens when clicking on the menu button
function toggleFunction() {
var x = document.getElementById("navDemo");
if (x.className.indexOf("w3-show") == -1) {
x.className += " w3-show";
} else {
x.className = x.className.replace(" w3-show", "");
}
}
</script>
</body>
</html>