Accuracy of this speedtest code is unknown

[virtuallynathan]

Speedtests can be pretty all over the place, I haven't had a chance to fully evaluate this one, but with a few initial tests it does not appear to be very accurate. I think for maximum accuracy, just exec'ing the official speedtest.net binary would probably be best. It will output json and other formats.

Great work overall on this project, I just set it up.

[virtuallynathan]

Target Server: [35180] 17.14km Seattle, WA (United States) by Ziply Fiber
Latency: 2.483086ms
Download Test: .
Upload Test: .

Download: 733.27 Mbit/s
Upload: 866.26 Mbit/s

vs:

Server: Ziply Fiber - Seattle, WA (id = 35180)
    ISP: Ziply Fiber
Latency:     1.95 ms   (0.16 ms jitter)

Download: 919.35 Mbps (data used: 450.2 MB)
Upload: 934.45 Mbps (data used: 420.9 MB)

[dwillcocks]

Totally agree, every speed test shows different results, but by using the same one yields some kind of consistency. For most users (me included at the point of writing it) it only needed to show tends. For me it was to tend the speed changes as Starlink release software changes to their hardware.

I have not looked deeply into it, but sometimes it feels Docker is constraining the speedtest, but then I'm running it on Mac so Docker is inside a VM aswell.

I did POC using Speedtests.net CLI binary but it started to get messy (cant remember exactly but I ended up with external dependencies) then a bigger final image size, not running on distroless/static, so included vulnerabilities. and needing to accept end user agreements.

I may try again now I have my external Starlink monitoring up and running, I can iterate on the exporters and work on accuracy.

😉 If only Netflix would release more information around fast.com

[virtuallynathan]

That makes sense, I may take a look at writing some code to run the speedtest-cli and see how messy it becomes. I think i'll also poke around with the Go speedtest impl you are using and see if any tweaks make it more accurate.

If only... haha.

[dwillcocks]

If only... haha.

fast_exporter maybe!?!

[CharlesGodwin]

I agree with accuracy problem. I am currently getting negative results. These make it unusable.

curl http://localhost:9092/metrics
# HELP speedtest_download_speed_Bps Last download speedtest result
# TYPE speedtest_download_speed_Bps gauge
speedtest_download_speed_Bps{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} 7.87970036405554e+07
# HELP speedtest_latency_seconds Measured latency on last speed test
# TYPE speedtest_latency_seconds gauge
speedtest_latency_seconds{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} 2.058390779
# HELP speedtest_scrape_duration_seconds Time to preform last speed test
# TYPE speedtest_scrape_duration_seconds gauge
speedtest_scrape_duration_seconds{test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb"} 25.850604899
# HELP speedtest_up Was the last speedtest successful.
# TYPE speedtest_up gauge
speedtest_up{test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb"} 1
# HELP speedtest_upload_speed_Bps Last upload speedtest result
# TYPE speedtest_upload_speed_Bps gauge
speedtest_upload_speed_Bps{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} -2.2512346881061174e+07

[CharlesGodwin]

Is there any way to encourage you to deal with this. I am currently getting persistent negative values for uploads. I'd look inot it myself bu I have no experience with Prometheus or Go.

Thank you

[keyaertc]

Same for me, unfortunately, the results are inaccurate. Any we can do to help ?

[jbartak]

Same for me, constant negative upload speeds