Feature request: Run one instance of WA across multiple devices

[lisatn]

I'm submitting a feature request/enhancement request for WA to run an agenda across multiple devices simultaneously, especially in a board farm setting. IIUC, users can create a config file per device, but there isn't a way to run one instance of WA and have the devices run the workloads in parallel. It'd be nice to say "wa run agenda.yaml -c [list of devices]" or something similar to that in one go.

[ep1cman]

In the past I have run multiple WA instances using a shell script something like:

wa run experiment.yaml -c device1.yaml -d device1_output -f &
wa run experiment.yaml -c device2.yaml -d device2_output -f &
wa run experiment.yaml -c device3.yaml -d device3_output -f &
wait

This will show the output of all 3 wa instances on the screen at the same time, you can check the individual logs if you want to see the output of a specific WA instance

[credp]

Does anyone know how WA would handle multiple instances updating the same sqlite DB? I know you can direct output from multiple runs into the same database file, but I haven't tried with simultaneous writers. If it's OK, you can get a single result file out without post processing outside of WA. The results view of the database is more-or-less exactly what you'd obtain from parsing results.csv, so it's easy to use in tools as well.

The only issue I have using SQL db is that I usually tell the results apart with a combination of workload name, spec id and run_uuid which might be an issue for other users too. I have a script which can combine results from separate runs (for csv files only at the moment) to get a single set of results to use in other tools but you need to be careful to be sure you're combining runs which you know come from the same board with the same software so it's not generally applicable.

[setrofim]

Does anyone know how WA would handle multiple instances updating the same sqlite DB?

The short answer is, it won't. Sqlite does not support simultaneous connections, so if two running WA instances attempt writing results to it at the same time, one with them will fail, probably with something like "could not open file for writing".

Another alternative for collecting multiple run results in one place is MongoDB. That will support multiple parallel uploads and will contain full results, including traces, logs, etc. It's a bit more hassle to set up and query though.

[credp]

Yep, I thought as much. I'm looking to put all our monthly results into a database somewhere in the very near future, directly from the jenkins jobs, hence me working on changing my reporting scripts over to obtaining data with SQL queries. I guess I will need to ask for a MongoDB VM somewhere as well :)

[lisatn]

Thanks Sebastian for sharing your script idea. I'll give it a try and see if it works for me. It's something until a better solution comes along :)