Worker tests: use unique pointers when possible #4695

Job	Run time
ci (ubuntu-20.04, gcc, g++, 20, Release)	4m 27s
ci (ubuntu-20.04, gcc, g++, 20, Debug)	4m 21s
ci (ubuntu-20.04, clang, clang++, 20, Release)	4m 28s
ci (ubuntu-20.04, clang, clang++, 20, Debug)	4m 13s
ci (ubuntu-22.04, gcc, g++, 20, Release)	4m 27s
ci (ubuntu-22.04, gcc, g++, 20, Debug)	4m 28s
ci (ubuntu-22.04, clang, clang++, 20, Release)	4m 27s
ci (ubuntu-22.04, clang, clang++, 20, Debug)	4m 26s
ci (macos-12, gcc, g++, 20, Release)	4m 30s
ci (macos-12, gcc, g++, 20, Debug)	4m 28s
ci (macos-14, clang, clang++, 20, Release)	4m 27s
ci (macos-14, clang, clang++, 20, Debug)	4m 11s
ci (windows-2022, cl, cl, 20, Release)	4m 12s
ci (windows-2022, cl, cl, 20, Debug)	4m 12s
	1h 1m 17s

Provide feedback