Worker tests: use unique pointers when possible #4707

Job	Run time
ci (ubuntu-20.04, gcc, g++, 20, Release)	7m 50s
ci (ubuntu-20.04, gcc, g++, 20, Debug)	7m 10s
ci (ubuntu-20.04, clang, clang++, 20, Release)	8m 6s
ci (ubuntu-20.04, clang, clang++, 20, Debug)	6m 49s
ci (ubuntu-22.04, gcc, g++, 20, Release)	7m 18s
ci (ubuntu-22.04, gcc, g++, 20, Debug)	6m 21s
ci (ubuntu-22.04, clang, clang++, 20, Release)	8m 56s
ci (ubuntu-22.04, clang, clang++, 20, Debug)	8m 17s
ci (macos-12, gcc, g++, 20, Release)	6m 13s
ci (macos-12, gcc, g++, 20, Debug)	6m 30s
ci (macos-14, clang, clang++, 20, Release)	4m 34s
ci (macos-14, clang, clang++, 20, Debug)	3m 50s
ci (windows-2022, cl, cl, 20, Release)	6m 38s
ci (windows-2022, cl, cl, 20, Debug)	6m 19s
	1h 34m 51s

Provide feedback