Worker tests: use unique pointers when possible #4706

Job	Run time
ci (ubuntu-20.04, gcc, g++, 20, Release)	7m 43s
ci (ubuntu-20.04, gcc, g++, 20, Debug)	7m 16s
ci (ubuntu-20.04, clang, clang++, 20, Release)	7m 53s
ci (ubuntu-20.04, clang, clang++, 20, Debug)	6m 53s
ci (ubuntu-22.04, gcc, g++, 20, Release)	7m 7s
ci (ubuntu-22.04, gcc, g++, 20, Debug)	6m 14s
ci (ubuntu-22.04, clang, clang++, 20, Release)	7m 51s
ci (ubuntu-22.04, clang, clang++, 20, Debug)	7m 32s
ci (macos-12, gcc, g++, 20, Release)	7m 52s
ci (macos-12, gcc, g++, 20, Debug)	6m 52s
ci (macos-14, clang, clang++, 20, Release)	3m 58s
ci (macos-14, clang, clang++, 20, Debug)	3m 39s
ci (windows-2022, cl, cl, 20, Release)	7m 0s
ci (windows-2022, cl, cl, 20, Debug)	6m 2s
	1h 33m 52s

Provide feedback