Worker tests: use unique pointers when possible #4711

Job	Run time
ci (ubuntu-20.04, gcc, g++, 20, Release)	7m 54s
ci (ubuntu-20.04, gcc, g++, 20, Debug)	7m 18s
ci (ubuntu-20.04, clang, clang++, 20, Release)	7m 58s
ci (ubuntu-20.04, clang, clang++, 20, Debug)	7m 3s
ci (ubuntu-22.04, gcc, g++, 20, Release)	7m 6s
ci (ubuntu-22.04, gcc, g++, 20, Debug)	6m 41s
ci (ubuntu-22.04, clang, clang++, 20, Release)	9m 10s
ci (ubuntu-22.04, clang, clang++, 20, Debug)	7m 46s
ci (macos-12, gcc, g++, 20, Release)	7m 22s
ci (macos-12, gcc, g++, 20, Debug)	5m 49s
ci (macos-14, clang, clang++, 20, Release)	3m 58s
ci (macos-14, clang, clang++, 20, Debug)	3m 39s
ci (windows-2022, cl, cl, 20, Release)	7m 2s
ci (windows-2022, cl, cl, 20, Debug)	6m 48s
	1h 35m 34s

Provide feedback