Worker tests: use unique pointers when possible #4698

Job	Run time
ci (ubuntu-20.04, gcc, g++, 20, Release)	7m 44s
ci (ubuntu-20.04, gcc, g++, 20, Debug)	7m 28s
ci (ubuntu-20.04, clang, clang++, 20, Release)	8m 50s
ci (ubuntu-20.04, clang, clang++, 20, Debug)	6m 53s
ci (ubuntu-22.04, gcc, g++, 20, Release)	7m 1s
ci (ubuntu-22.04, gcc, g++, 20, Debug)	6m 26s
ci (ubuntu-22.04, clang, clang++, 20, Release)	8m 48s
ci (ubuntu-22.04, clang, clang++, 20, Debug)	7m 56s
ci (macos-12, gcc, g++, 20, Release)	10m 55s
ci (macos-12, gcc, g++, 20, Debug)	10m 7s
ci (macos-14, clang, clang++, 20, Release)	4m 38s
ci (macos-14, clang, clang++, 20, Debug)	4m 6s
ci (windows-2022, cl, cl, 20, Release)	7m 38s
ci (windows-2022, cl, cl, 20, Debug)	6m 16s
	1h 44m 46s

Provide feedback