Worker tests: use unique pointers when possible #4710

Job	Run time
ci (ubuntu-20.04, gcc, g++, 20, Release)	4m 49s
ci (ubuntu-20.04, gcc, g++, 20, Debug)	4m 49s
ci (ubuntu-20.04, clang, clang++, 20, Release)	4m 58s
ci (ubuntu-20.04, clang, clang++, 20, Debug)	4m 56s
ci (ubuntu-22.04, gcc, g++, 20, Release)	4m 56s
ci (ubuntu-22.04, gcc, g++, 20, Debug)	4m 54s
ci (ubuntu-22.04, clang, clang++, 20, Release)	4m 55s
ci (ubuntu-22.04, clang, clang++, 20, Debug)	4m 55s
ci (macos-12, gcc, g++, 20, Release)	4m 56s
ci (macos-12, gcc, g++, 20, Debug)	4m 57s
ci (macos-14, clang, clang++, 20, Release)	3m 59s
ci (macos-14, clang, clang++, 20, Debug)	4m 6s
ci (windows-2022, cl, cl, 20, Release)	4m 40s
ci (windows-2022, cl, cl, 20, Debug)	4m 38s
	1h 6m 28s

Provide feedback