Failed to follow TL;DR chapter

[lukas-vlcek]

I was following the 4 TL;DR version doc chapter and quickly run into issues:

$ register-tool-set
***************************************************************************
WARNING: the script "register-tool-set" has been renamed to "pbench-register-tool-set" 
Please fix your scripts: this compatibility layer will go
away in release 0.38 (approximately 2016-04-04).
The old name "register-tool-set" will NOT work after that - you have been warned!
***************************************************************************

mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
/opt/pbench-agent/base: line 75: /var/lib/pbench-agent/pbench.log: No such file or directory
/opt/pbench-agent/base: line 75: /var/lib/pbench-agent/pbench.log: No such file or directory
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
touch: cannot touch ‘/var/lib/pbench-agent/tools-default/perf’: No such file or directory
/opt/pbench-agent/util-scripts/pbench-register-tool: line 146: /var/lib/pbench-agent/tools-default/perf: No such file or directory
perf tool is now registered in group default

# Giving it a second try using new suggested name
$ pbench-register-tool-set
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
/opt/pbench-agent/base: line 75: /var/lib/pbench-agent/pbench.log: No such file or directory
/opt/pbench-agent/base: line 75: /var/lib/pbench-agent/pbench.log: No such file or directory
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
mkdir: cannot create directory ‘/var/lib/pbench-agent’: Permission denied
touch: cannot touch ‘/var/lib/pbench-agent/tools-default/perf’: No such file or directory
/opt/pbench-agent/util-scripts/pbench-register-tool: line 146: /var/lib/pbench-agent/tools-default/perf: No such file or directory
perf tool is now registered in group default

I am not sure what I did wrong.

$ uname -a
Linux [...] 4.3.4-300.fc23.x86_64 #1 SMP Mon Jan 25 13:39:23 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

$ dnf list pbench-agent
Last metadata expiration check: 0:00:11 ago on Fri Apr 29 13:52:58 2016.
Installed Packages
pbench-agent.noarch                  0.37-1g2139799                  @System      
Available Packages
pbench-agent.src                     0.37-1g2139799                  ndokos-pbench

[lukas-vlcek]

May be this is important, I installed pbench using the following CL:
sudo dnf install pbench-agent

[jeremyeder]

pbench currently requires everything runs as root.

#89

[richm]

Try this:

$ sudo bash
# pbench-register-tool-set

If this works, please close this issue.

[ndokos]

Lukáš Vlček [email protected] writes:

May be this is important, I installed pbench using the following CL:
sudo dnf install pbench-agent

The pbench-agent RPM is not complete: it's missing ssh keys that allow
the agent to send results to the server and it's also missing the
configuration file that lets the agent know who the server is.

If you are installing within Red Hat and want to use the "production"
server that we have set up, please do this:

sudo dnf install pbench-agent-internal

That assumes that you have both the COPR pbench repo and the internal
pbench repo installed in /etc/yum.repos.d.

The pbench-agent-internal RPM has a dependency on pbench-agent, so
installing it is enough to install everything.

If you are trying to install using a different environment, please let
me know: that's a more manual process.

Thanks!

[lukas-vlcek]

@richm I tried:

$ sudo bash
# pbench-register-tool-set
perf tool is now registered in group default

which works but then the next step (user-benchmark) is not working:

[root@dhcp-2-178 pbench_test]# user-benchmark --config=foo -- ./stress.sh 
bash: user-benchmark: command not found...

Just FYI:

# cat ./stress.sh 
#!/bin/bash
sleep 2
stress --cpu 2 --io 1 --vm 1 --vm-bytes 128M --timeout 10s --verbose

[ekuric]

I guess correct one is pbench-user-benchmark. Some time back all pbench commands were prefixed with pbench-
# rpm -ql pbench-agent will list them all.

[lukas-vlcek]

@ekuric Ah! Looks much better now:

[root@dhcp-2-178 pbench_test]# pbench-user-benchmark --config=foo -- ./stress.sh
Running ./stress.sh
stress: info: [8318] dispatching hogs: 2 cpu, 1 io, 1 vm, 0 hdd
stress: dbug: [8318] using backoff sleep of 12000us
stress: dbug: [8318] setting timeout to 10s
stress: dbug: [8318] --> hogcpu worker 2 [8319] forked
stress: dbug: [8318] --> hogio worker 1 [8320] forked
stress: dbug: [8318] --> hogvm worker 1 [8321] forked
stress: dbug: [8318] using backoff sleep of 3000us
stress: dbug: [8318] setting timeout to 10s
stress: dbug: [8318] --> hogcpu worker 1 [8322] forked
stress: dbug: [8321] allocating 134217728 bytes ...
stress: dbug: [8321] touching bytes in strides of 4096 bytes ...
stress: dbug: [8321] freed 134217728 bytes
stress: dbug: [8321] allocating 134217728 bytes ...
....
stress: dbug: [8321] touching bytes in strides of 4096 bytes ...
stress: dbug: [8318] <-- worker 8319 signalled normally
stress: dbug: [8318] <-- worker 8322 signalled normally
stress: dbug: [8318] <-- worker 8320 signalled normally
stress: dbug: [8318] <-- worker 8321 signalled normally
stress: info: [8318] successful run completed in 10s
Collecting system information

And there are some results:

# cat /var/lib/pbench-agent/pbench.log 
[debug][20160506_08:36:29.452522024] tool_opts: --record-opts=record -a --freq=100
[debug][20160506_08:36:29.460510548] checking to see if tool is installed...
[debug][20160506_08:46:21.110981422] [pbench-user-benchmark] processing options
[debug][20160506_08:46:22.437173257] [pbench-start-tools]started: --group=default --iteration=1 --dir=/var/lib/pbench-agent/pbench-user-benchmark_foo_2016-05-06_08:46:21/1/reference-result
[debug][20160506_08:46:22.445333584] /opt/pbench-agent/tool-scripts/perf --start --iteration=1 --group=default --dir=/var/lib/pbench-agent/pbench-user-benchmark_foo_2016-05-06_08:46:21/1/reference-result --record-opts=record -a --freq=100
[debug][20160506_08:46:22.453258826] [pbench-start-tools]completed: 
[info][20160506_08:46:22.454824077] [pbench-user-benchmark] Running ./stress.sh
[debug][20160506_08:46:34.518694418] [pbench-stop-tools]started: --group=default --iteration=1 --dir=/var/lib/pbench-agent/pbench-user-benchmark_foo_2016-05-06_08:46:21/1/reference-result
[debug][20160506_08:46:34.525026806] /opt/pbench-agent/tool-scripts/perf --stop --iteration=1 --group=default --dir=/var/lib/pbench-agent/pbench-user-benchmark_foo_2016-05-06_08:46:21/1/reference-result --record-opts=record -a --freq=100
[debug][20160506_08:46:34.613265375] waiting for PID 8317 (perf) to finish
[debug][20160506_08:46:35.120469393] [pbench-stop-tools]completed: 
[debug][20160506_08:46:35.261752007] [pbench-postprocess-tools]started: --group=default --iteration=1 --dir=/var/lib/pbench-agent/pbench-user-benchmark_foo_2016-05-06_08:46:21/1/reference-result
[debug][20160506_08:46:35.273493885] /opt/pbench-agent/tool-scripts/perf --postprocess --iteration=1 --group=default --dir=/var/lib/pbench-agent/pbench-user-benchmark_foo_2016-05-06_08:46:21/1/reference-result --record-opts=record -a --freq=100
[debug][20160506_08:46:36.872181768] [pbench-postprocess-tools]completed: 
[debug][20160506_08:46:37.031767404] [pbench-sysinfo-dump]collecting libvirt data
[debug][20160506_08:46:37.031693893] [pbench-sysinfo-dump]collecting ethtool data
[debug][20160506_08:46:37.032328884] [pbench-sysinfo-dump]collecting block data
[debug][20160506_08:46:37.032710089] [pbench-sysinfo-dump]collecting system topology
[debug][20160506_08:46:37.034204221] [pbench-sysinfo-dump]done collecting system topology
[debug][20160506_08:46:37.041080968] [pbench-sysinfo-dump]done collecting libvirt data
[debug][20160506_08:46:37.046273925] [pbench-sysinfo-dump]done collecting ethtool data
[debug][20160506_08:46:37.056231136] [pbench-sysinfo-dump]collecting sosreport
[debug][20160506_08:46:37.095634610] [pbench-sysinfo-dump]done collecting block data
[debug][20160506_08:46:43.102218325] [pbench-sysinfo-dump]sosreport collection failed!

Now sure what sosreport collection failed! means but this is a good start.

[lukas-vlcek]

Now, when I want to do the last step:

[root@dhcp-2-178 pbench_test]# pbench-move-results 
[error][20160506_09:03:29.751691725] ERROR: /opt/pbench-agent/id_rsa required for moving results to archive host

I guess this might be related to what @ndokos said above. Am I missing ssh keys?
Is there any option to move results into local folder instead of pushing it into remote server?

[ekuric]

@lukas-vlcek yes, results are at /var/lib/pbench-agent and for every test will get unique directory name, from there one can scp them to any location. Anyway in order to use move-results web server has to be configured where move-resuts will send them need to be configured and pbench-agent.conf needs to be configured on client side. move-results is in very simplified description, set of scp/ssh/tar which uploads results to web server

examples are here
# rpm -ql pbench-agent | grep /opt/pbench-agent/config/pbench-agent.conf /opt/pbench-agent/config/pbench-agent.conf /opt/pbench-agent/config/pbench-agent.conf.example

[lukas-vlcek]

OK, after I installed pbench-agent-internal the move results worked:

[root@dhcp-2-178 pbench_test]# pbench-move-results 
tar Jcf pbench-user-benchmark_foo_2016-05-06_08:46:21.tar.xz pbench-user-benchmark_foo_2016-05-06_08:46:21
pbench-user-benchmark_foo_2016-05-06_08:46:21.tar.xz                                                                                          100% 1295KB   1.3MB/s   00:00    
pbench-user-benchmark_foo_2016-05-06_08:46:21.tar.xz.md5                                                                                      100%   87     0.1KB/s   00:00    
pbench-user-benchmark_foo_2016-05-06_08:46:21.tar.xz: OK
tar Jcf pbench-user-benchmark_bar_2016-05-06_08:59:54.tar.xz pbench-user-benchmark_bar_2016-05-06_08:59:54
pbench-user-benchmark_bar_2016-05-06_08:59:54.tar.xz                                                                                          100% 1297KB   1.3MB/s   00:00    
pbench-user-benchmark_bar_2016-05-06_08:59:54.tar.xz.md5                                                                                      100%   87     0.1KB/s   00:00    
pbench-user-benchmark_bar_2016-05-06_08:59:54.tar.xz: OK

[lukas-vlcek]

Closing. I was able to get through TL;DR chapter finally :-) ... but I needed some help (mostly hitting things due to not up to date documentation).

[ndokos]

I think the documentation is now up-to-date and correct. You should be able to get through TL;DR with
no (or at least fewer) problems.