From 09e1186f8a3a71cc59d1946fcb8a59bcd3aa7c18 Mon Sep 17 00:00:00 2001 From: Travis Wichert Date: Tue, 31 Oct 2017 12:50:49 -0400 Subject: [PATCH] Develop (#372) * Update ECS CE Docker image with ECS 3.0.0 Hotfix 2 (#312) * Reorg base files (fix my glitch) * Fixed Dockerfile so it patches everything in-place. (cherry picked from commit 867189eb92e36c8388d389273f2942b9f277380e) * update release files to use new image. * Duct tape for #301 Update to ECS 3.0.0.2 (3.0.0 HF 2) (#314) * fix a small typo * fix package install issue because EPEL is between versions again. * quick fixes for crashing HF2 to unblock clients * Docs update (#313) * Adds FAQ page * installation troubleshooting * formatting * Addition of network troubleshooting * Addition of network troubleshooting * More Troubleshooting * sidebar implementation attempt * implements important links dropdown, disables page dropdown. * whoops * Adds migration page and some small updates. * Bunch of docs updates * bugfix-hf2 (#315) * multitail and dstat are coming in handy right now * change fact cache location to /var/cache/emc/ecs-install * log the state of Docker at the end of bootstrapping for troubleshooting help (those hashes are good to see!) * [Ansible] Stop templating, start regex replacing props/confs * release prep 2.3.0 (#316) * OVA prep (#318) * Configure Jenkins pipeline to test installation process * Get repository information from Jenkins SCM config * Fix env var * Obtain TF options from Jenkins params. Moved deprovision step to post action * Allow multi-node configuration * Update checkout step in jenkinsfile * add zerofill.sh to /tools (#324) (cherry picked from commit 58f7e8eea587c5a005eff3df2c2733ac2f5e1a9c) * Add slack notifications to jenkins pipeline * [WiP] Configure Jenkins server to build PR and provide feedback (#328) Configure Jenkins server to build PR and provide feedback * Docs pass 2 (#326) * Removal of deprecated procedures * Templates * OVA install guide added * Fixed broken links * preflight remove all bootstrap packages if installed (#330) * Non-PR Jenkins jobs do not provide URL and commit author (#331) Fix variables in Slack notifications * put yum actions in retry loop with timeout (#332) Put yum actions in retry loop with timeout * Implements #205 Installer must have public key initial auth capability (#270) * ECS-CommunityEdition-205 Installer must have public key initial auth capability (cherry picked from commit 6eea10b5db3985f960d7b313d2e705a0f913ba55) * More sausage for the initial ssh key auth (cherry picked from commit 8535ccb5430e89b79d253ea1e74390a39b8b20f3) * more sausage (cherry picked from commit edf961e0765cd9a06ccea1a6d1e2406816533f46) * deploy.yml change ideas (cherry picked from commit ef48e2cc57fa6d0a57aa30bc62ba816fc167aed9) * bootstrap.sh modifications (cherry picked from commit d0b3c630f0a2004fe23534aca7e4a95986dce383) * bootstrap.sh modifications (cherry picked from commit 86f897af9395a57af5b33c2162cd85422b4e6ded) * move generic help to generic_help.j2.yml file from config.yml * include shipit.lib.sh * build install paths early add copy action for ssh PKI material * fix a couple gitopt bugs * add create_install_tree() to plugin-defaults.sh * copy ssh keys in bootstrap.sh * more longopts adjustments * add loop delay in retry_with_timeout() * stop trying to autoremove curl, it'll always error. * key_vals need basename not full path set 0700 bits on ssh/ssl stores * remove optarg debugging * more ssh pubkey sausage * update reference.deploy.yml to include feature * jenkins changes * jenkins changes * jenkins changes * jenkins changes * jenkins changes * bump versions and move OVA download links. (#335) * open-vm-tools now has a cross dependency (#337) with open-vm-tools-desktop and yum fails to install open-vm-tools on remote nodes when open-vm-tools-desktop is not installed. * ECS-CommunityEdition-317 Make `ecsconfig ping -cx` loop when dtquery fails (#344) (cherry picked from commit f5c7810a3385352a7ceb3bd9af66f9f824927dca) * Change the way ecs-install is pushed to repo (#346) * invoke zerofill via bash rather than expecting exec bit (#343) * Remove Ansible verbosity flag from Jenkinsfile (#350) * OVA QoL Improvements (#351) * add `ova-step1` and `ova-step2` macros * add `ecsdeploy noop` for some ova macros to look better * make videploy more intelligent and play nice with update_deploy * Implement Ansible global OVA flag fact (#349) * implements Ansible global ova flag fact - custom fact in /etc/ansible/facts.d/ova.fact - ova conditional flags in playbooks * misaligned `when` * skip rebooting when using the OVA. * Upgrade Ansible to 2.3 (#347) * install ansible package from @edge_main for 2.3 * Ansible changes for Ansible 2.3 * ECS-CommunityEdition-235 Bump Ansible version to 2.3 * refactor Ansible task `when:` clauses to Ansible 2.3 spec * refactor node reboot actions for Ansible 2.3 Also resolve #342 * remove unused json_file plugin * must ignore_errors: True `needs-restarting -r` * refactor port-check `when`s to Ansible 2.3 spec * cleanup * add loop_control to path permissions entries * add loop_control labels to many iterators * add loop_control labels to many iterators * break out one directive per line * add loop_control labels to many iterators * incorrect `when` * speling * Switch to Alpine 3.6 release (#359) * Switch to Alpine 3.6 release Install Python 2 from APK * Changes to Rockerfile for python:2-alpine parity * Split steps out from Ansible to get realtime console logging (#358) * Split steps out from Ansible to get realtime console logging * use /tmp? * template out a script to run command on install node via IP * Jenkinsfile sausage * Jenkinsfile sausage * Jenkinsfile sausage * Jenkinsfile sausage * Jenkinsfile sausage * Jenkinsfile sausage * Jenkinsfile sausage * Jenkinsfile sausage * Jenkinsfile sausage * log environment info to file log only, never to console. * Add CentOS 7.4 support (#360) (cherry picked from commit 1a960873d5d6f0b1011caabcbb3987114723dd8a) * [WiP] Misc. 2.5.0 bugfixes (#352) * update reference deploy version * bugfix typo in ed25519 private key filename * fix ova flag implementation * fix ova flag implementation round 2 * fix ova flag implementation round 3 * update entrypoint.sh * bump version to 2.5.0b1 (#364) * [WiP] Update ecs-install Python requirements (#356) * update python requirements * pin python requirements to major versions rather than patches. * add python2-dev to temporary build environment * need cryptography>=1.9 * [WiP] ECS 3.1.0.0 Reduced GA and CE Support (#353) * make 3.0.0.2 to use 100% regex (cherry picked from commit 0689ce59ec9e61beea4082b1f23222031b17f62d) * prep 3.1.0.0 RC3 (cherry picked from commit 2c3d5b9e92bcec83e01362c674a00158e6a61289) * prep 3.1.0.0 RC3 (cherry picked from commit ea12c296204442adf1e5079d778171140dede101) * ECS 3.1 templates * local facts must be fully qualified? * interface roles should be defined in deploy.yml * actually use a comma in the jinja joiner() func * use ansible_fqdn for agent strings not ansible_hostname * ECS 3.1.0.0 RC4 * joiner() needs to be the prefix not the suffix * the infamous missing comma * no trailing comma * remove redundant spaces * set host: field in testing * make object-main_network.json.j2 VDC-aware + formatting * Set georeceiver initialBufferNumOnHeap to 10 * Mount /usr instead of /usr/local to capture new install path * [WiP] ECS 3.1 Full GA and CE support (#367) * Use nodeId instead of the node IP to create data store * Fix errors getting node ID * fix 3.1 patch again * migrate cm.object.properties/'MustHaveEnoughResources=false' into Dockerfile * Run cf_client in container for new low partition count vars * Run cf_client in container for new low partition count vars * migrate cf_client variable settings into Dockerfile * update comments in Dockerfile for 3.1.0.0 * release-2.5.0-prep (#370) * Update ECS-Installation.md (cherry picked from commit f8be70f53b55bf718e2f1bb32df206484a12e7e8) * Update ECS-Installation.md (cherry picked from commit b479b0722308aaa40345af524f5e4430d29b11ed) * bump versions --- Jenkinsfile | 45 +++- bootstrap.sh | 4 +- bootstrap_plugins/centos74.plugin.sh | 200 ++++++++++++++++++ bootstrap_plugins/os-router.plugin.sh | 4 +- docs/design/reference.deploy.yml | 2 +- patches/3.0.0.2/Dockerfile | 5 +- patches/3.1.0.0/Dockerfile | 23 +- .../3.1.0.0/storageserver-partition-config.sh | 119 ----------- patches/3.1.0.0/transformsvc | 27 --- patches/3.1.0.0/vnest.object.properties | 126 ----------- tests/ansible/install_node_reboot.yml | 6 + tests/ansible/install_node_setup.yml | 6 + .../roles/install_node_reboot/tasks/main.yml | 8 + .../roles/install_node_setup/tasks/main.yml | 28 +++ tests/tf_to_hosts.py | 25 +++ tests/tf_to_ssh.py | 28 +++ ui/ansible/clicmd_start.yml | 1 + ui/ansible/library/json_file | 83 -------- .../CentOS_7_baseline_install/tasks/main.yml | 10 +- .../CentOS_7_configure_ssh/tasks/main.yml | 14 +- .../roles/CentOS_7_reboot/tasks/main.yml | 41 ++-- .../CentOS_7_sync_caches_prep/tasks/main.yml | 12 +- .../common_baseline_check/tasks/main.yml | 7 +- .../common_baseline_install/tasks/main.yml | 26 ++- .../templates/agent.json.j2 | 3 + .../templates/config_cluster_network.j2 | 62 ------ .../templates/{id.j2 => id-old.json.j2} | 0 .../templates/id.json.j2 | 3 + .../templates/object-main_network.json.j2 | 29 +++ .../{network-json.j2 => rev0-network.json.j2} | 0 .../templates/rev1-network.json.j2 | 15 ++ .../templates/data_node.fact.j2 | 3 +- ui/ansible/roles/common_deploy/tasks/main.yml | 4 + .../roles/common_purge_cleanup/tasks/main.yml | 6 +- .../roles/common_run_cf_client/tasks/main.yml | 8 + .../roles/common_set_hostname/tasks/main.yml | 2 +- .../common_sync_caches_by_copy/tasks/main.yml | 4 +- .../tasks/main.yml | 8 +- .../installer_build_cache/tasks/main.yml | 16 +- .../tasks/main.yml | 2 + ui/ansible/roles/testing/tasks/main.yml | 39 ++++ .../roles/testing/templates/agent.json.j2 | 3 + .../roles/testing/templates/id-old.json.j2 | 7 + ui/ansible/roles/testing/templates/id.json.j2 | 3 + .../templates/object-main_network.json.j2 | 29 +++ .../testing/templates/rev0-network.json.j2 | 8 + .../testing/templates/rev1-network.json.j2 | 15 ++ ui/ansible/roles/testing/templates/seeds.j2 | 8 + ui/ansible/roles/testing/vars/main.yml | 44 ++++ ui/ansible/testing.yml | 1 + ui/build_image.sh | 11 +- ui/ecsconfig.py | 30 ++- ui/ecsdeploy.py | 8 + ui/etc/config.yml | 4 +- ui/etc/release.conf | 12 +- ui/libexec/console.sh | 10 + ui/libexec/osutils.sh | 78 +++---- ui/resources/docker/Rockerfile | 17 +- .../docker/ecs-install-requirements.txt | 43 ++-- ui/resources/docker/entrypoint.sh | 6 +- ui/run.sh | 25 ++- ui/setup.py | 2 +- 62 files changed, 837 insertions(+), 581 deletions(-) create mode 100644 bootstrap_plugins/centos74.plugin.sh delete mode 100644 patches/3.1.0.0/storageserver-partition-config.sh delete mode 100644 patches/3.1.0.0/transformsvc delete mode 100644 patches/3.1.0.0/vnest.object.properties create mode 100644 tests/ansible/install_node_reboot.yml create mode 100644 tests/ansible/install_node_setup.yml create mode 100644 tests/ansible/roles/install_node_reboot/tasks/main.yml create mode 100644 tests/ansible/roles/install_node_setup/tasks/main.yml create mode 100644 tests/tf_to_hosts.py create mode 100644 tests/tf_to_ssh.py delete mode 100644 ui/ansible/library/json_file create mode 100644 ui/ansible/roles/common_baseline_install/templates/agent.json.j2 delete mode 100644 ui/ansible/roles/common_baseline_install/templates/config_cluster_network.j2 rename ui/ansible/roles/common_baseline_install/templates/{id.j2 => id-old.json.j2} (100%) create mode 100644 ui/ansible/roles/common_baseline_install/templates/id.json.j2 create mode 100644 ui/ansible/roles/common_baseline_install/templates/object-main_network.json.j2 rename ui/ansible/roles/common_baseline_install/templates/{network-json.j2 => rev0-network.json.j2} (100%) create mode 100644 ui/ansible/roles/common_baseline_install/templates/rev1-network.json.j2 create mode 100644 ui/ansible/roles/common_run_cf_client/tasks/main.yml create mode 100644 ui/ansible/roles/testing/tasks/main.yml create mode 100644 ui/ansible/roles/testing/templates/agent.json.j2 create mode 100644 ui/ansible/roles/testing/templates/id-old.json.j2 create mode 100644 ui/ansible/roles/testing/templates/id.json.j2 create mode 100644 ui/ansible/roles/testing/templates/object-main_network.json.j2 create mode 100644 ui/ansible/roles/testing/templates/rev0-network.json.j2 create mode 100644 ui/ansible/roles/testing/templates/rev1-network.json.j2 create mode 100644 ui/ansible/roles/testing/templates/seeds.j2 create mode 100644 ui/ansible/roles/testing/vars/main.yml diff --git a/Jenkinsfile b/Jenkinsfile index f180a985..1d74b631 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -62,19 +62,54 @@ pipeline { sh 'terraform output -json > output.json' } } - stage('Deploy ECS'){ + stage('Setup install node'){ steps { - sh './tests/tf_to_hosts output.json hosts.ini' + sh 'chmod +x ./tests/tf_to_hosts.py' + sh 'chmod +x ./tests/tf_to_ssh.py' + sh './tests/tf_to_hosts.py output.json hosts.ini' + sh "./tests/tf_to_ssh.py output.json ./ssh.sh $SSH_USR" + sh 'chmod +x ./ssh.sh' + sh 'cat output.json' + sh 'cat hosts.ini' + sh 'cat ./ssh.sh' ansiblePlaybook \ - playbook: 'tests/ansible/install_node.yml', + playbook: 'tests/ansible/install_node_setup.yml', inventory: 'hosts.ini', extraVars: [ ansible_ssh_user: "$SSH_USR", ansible_ssh_pass: "$SSH_PSW", ansible_become_pass: "$SSH_PSW", current_directory: "$WORKSPACE" - ], - extras: '-vvv' + ] + } + } + stage('Bootstrap install node'){ + steps { + sh './ssh.sh curl http://10.1.83.5/registry.crt -o /tmp/registry.crt' + sh './ssh.sh /root/ecs/bootstrap.sh -n -v --build-from http://10.1.83.5/alpine --vm-tools --proxy-cert /root/ecs/contrib/sslproxycert/emc_ssl.pem --proxy-endpoint 10.1.83.5:3128 -c /root/ecs/deploy.yml --centos-mirror 10.1.83.5 --registry-cert /tmp/registry.crt --registry-endpoint cache.gotham.local:5000 --override-dns 10.1.83.19' + } + } + stage('Reboot install node'){ + steps { + ansiblePlaybook \ + playbook: 'tests/ansible/install_node_reboot.yml', + inventory: 'hosts.ini', + extraVars: [ + ansible_ssh_user: "$SSH_USR", + ansible_ssh_pass: "$SSH_PSW", + ansible_become_pass: "$SSH_PSW", + current_directory: "$WORKSPACE" + ] + } + } + stage('Deploy ECS'){ + steps { + sh './ssh.sh step1' + } + } + stage('Configure ECS'){ + steps { + sh './ssh.sh step2' } } } diff --git a/bootstrap.sh b/bootstrap.sh index 834b3a14..7d82eec8 100755 --- a/bootstrap.sh +++ b/bootstrap.sh @@ -251,7 +251,7 @@ while true; do export mirror_val="${2}" shift 2 ;; - -o|--override-dhcp-dns) + -o|--override-dns) export dhcpdns_flag=true export dhcpdns_val="${2}" shift 2 @@ -781,7 +781,7 @@ if get_os_needs_restarting; then fi if ${zerofill_flag}; then - sudo "${INSTALL_ROOT}/tools/zerofill.sh" + sudo bash "${INSTALL_ROOT}/tools/zerofill.sh" fi diff --git a/bootstrap_plugins/centos74.plugin.sh b/bootstrap_plugins/centos74.plugin.sh new file mode 100644 index 00000000..9ef8a454 --- /dev/null +++ b/bootstrap_plugins/centos74.plugin.sh @@ -0,0 +1,200 @@ +#@IgnoreInspection BashAddShebang + +# Copyright (c) 2015 EMC Corporation +# All Rights Reserved +# +# This software contains the intellectual property of EMC Corporation +# or is licensed to EMC Corporation from third parties. Use of this +# software and the intellectual property contained therein is expressly +# limited to the terms and conditions of the License Agreement under which +# it is provided by or on behalf of EMC. + +# OS Support library for CentOS 7.3 + +os_supported=true + +# Docker binary +docker_binary='/bin/docker' + +# packages to clean up during preflight +# Don't `yum autoremove curl`. Yum is a dependency and it will throw errors. +list_preflight_packages="git nfs-client nfs-tools rsync wget ntp docker vim pigz gdisk aria2 htop iotop iftop multitail dstat jq python-docker-py dkms qemu-guest-agent open-vm-tools open-vm-tools-desktop docker" + +# Do any OS-specific tasks that must be done prior to bootstrap +do_preflight() { + rm_repo_pkg "$list_preflight_packages" +} + +# packages to install before others +list_prefix_packages='wget curl epel-release yum-utils' + +# script to run for installing prefix_packages +in_prefix_packages() { + in_repo_pkg "$list_prefix_packages" +} + +# packages to install +# list_general_packages='yum-utils git python-pip python-docker-py' +list_general_packages='git ntp docker vim rsync pigz gdisk aria2' + +# script to run for installing general_packages +in_general_packages() { + in_repo_pkg "$list_general_packages" +# if ! docker version; then +# curl -fsSL https://get.docker.com/ | sudo sh +# fi + sudo systemctl enable docker + sudo systemctl start docker + sudo usermod -aG docker $(whoami) +} + +# packages to install after others +list_suffix_packages='htop iotop iftop multitail dstat jq python-docker-py' +# list_suffix_packages='htop jq pigz gdisk aria2 python-docker-py' + +# script to run for installing suffix_packages +in_suffix_packages() { + in_repo_pkg "$list_suffix_packages" + + # Install Rocker + curl -fsSL ${rocker_artifact_url} \ + | sudo tar -xzC /usr/local/bin && sudo chmod +x /usr/local/bin/rocker +} + +# packages to install if a VM +list_vm_packages='dkms qemu-guest-agent open-vm-tools open-vm-tools-desktop' + +# command to run for installing vm_packages +in_vm_packages() { + in_repo_pkg "$list_vm_packages" + # return 0 +} + +# command to install one or more os package manager package +in_repo_pkg() { + retry_with_timeout 10 300 sudo yum -y install $* +} + +rm_repo_pkg() { + retry_with_timeout 10 300 sudo yum -y autoremove $* +} + +# command to update all packages in the os package manager +up_repo_pkg_all() { + retry_with_timeout 10 300 sudo yum -y update +} + +# command to rebuild the os package manager's database +up_repo_db() { + retry_with_timeout 10 300 sudo yum -y makecache +} + +# command to set os package manager proxy +set_repo_proxy_conf() { + sudo sed -i -e '/^proxy=/d' /etc/yum.conf + echo "proxy=${http_proxy}" \ + | append /etc/yum.conf +} + +# command to set os package manager to keep its cache +set_repo_keepcache_conf() { + sudo sed -i -e '/^keepcache=/d' /etc/yum.conf + echo "keepcache=1" \ + | append /etc/yum.conf +} + +# idempotent config script to fixup repos to properly use proxycaches +set_repo_cacheable_idempotent() { + sudo sed -i -e 's/^#baseurl=/baseurl=/' /etc/yum.repos.d/* + sudo sed -i -e 's/^mirrorlist=/#mirrorlist=/' /etc/yum.repos.d/* +} + +set_repo_mirror_idempotent() { + # sudo sed -i -e "s#http:///centos#http://${mirror_val}/centos#g" /etc/yum.repos.d/* + sudo sed -i -e "s#http://.*/centos#http://${mirror_val}/centos#g" /etc/yum.repos.d/* +} + +# command to set the proxy for the whole OS +set_os_proxy() { + sudo sed -i -e '/_proxy/d' /etc/environment + echo -n "http_proxy=${http_proxy}\nhttps_proxy=${http_proxy}\nftp_proxy=${http_proxy}\n" \ + | append /etc/environment + if $mirror_flag; then + echo -n "no_proxy=${mirror_val}\n" | append /etc/environment + fi +} + +# command to determine if the OS needs restarting after package updates +get_os_needs_restarting() { + if ! [ -z "$(sudo /usr/bin/needs-restarting)" ]; then + return 0 + else + return 1 + fi +} + +# command to reboot the system +do_reboot() { + sudo reboot +} + +# Command to configure docker's proxy under centos flavored systemd +set_docker_proxy() { + local tmpconf="/etc/systemd/system/docker.service.d/http-proxy.conf" + if ! [ -d "$(dirname $tmpconf)" ]; then + sudo mkdir "$(dirname $tmpconf)" + fi + log "sed error is OK here if the proxy config file does not yet exist." + sudo sed -i -e '/HTTP_PROXY/d' "$tmpconf" + echo "Environment=\"HTTP_PROXY=${http_proxy}\" \"NO_PROXY=localhost,127.0.0.1,$(hostname),$(hostname -f)\"" \ + | append "$tmpconf" + sudo systemctl daemon-reload + sudo systemctl restart docker + sudo systemctl status docker +} + +# command to add mitm cert to docker trust store +set_docker_reg_cert() { + local registry="${1}" + local cert="${2}" + if ! [ -d "/etc/docker/certs.d/${registry}" ]; then + sudo mkdir -p "/etc/docker/certs.d/${registry}" + sudo cp "${cert}" "/etc/docker/certs.d/${registry}/ca.crt" + else + if [ -f "/etc/docker/certs.d/${registry}/ca.crt" ]; then + echo "Reusing existing /etc/docker/certs.d/${registry}/ca.crt" + else + sudo cp "${cert}" "/etc/docker/certs.d/${registry}/ca.crt" + fi + fi + set_mitm_cert "${cert}" + sudo systemctl daemon-reload + sudo systemctl restart docker + sudo systemctl status docker +} + +# command to add mitm cert to local trust store +set_mitm_cert() { + sudo cp "${1}" "/etc/pki/ca-trust/source/anchors/$(basename ${1}).crt" + sudo update-ca-trust extract +} + +do_post_install() { + # Disable postfix since we don't need an MTA + sudo systemctl disable --now postfix +} + +override_dhcp_dns() { + nameserver_list="${1}" + sudo sed -i -e 's/PEERDNS="yes"/PEERDNS="no"/' /etc/sysconfig/network-scripts/ifcfg-* + sudo sed -i -e '/DNS[0-9]=/d' /etc/sysconfig/network-scripts/ifcfg-* + sudo sed -i -e '/nameserver/d' /etc/resolv.conf + nsnumber=1 + for nameserver in $(echo ${nameserver_list} | tr ',' ' '); do + echo "nameserver ${nameserver}" | append /etc/resolv.conf + for script in /etc/sysconfig/network-scripts/ifcfg-*; do + echo "DNS${nsnumber}=${nameserver}" | append "${script}" + done + nsnumber=$((nsnumber++)) + done +} diff --git a/bootstrap_plugins/os-router.plugin.sh b/bootstrap_plugins/os-router.plugin.sh index 6ae5a2b6..c8207d38 100755 --- a/bootstrap_plugins/os-router.plugin.sh +++ b/bootstrap_plugins/os-router.plugin.sh @@ -61,10 +61,12 @@ route_os() { centos\ linux\ release\ 7.2*) source ${plugins}/centos72.plugin.sh ;; - centos\ linux\ release\ 7.3*) source ${plugins}/centos73.plugin.sh ;; + centos\ linux\ release\ 7.4*) + source ${plugins}/centos74.plugin.sh + ;; # dockerized\ centos\ linux\ release\ 7.2*) # source ${plugins}/centos72-docker.plugin.sh diff --git a/docs/design/reference.deploy.yml b/docs/design/reference.deploy.yml index ec450a7b..d25be15e 100644 --- a/docs/design/reference.deploy.yml +++ b/docs/design/reference.deploy.yml @@ -1,4 +1,4 @@ -# deploy.yml reference implementation v2.2.0 +# deploy.yml reference implementation v2.5.0 # [Optional] # By changing the license_accepted boolean value to "true" you are diff --git a/patches/3.0.0.2/Dockerfile b/patches/3.0.0.2/Dockerfile index 95b19965..8cf1745b 100644 --- a/patches/3.0.0.2/Dockerfile +++ b/patches/3.0.0.2/Dockerfile @@ -1,4 +1,4 @@ -# Fixes to the default 3.0 HF2 image. +# Fixes to the default 3.0 HF2 reduced image. FROM emcvipr/object:3.0.0.0-86889.0a0ee19-reduced # Increase memory for transformsvc @@ -7,7 +7,8 @@ RUN sed -i s/-Xmx128m/-Xmx512m/ /opt/storageos/bin/transformsvc # Fix disk partitioning script RUN sed -i '/VMware/ s/$/ \&\& [ ! -e \/data\/is_community_edition ]/' /opt/storageos/bin/storageserver-partition-config.sh -COPY vnest-common-conf-template.xml /opt/storageos/conf/vnest-common-conf-template.xml +# COPY vnest-common-conf-template.xml /opt/storageos/conf/vnest-common-conf-template.xml +RUN f=/opt/storageos/conf/vnest-common-conf-template.xml; grep -q "object.UseSeparateThreadPools" $f || sed -i '/properties id="serviceProperties"/a \ \ \ \ \ \ \ \ true' $f # Make vnest use separate thread pools to prevent deadlock RUN printf "\n# Use separate thread pools to prevent deadlock in vnest init\nobject.UseSeparateThreadPools=true\n" >> /opt/storageos/conf/vnest.object.properties diff --git a/patches/3.1.0.0/Dockerfile b/patches/3.1.0.0/Dockerfile index 55e98f6d..00c778bf 100644 --- a/patches/3.1.0.0/Dockerfile +++ b/patches/3.1.0.0/Dockerfile @@ -1,14 +1,23 @@ -# Fixes to the default 3.0HF image. -FROM emcvipr/object:3.1.0.0-93256.00e3410-reduced +# Fixes to the default 3.1.0.0 reduced image. + +# Build on RC4 object image (GA release) +FROM emcvipr/object:3.1.0.0-95266.ab2753a-reduced # Increase memory for transformsvc -ADD transformsvc /opt/storageos/bin/ +RUN sed -i s/-Xmx128m/-Xmx512m/ /opt/storageos/bin/transformsvc # Fix disk partitioning script -ADD storageserver-partition-config.sh /opt/storageos/bin/ +RUN sed -i '/VMware/ s/$/ \&\& [ ! -e \/data\/is_community_edition ]/' /opt/storageos/bin/storageserver-partition-config.sh RUN /usr/bin/chmod +x /opt/storageos/bin/storageserver-partition-config.sh -# Make vnest use separate thread pools to prevent deadlock -ADD vnest.object.properties /opt/storageos/conf/ - +# Set VNets useSeperateThreadPools to True RUN f=/opt/storageos/conf/vnest-common-conf-template.xml; grep -q "object.UseSeparateThreadPools" $f || sed -i '/properties id="serviceProperties"/a \ \ \ \ \ \ \ \ true' $f + +# Set georeceiver's initialBufferNumOnHeap to something smaller for CE +RUN f=/opt/storageos/conf/georeceiver-conf.xml; grep -q 'name="initialBufferNumOnHeap" value="10"' $f || sed -i 's/name="initialBufferNumOnHeap" value="60"/name="initialBufferNumOnHeap" value="10"/' $f + +# Configure CM Object properties: Disable minimum storage device count +RUN f=/opt/storageos/conf/cm.object.properties; grep -q 'MustHaveEnoughResources=false' $f || sed -i 's/MustHaveEnoughResources=true/MustHaveEnoughResources=false/' $f + +# Allow allocation of different blocks of a chunk to be stored on the same partition +RUN sed -i 's###g' /opt/storageos/conf/ssm-cf-conf.xml diff --git a/patches/3.1.0.0/storageserver-partition-config.sh b/patches/3.1.0.0/storageserver-partition-config.sh deleted file mode 100644 index dbb9a0b4..00000000 --- a/patches/3.1.0.0/storageserver-partition-config.sh +++ /dev/null @@ -1,119 +0,0 @@ -#!/bin/bash -# Copyright (c) 2013 EMC Corporation -# All Rights Reserved -# -# This software contains the intellectual property of EMC Corporation -# or is licensed to EMC Corporation from third parties. Use of this -# software and the intellectual property contained therein is expressly -# limited to the terms and conditions of the License Agreement under which -# it is provided by or on behalf of EMC. -# - -date - -# This script will generate the partition file (ss-partition-conf.json) to be used on this server if one is not found. -# Also a general configuration file will be generated with main parameters and a ss uuid. -# Assumes this is running on gen 3 hardware or a vipr data test node using disks mounted with a -# /dae/uuid-58f917ab-23ba-457f-a9df-d0f4b0597e2d type pattern or will look for /data (vipr) - -export configdir="/opt/storageos/conf" - -export p_configfile="$configdir/storageserver-partition-conf.json" -export configfile="$configdir/storageserver.conf" - -blockbinsizegb=10 -vm_bb_cout=5 -vm_part_count=5 - -function setopt() { - opt=$1 - val=$2 - if grep -q "$opt" "$configfile" ; then - sed -i "s#$opt=.*#$opt=$val#g" "$configfile" - else - echo "$opt=$val" >> $configfile - fi - -} - -echo "creating main config file: $configfile" -if [ ! -e "$configfile" ]; then - echo "creating $configfile" - mkdir -p "$configdir" - cat << EOF > "$configfile" -listenaddress=0.0.0.0 -port=9099 -iothreads=100 -blockbinsizegb=$blockbinsizegb -debugenabled=false -logtoconsole=false -logtosyslog=true -maxoutstandingrequests=0 -initblockbins=0 -netserverListenPort=9069 -netserverLogLevel=INFO -ioThreadNumber=200 -perfCountEnabled=true -maxMemThresholdKb=3145728 -sendQHighWaterMark=1000 -recvThreadNicePriority=-15 -sendThreadNicePriority=-15 -enableAffinity=true -partitionroot=/dae -agentUrlEndPoint=/host/data/agent.json -agentDisksRestFragment=/v1/agent/node/storage/disk/disks -trustStore=/host/data/security/truststore.pem -verifyCerts=true -dbusEnabled=false -EOF -else - echo "$configfile already exists, no action taken" -fi - -if grep -q -E 'VMware.*Virtual.*disk' /proc/scsi/sg/device_strs && [ ! -e /data/is_community_edition ] -then - # this a virtual datanode, creating a json file with predefined values - echo "virtual node detected - using json config file" - setopt partitionconfig $p_configfile - setopt partitionroot=/dae - root="/data/storageserver" - - for part in $(seq 1 $vm_part_count); do - for bb in $(seq 1 $vm_bb_cout); do - bb=$(printf "%0*d\n" 4 $bb) - mkdir -m 777 -p $root/uuid-$part - fallocate -l${blockbinsizegb}G $root/uuid-$part/$bb - chmod 777 $root/uuid-$part/$bb - done - done - - if [ ! -e "$p_configfile" ]; then - echo "creating $p_configfile" - mkdir -p "$configdir" - - printf '{\n%2s"disks": [\n' > $p_configfile - for part in $(seq 1 "$vm_part_count"); do - printf '%4s{\n%6s"uuid": "'$part'",\n%6s"health": "Good"\n%4s}' >> $p_configfile - [ "$part" -lt "$vm_part_count" ] && echo "," >> $p_configfile || echo "" >> $p_configfile - done - printf '%2s]\n}' >> $p_configfile - - else - echo "$p_configfile already exists, no action taken" - fi -else - # this is a commodity hardware node. do not generate a json file - ss will use dbus to obtain partitions - echo "commodity node detected - using dbus api" -fi - -if ! grep --quiet "uuid" "$configfile"; then - echo "generating server uuid" - uuidgen="$(which uuidgen)" - if [ ! -z "$uuidgen" ]; then - uuid="$(uuidgen)" - else - echo "warning: uuidgen utility was not found on your system. using timestamp as a uuid" - uuid="$(date +%s)" - fi - setopt uuid "$uuid" -fi diff --git a/patches/3.1.0.0/transformsvc b/patches/3.1.0.0/transformsvc deleted file mode 100644 index 144f5edc..00000000 --- a/patches/3.1.0.0/transformsvc +++ /dev/null @@ -1,27 +0,0 @@ -#!/bin/sh -LIB_DIR="/opt/storageos/lib" -export JAVA_HOME="/usr/lib64/jvm/java-1.8.0-oracle/jre" -export PATH="${JAVA_HOME}/bin:/opt/storageos/bin:/bin:/usr/bin" - -export CLASSPATH="/opt/storageos/conf:${LIB_DIR}:${LIB_DIR}/storageos-transformsvc.jar:${LIB_DIR}/rsa-bsafe-for-centera.jar:${LIB_DIR}/slf4j-api-1.7.5.jar:${LIB_DIR}/jul-to-slf4j-1.7.5.jar:${LIB_DIR}/jcl-over-slf4j-1.7.5.jar:${LIB_DIR}/log4j-core-2.5.jar:${LIB_DIR}/log4j-api-2.5.jar:${LIB_DIR}/log4j-slf4j-impl-2.5.jar:${LIB_DIR}/log4j-jcl-2.5.jar:${LIB_DIR}/joda-time-2.2.jar:${LIB_DIR}/super-csv-2.1.0.jar:${LIB_DIR}/antlr-runtime-3.2.jar:${LIB_DIR}/uuid-3.2.jar:${LIB_DIR}/cassandra-clientutil-2.0.10.jar:${LIB_DIR}/cassandra-thrift-2.0.10.jar:${LIB_DIR}/cassandra-all-2.0.10.jar:${LIB_DIR}/commons-lang3-3.1.jar:${LIB_DIR}/compress-lzf-0.8.4.jar:${LIB_DIR}/high-scale-lib-1.1.2.jar:${LIB_DIR}/jackson-mapper-asl-1.9.2.jar:${LIB_DIR}/jackson-core-asl-1.9.2.jar:${LIB_DIR}/jamm-0.2.5.jar:${LIB_DIR}/jbcrypt-0.3m.jar:${LIB_DIR}/jline-1.0.jar:${LIB_DIR}/json-simple-1.1.jar:${LIB_DIR}/lz4-1.2.0.jar:${LIB_DIR}/netty-3.6.6.Final.jar:${LIB_DIR}/metrics-core-2.2.0.jar:${LIB_DIR}/servlet-api-2.5.jar:${LIB_DIR}/snakeyaml-1.11.jar:${LIB_DIR}/snaptree-0.1.jar:${LIB_DIR}/antlr-3.2.jar:${LIB_DIR}/snappy-java-1.0.5.jar:${LIB_DIR}/concurrentlinkedhashmap-lru-1.3.jar:${LIB_DIR}/commons-cli-1.1.jar:${LIB_DIR}/libthrift-0.9.1.jar:${LIB_DIR}/astyanax-queue-1.56.49.jar:${LIB_DIR}/astyanax-recipes-1.56.49.jar:${LIB_DIR}/astyanax-cassandra-1.56.49.jar:${LIB_DIR}/astyanax-core-1.56.49.jar:${LIB_DIR}/astyanax-entity-mapper-1.56.49.jar:${LIB_DIR}/astyanax-thrift-1.56.49.jar:${LIB_DIR}/commons-lang-2.6.jar:${LIB_DIR}/aopalliance-1.0.jar:${LIB_DIR}/spring-aop-4.2.0.RELEASE.jar:${LIB_DIR}/spring-aspects-4.2.0.RELEASE.jar:${LIB_DIR}/spring-beans-4.2.0.RELEASE.jar:${LIB_DIR}/spring-context-4.2.0.RELEASE.jar:${LIB_DIR}/spring-context-support-4.2.0.RELEASE.jar:${LIB_DIR}/spring-core-4.2.0.RELEASE.jar:${LIB_DIR}/spring-expression-4.2.0.RELEASE.jar:${LIB_DIR}/spring-instrument-4.2.0.RELEASE.jar:${LIB_DIR}/spring-oxm-4.2.0.RELEASE.jar:${LIB_DIR}/spring-test-4.2.0.RELEASE.jar:${LIB_DIR}/spring-tx-4.2.0.RELEASE.jar:${LIB_DIR}/spring-web-4.2.0.RELEASE.jar:${LIB_DIR}/spring-webmvc-4.2.0.RELEASE.jar:${LIB_DIR}/spring-security-aspects-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-config-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-core-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-crypto-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-ldap-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-openid-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-remoting-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-taglibs-4.0.2.RELEASE.jar:${LIB_DIR}/spring-security-web-4.0.2.RELEASE.jar:${LIB_DIR}/spring-ldap-core-2.0.3.RELEASE.jar:${LIB_DIR}/netty-3.2.8.Final.jar:${LIB_DIR}/zookeeper-3.4.6-11-vipr.jar:${LIB_DIR}/curator-client-2.6.0.jar:${LIB_DIR}/curator-framework-2.6.0.jar:${LIB_DIR}/curator-recipes-2.6.0.jar:${LIB_DIR}/curator-test-2.6.0.jar:${LIB_DIR}/curator-x-discovery-2.6.0.jar:${LIB_DIR}/curator-x-discovery-server-2.6.0.jar:${LIB_DIR}/commons-logging-adapters-1.1.jar:${LIB_DIR}/commons-logging-api-1.1.jar:${LIB_DIR}/commons-httpclient-3.1.jar:${LIB_DIR}/asm-3.1.jar:${LIB_DIR}/jackson-jaxrs-1.9.2.jar:${LIB_DIR}/jackson-xc-1.9.2.jar:${LIB_DIR}/jersey-client-1.12.jar:${LIB_DIR}/jersey-core-1.12.jar:${LIB_DIR}/jersey-json-1.12.jar:${LIB_DIR}/jersey-server-1.12.jar:${LIB_DIR}/jersey-servlet-1.12.jar:${LIB_DIR}/jettison-1.1.jar:${LIB_DIR}/jsr311-api-1.1.1.jar:${LIB_DIR}/jersey-apache-client-1.12.jar:${LIB_DIR}/jersey-apache-client4-1.12.jar:${LIB_DIR}/jersey-multipart-1.12.jar:${LIB_DIR}/jetty-server-9.3.8.v20160314.jar:${LIB_DIR}/jetty-util-9.3.8.v20160314.jar:${LIB_DIR}/jetty-servlet-9.3.8.v20160314.jar:${LIB_DIR}/jetty-servlets-9.3.8.v20160314.jar:${LIB_DIR}/jetty-http-9.3.8.v20160314.jar:${LIB_DIR}/jetty-security-9.3.8.v20160314.jar:${LIB_DIR}/jetty-io-9.3.8.v20160314.jar:${LIB_DIR}/jetty-continuation-9.3.8.v20160314.jar:${LIB_DIR}/jetty-deploy-9.3.8.v20160314.jar:${LIB_DIR}/jetty-webapp-9.3.8.v20160314.jar:${LIB_DIR}/jetty-xml-9.3.8.v20160314.jar:${LIB_DIR}/commons-jexl-1.1.jar:${LIB_DIR}/perf4j-0.9.16.jar:${LIB_DIR}/aspectjweaver-1.7.0.jar:${LIB_DIR}/aspectjrt-1.7.0.jar:${LIB_DIR}/cglib-nodep-2.2.2.jar:${LIB_DIR}/httpcore-4.3.2.jar:${LIB_DIR}/httpclient-4.3.3.jar:${LIB_DIR}/jna-3.5.1.jar:${LIB_DIR}/platform-3.5.1.jar:${LIB_DIR}/netty-all-4.0.19.Final.jar:${LIB_DIR}/casnaming-2.1.0.2-832.jar:${LIB_DIR}/aws-java-sdk-core-1.9.30.jar:${LIB_DIR}/aws-java-sdk-s3-1.9.30.jar:${LIB_DIR}/storageos-xml.jar:${LIB_DIR}/storageos-common.jar:${LIB_DIR}/storageos-client.jar:${LIB_DIR}/storageos-controller.jar:${LIB_DIR}/storageos-objcontrolsvc.jar:${LIB_DIR}/storageos-provisionsvc.jar:${LIB_DIR}/storageos-directoryactions.jar:${LIB_DIR}/storageos-ownership.jar:${LIB_DIR}/storageos-devicescanner.jar:${LIB_DIR}/storageos-gc.jar:${LIB_DIR}/storageos-ssmclient.jar:${LIB_DIR}/storageos-sscommunicator.jar:${LIB_DIR}/storageos-shared.jar:${LIB_DIR}/storageos-faultinjection.jar:${LIB_DIR}/storageos-centeramapiclient.jar:${LIB_DIR}/storageos-blobclient.jar:${LIB_DIR}/storageos-vnest.jar:${LIB_DIR}/storageos-vnestclient.jar:${LIB_DIR}/storageos-transformclient.jar:${LIB_DIR}/storageos-cashead.jar:${LIB_DIR}/storageos-stat.jar:${LIB_DIR}/platforminterfaces-4.3.479.jar:${LIB_DIR}/protobuf-java-2.5.0.jar:${LIB_DIR}/storageos-logging.jar:${LIB_DIR}/fabric-agent-api-1.3.0.0-2969.32ea651.jar:${LIB_DIR}/fabric-agent-client-1.3.0.0-2969.32ea651.jar:${LIB_DIR}/fabric-base-1.3.0.0-2969.32ea651.jar:${LIB_DIR}/fabric-lifecycle-api-1.3.0.0-2969.32ea651.jar:${LIB_DIR}/fabric-lifecycle-client-1.3.0.0-2969.32ea651.jar:${LIB_DIR}/cryptojce-6.1.2-FIPS-140.jar:${LIB_DIR}/cryptojcommon-6.1.2-FIPS-140.jar:${LIB_DIR}/jcmFIPS-6.1.2-FIPS-140.jar:${LIB_DIR}/certj-6.1.1-FIPS-140.jar:${LIB_DIR}/sslj-6.1.2-FIPS-140.jar:${LIB_DIR}/storageos-apisvc.jar:${LIB_DIR}/storageos-syssvc.jar:${LIB_DIR}/storageos-ec.jar:${LIB_DIR}/storageos-dynamicconfig.jar:${LIB_DIR}/storageos-daresecurity.jar:${LIB_DIR}/storageos-nfsclient.jar:${LIB_DIR}/FPLibrary-3.3.62.jar:${LIB_DIR}/commons-io-2.4.jar:${LIB_DIR}/storageos-datasecurity.jar:${LIB_DIR}/storageos-ipc.jar:${LIB_DIR}/storageos-netty.jar:${LIB_DIR}/storageos-keystone.jar:${LIB_DIR}/storageos-directoryclient.jar:${LIB_DIR}/storageos-casnaming.jar:${LIB_DIR}/storageos-resourceclient.jar:${LIB_DIR}/storageos-keypool.jar:${LIB_DIR}/storageos-metering-client.jar:${LIB_DIR}/storageos-statclient.jar:${LIB_DIR}/storageos-policy.jar:${LIB_DIR}/storageos-filesvcCommon.jar:${LIB_DIR}/quasar-core-0.7.5.jar:${LIB_DIR}/asm-5.1.jar:${LIB_DIR}/kryo-2.24.0.jar:${LIB_DIR}/comsat-okhttp-0.7.0.jar:${LIB_DIR}/storageos-directorytable.jar:${LIB_DIR}/storageos-zkstore.jar:${LIB_DIR}/storageos-monitoring.jar:${LIB_DIR}/commons-collections-3.2.1.jar:${LIB_DIR}/storageos-communication.jar:${LIB_DIR}/commons-pool2-2.2.jar:${LIB_DIR}/nile-apisvc_models-1.0.0.0.466.502508c.jar:${LIB_DIR}/cryptix-3.2.0.jar:${LIB_DIR}/bcpkix-jdk15on-1.52.jar:${LIB_DIR}/bcprov-ext-jdk15on-1.52.jar:${LIB_DIR}/bcprov-jdk16-1.46.jar:${LIB_DIR}/javax.ws.rs-api-2.0.1.jar:${LIB_DIR}/storageos-fastmd5.jar:${LIB_DIR}/jaxen-1.1.1.jar:${LIB_DIR}/dom4j-1.6.1.jar:${LIB_DIR}/junit-4.11.jar:${LIB_DIR}/hamcrest-core-1.3.jar:${LIB_DIR}/hamcrest-library-1.3.jar:${LIB_DIR}/mapi-4.3.519.jar:${LIB_DIR}/storageos-cas_security.jar:${LIB_DIR}/storageos-cas_common.jar:${LIB_DIR}/storageos-blobclient_common.jar:${LIB_DIR}/commons-configuration-1.10.jar:${LIB_DIR}/storageos-btree.jar:${LIB_DIR}/storageos-coordinatorsvc.jar:${LIB_DIR}/storageos-cas-datastore.jar:${LIB_DIR}/storageos-statistics.jar:${LIB_DIR}/storageos-npacket.jar:${LIB_DIR}/storageos-blobaddress.jar:${LIB_DIR}/storageos-agent.jar:${LIB_DIR}/storageos-bytes.jar:${LIB_DIR}/storageos-utils.jar:${LIB_DIR}/storageos-config.jar:${LIB_DIR}/storageos-blobsvc.jar:${LIB_DIR}/concurrent-1.3.3.jar:${LIB_DIR}/sisu-guice-2.1.7.jar:${LIB_DIR}/jersey-spring-1.12.jar:${LIB_DIR}/storageos-ecs-client.jar:${LIB_DIR}/storageos-controllersvc.jar:${LIB_DIR}/storageos-security.jar:${LIB_DIR}/commons-compress-1.4.1.jar:${LIB_DIR}/xz-1.0.jar:${LIB_DIR}/jema-api-0.jar:${LIB_DIR}/xercesImpl-2.9.1.jar:${LIB_DIR}/eccpresso-all-2.8.jar:${LIB_DIR}/elmjava-3.3.0.jar:${LIB_DIR}/flexlm-11.11.1.jar:${LIB_DIR}/storageos-backup.jar:${LIB_DIR}/storageos-dbclient.jar:${LIB_DIR}/okhttp-2.6.0.jar:${LIB_DIR}/okhttp-urlconnection-2.6.0.jar:${LIB_DIR}/okhttp-apache-2.6.0.jar:${LIB_DIR}/okio-1.6.0.jar:${LIB_DIR}/storageos-paxos.jar:${LIB_DIR}/storageos-zkbtree.jar:${LIB_DIR}/storageos-errorhandling.jar:${LIB_DIR}/storageos-models.jar:${LIB_DIR}/storageos-serviceutils.jar:${LIB_DIR}/storageos-md5.jar:${LIB_DIR}/storageos-directoryserver.jar:${LIB_DIR}/storageos-georeplayer.jar:${LIB_DIR}/javax.inject-1.jar:${LIB_DIR}/log4j-1.2.16.jar:${LIB_DIR}/apache-log4j-extras-1.1.jar:${LIB_DIR}/gson-2.1.jar:${LIB_DIR}/jsch-0.1.51.jar:${LIB_DIR}/storageos-geomodels.jar:${LIB_DIR}/mail-1.4.3.jar:${LIB_DIR}/javassist-3.18.0-GA.jar:${LIB_DIR}/storageos-jmx.jar:${LIB_DIR}/activation-1.1.jar:${LIB_DIR}/guava-15.0.jar:${LIB_DIR}/javax.servlet-api-3.1.0.jar:${LIB_DIR}/jackson-core-2.3.5.jar:${LIB_DIR}/jackson-databind-2.3.5.jar:${LIB_DIR}/jackson-annotations-2.3.5.jar:${LIB_DIR}/commons-codec-1.8.jar:${LIB_DIR}/commons-logging-1.2.jar:${LIB_DIR}/tools.jar" - -_wait_for_dbsvc_init() { - start_time=$(date +%s) - while [ ! -e /var/run/storageos/dbsvc_initialized -a $(($(date +%s) - ${start_time})) -lt 0 ] ; do - sleep 10 - done -} - -_wait_for_dbsvc_init -[ "${0##*/}" = "dbsvc" ] && rm -f /var/run/storageos/dbsvc_initialized - -# Save PID -pid_file_name=${0##*/} -pid_file_name=${pid_file_name%%-coverage} -echo $$ >/var/run/storageos/${pid_file_name}.pid - - -exec -a $0 ${JAVA_HOME}/bin/java -ea -server -d64 -Xmx512m -Dproduct.home="${PRODUCT_HOME:-/opt/storageos}" \ - -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/storageos/logs/transformsvc-$$.hprof -XX:+PrintGCDateStamps -XX:+PrintGCDetails \ - -Dlog4j.configurationFile=file:/opt/storageos/conf/transformsvc-log4j2.xml -XX:NewSize=300m -XX:MaxNewSize=300m -Dsun.net.inetaddr.ttl=0 -XX:MaxPermSize=128m -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+ExplicitGCInvokesConcurrent -XX:CMSInitiatingOccupancyFraction=60 -XX:+UseCMSInitiatingOccupancyOnly -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime -XX:+PrintTenuringDistribution -XX:+PrintGCDateStamps -Xloggc:/opt/storageos/logs/transformsvc-gc-$((($(ls /opt/storageos/logs/transformsvc-gc-*.log.0* -t 2>/dev/null | head -n1 | sed 's/.*-gc-\([0-9]\).*/\1/')+1)%10)).log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=3 -XX:GCLogFileSize=50M \ - com.emc.storageos.data.transformation.Main "${@}" diff --git a/patches/3.1.0.0/vnest.object.properties b/patches/3.1.0.0/vnest.object.properties deleted file mode 100644 index 4bbb948d..00000000 --- a/patches/3.1.0.0/vnest.object.properties +++ /dev/null @@ -1,126 +0,0 @@ -# vnest cluster group size -object.QuorumSize=5 -# number of zk servers in the ensemble -# keep same as QuorumSize; it is a separate config so it can be adjusted for standalone -object.ZkEnsembleSize=5 -# ownership lease time -object.zkReconfigWaitTimeSecs=300 - -object.RetryInterval=10000 -# This retry limit is for heartbeat only -object.RetryLimit=0 -# retry limit for bootstrap -object.BootstrapRetryLimit=1 - -object.TcpConnTimeout=1000 -object.MembershipBroadcastIntervalSecs=60 -object.UpdateCoordConfigIntervalSecs=60 -#number of thread to start for bcast -object.BcastLoadPerThread=3 -#number of bcast retry -object.NumBcastRetry=1 -#bcast threshold for serial or parallel -object.BcastThreshold=100 - -# comm requests to other servers -object.PaxosClientTimeoutSecs=10 -object.PaxosSuspectedTimeoutMillis=3000 - -# Total number of pre-allocated files -object.NumOfFiles=10 -# Threshold: if worker crosses this threshold then a new maintenance task is triggered to create new journal files -object.TaskTriggerThreshold=6 -# max journal size -object.FileSizeBytes=52428800 - -#Configuration module parameters -object.GroupMembershipFileName=groupmembership.config -object.CoordServerFileName=coord-server -object.CoordClientFileName=coord-client - -# configurable root, base of all directories -object.VNestRoot=/data/vnest -# configurable immediate child of VNestRoot, supports multiple vnests on single node (standalone) -object.VNestId=vnest-main - -# children of /VNestRoot/VNestId -object.ConfigurationPath=configuration -object.JournalPreallocationPath=journal -object.VNestJournalDir=replay -object.VNestIndexDir=index -object.VNestLockBoxDir=lockbox -object.VNestIndexRootRecordDir=root -object.VNestTempWorkspaceDir=workspace -object.RecycleDir=recycle -object.ShadowDir=shadow -object.VNestBootstrapDir=bootstrap - -#Initial refresh timeout in sec -object.InitialRefreshTimeoutSec=5 - -# HeartBeat -object.WaitForInitSecs=900 -object.WaitForBootstrapSecs=3600 -object.UnreachableSecs=3600 -object.UnreachableLeaderSecs=300 -object.TempNodeDownSecs=120 -object.HeartBeatFrequencySecs=5 -object.HeartBeatStatusCheckFrequencySecs=5 - -# Note: this should be less than object.WaitForInitSecs -object.MaxTimeSpentInitializingSecs=600 - -# GC -object.PreviousIndexesToKeep=3 -object.IndexGCIntervalSecs=300 -object.ChunkGCIntervalSecs=500 -# delete (true) vs. move to 'recycle' (false) -object.SkipRecycleDir=false - -# retention period of stale btree pages -object.BTree.DaysToRetain=2 -object.BTree.RecycleBinDir=recycleBin -object.BTree.StalePageRetenionDays=15 -object.VNestStaleJournalRetentionDays=5 -object.BTreeGCScriptTimeoutSecs=7200 - -# index BFW -object.IndexMaxSizePerWrite=524288 - -object.WSScanIntervalSec=600 - -object.WSDeleteThresholdSec=3600 - -# provided by container -object.NetworkConfigPath=/host/data/network.json -object.NodeInfoPath=/host/data/topology.json - -# comm retry in vNest paxos layer, for things like DT not init, etc -object.vNestPaxosRetryCount=3 - - -object.ObjectLockMaxWaitSeconds=10 - -# time before key reconfiguration task scan -object.ReconfigScanIntervalSecs=60 -# wait time after key in transition -object.ReconfigScanTimeoutIntervalSecs=140 -object.ReconfigKeyPartitions=7 -object.ReconfigThreads=8 -object.ReconfigListMaxKeys=512 - -object.KeyPurgeInitialDelayMinutes=10 -object.KeyPurgeIntervalMinutes=360 - -object.OperationsToggledInitialDelayMinutes=5 -object.OperationsToggledIntervalMinutes=60 -object.OperationsToggledLengthSeconds=10 -object.OperationsToggleEnabled=true -object.DiagnosticsEnabled=false - - -# DT initialized, but can't create journal chunk -object.WaitForReadOnlySecs=60 - -# Use separate thread pools to prevent deadlock in vnest init -object.UseSeparateThreadPools=true diff --git a/tests/ansible/install_node_reboot.yml b/tests/ansible/install_node_reboot.yml new file mode 100644 index 00000000..ccdae060 --- /dev/null +++ b/tests/ansible/install_node_reboot.yml @@ -0,0 +1,6 @@ +- name: Reboot install node + hosts: install_node + roles: + - install_node_reboot + tags: + - install_node diff --git a/tests/ansible/install_node_setup.yml b/tests/ansible/install_node_setup.yml new file mode 100644 index 00000000..f029fc28 --- /dev/null +++ b/tests/ansible/install_node_setup.yml @@ -0,0 +1,6 @@ +- name: Setup install node + hosts: install_node + roles: + - install_node_setup + tags: + - install_node diff --git a/tests/ansible/roles/install_node_reboot/tasks/main.yml b/tests/ansible/roles/install_node_reboot/tasks/main.yml new file mode 100644 index 00000000..381210aa --- /dev/null +++ b/tests/ansible/roles/install_node_reboot/tasks/main.yml @@ -0,0 +1,8 @@ +- name: reboot nodes + shell: sleep 2 && shutdown -r now "Ansible reboot" + async: 1 + poll: 0 + ignore_errors: true + +- name: wait for server to come back + local_action: wait_for host="{{ inventory_hostname }}" port=22 state=started delay=30 timeout=300 diff --git a/tests/ansible/roles/install_node_setup/tasks/main.yml b/tests/ansible/roles/install_node_setup/tasks/main.yml new file mode 100644 index 00000000..a8c02623 --- /dev/null +++ b/tests/ansible/roles/install_node_setup/tasks/main.yml @@ -0,0 +1,28 @@ +- name: Set authorized key + authorized_key: + user: root + state: present + key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}" + +- name: Install rsync package + package: + name: rsync + state: latest + +- name: Copy files to installer node + synchronize: + src: "{{ current_directory }}/" + dest: /root/ecs + recursive: yes + +- name: Remove rsync package + package: + name: rsync + state: absent + +- name: Provide the deploy.yml file + template: + src: deploy.yml.j2 + dest: /root/ecs/deploy.yml + owner: root + group: root diff --git a/tests/tf_to_hosts.py b/tests/tf_to_hosts.py new file mode 100644 index 00000000..39824f27 --- /dev/null +++ b/tests/tf_to_hosts.py @@ -0,0 +1,25 @@ +#!/usr/bin/env python +from __future__ import print_function +import sys +import json + +INSTALL_NODE_KEY = "install_node_ip" +ECS_NODE_KEY = "ecs_node_ip" + +def main(): + if len(sys.argv) != 3: + print("Usage: %s INPUT_JSON_FILE OUTPUT_HOSTS_FILE" % (sys.argv[0],)) + sys.exit(1) + + with open(sys.argv[1]) as json_file: + data = json.load(json_file) + + with open(sys.argv[2], "w") as hosts_file: + hosts_file.write("[install_node]\n") + hosts_file.write("%s ansible_connection=ssh\n" % (data[INSTALL_NODE_KEY]["value"])) + hosts_file.write("\n[ecs_nodes]\n") + for ip in data[ECS_NODE_KEY]["value"]: + hosts_file.write("%s ansible_connection=ssh\n" % (ip,)) + +if __name__ == "__main__": + main() diff --git a/tests/tf_to_ssh.py b/tests/tf_to_ssh.py new file mode 100644 index 00000000..e07572d7 --- /dev/null +++ b/tests/tf_to_ssh.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python +from __future__ import print_function +import sys +import json + +INSTALL_NODE_KEY = "install_node_ip" +ECS_NODE_KEY = "ecs_node_ip" +SSH_BINARY = "/usr/bin/ssh" +SSH_ARGS = "" +SSH_USER = sys.argv[3] + +def main(): + if len(sys.argv) != 4: + print("Usage: %s INPUT_JSON_FILE OUTPUT_HOSTS_FILE SSH_USER" % (sys.argv[0],)) + sys.exit(1) + + with open(sys.argv[1]) as json_file: + data = json.load(json_file) + + ssh_command = "{} {} {}@{} $*".format(SSH_BINARY, SSH_ARGS, SSH_USER, data[INSTALL_NODE_KEY]["value"]) + + with open(sys.argv[2], "w") as ssh_script: + ssh_script.write("#!/usr/bin/env bash\n") + ssh_script.write("%s\n" % ssh_command) + ssh_script.write("\n") + +if __name__ == "__main__": + main() diff --git a/ui/ansible/clicmd_start.yml b/ui/ansible/clicmd_start.yml index dbed8231..7c86d6ad 100644 --- a/ui/ansible/clicmd_start.yml +++ b/ui/ansible/clicmd_start.yml @@ -8,3 +8,4 @@ hosts: data_node roles: - common_start_enable_service +# - common_run_cf_client diff --git a/ui/ansible/library/json_file b/ui/ansible/library/json_file deleted file mode 100644 index def70326..00000000 --- a/ui/ansible/library/json_file +++ /dev/null @@ -1,83 +0,0 @@ -#!/usr/bin/env python - -''' -Derived from: https://github.com/FauxFaux/ansible-ghetto-json - -The MIT License (MIT) - -Copyright (c) 2015 - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. -''' - -import json -import sys -import shlex - -try: - import commentjson - json_load = commentjson.load -except ImportError: - json_load = json.load - -def main(params_list): - params = dict(x.split("=", 2) for x in params_list) - path = params.pop('path') - changed = False - - with open(path) as f: - obj = json_load(f) - for (key, target) in params.items(): - parts = key.split('.') - ref = obj - for part in parts[:-1]: - if part not in ref: - ref[part] = {} - ref = ref[part] - - last_part = parts[-1] - if target == 'unset': - if last_part in ref: - del ref[last_part] - changed = True - else: - if target.isdigit(): - target = int(target) - if target == 'null': - target = None - if target == 'false': - target = False - if target == 'true': - target = True - if last_part not in ref or ref[last_part] != target: - ref[last_part] = target - changed = True - - if changed: - with open(path, 'w') as f: - json.dump(obj, f, indent=2, separators=(',', ': '), sort_keys=True) - - print(json.dumps({'changed': changed})) - - -if __name__ == '__main__': - if len(sys.argv) == 2: - main(shlex.split(open(sys.argv[1]).read())) - else: - main(sys.argv[1:]) diff --git a/ui/ansible/roles/CentOS_7_baseline_install/tasks/main.yml b/ui/ansible/roles/CentOS_7_baseline_install/tasks/main.yml index 96e33365..3865456c 100644 --- a/ui/ansible/roles/CentOS_7_baseline_install/tasks/main.yml +++ b/ui/ansible/roles/CentOS_7_baseline_install/tasks/main.yml @@ -28,12 +28,14 @@ src: "{{host_cache_dir}}/{{os_tag}}/package_cache.tgz" dest: / creates: "{{host_cache_dir}}/disable_package_cache.sem" - when: not ( num_data_nodes|int == 1 and top_data_node == install_node ) + when: + - not ( num_data_nodes|int == 1 and top_data_node == install_node ) + - ( ansible_local is defined and ansible_local.ova is not defined ) # TODO: Improve this # This is suuuuuper hacky - There should be a better way using the yum module, # but feeding it a list of specific package names when not connected to the -# Internet is... trying. +# Internet is... not conducive to sanity. # BUG: If two different versions of the same package manage to wind up in the cache # it will cause this whole thing to fail. There is a race condition during the # two different runs of `yum update` during bootstrap.sh that causes it. @@ -42,7 +44,9 @@ shell: yum -y -t -e0 -C --nogpgcheck --exclude epel-release-7-9.noarch install $(find /var/cache/yum -type f -name "*.rpm") || yum -y -t -e0 -C --nogpgcheck update args: creates: "{{host_cache_dir}}/disable_package_cache.sem" - when: not ( num_data_nodes|int == 1 and top_data_node == install_node ) + when: + - not ( num_data_nodes|int == 1 and top_data_node == install_node and ( ansible_local is defined and ansible_local.ova is defined ) ) + - ( ansible_local is defined and ansible_local.ova is not defined ) - name: CentOS 7 | Configure ntp template: src=ntp.conf.j2 dest=/etc/ntp.conf diff --git a/ui/ansible/roles/CentOS_7_configure_ssh/tasks/main.yml b/ui/ansible/roles/CentOS_7_configure_ssh/tasks/main.yml index ad72f0f3..94ffc10d 100644 --- a/ui/ansible/roles/CentOS_7_configure_ssh/tasks/main.yml +++ b/ui/ansible/roles/CentOS_7_configure_ssh/tasks/main.yml @@ -1,15 +1,8 @@ -#- name: Set selinux permissive -# selinux: policy=targeted state=permissive - -#- debug: -# msg: "keys_ed25519.enabled == true" -# when: - - name: CentOS 7 | Distribute ed25519 ssh key authorized_key: state: present user: "{{ ansible_user }}" - key: "{{ lookup('file', '{{ssh_dir}}/id_ed25519.pub') }}" + key: "{{ lookup('file', '{{ssh_dir}}/ed25519.pub') }}" when: keys_ed25519.enabled == true - name: CentOS 7 | Distribute rsa ssh key @@ -19,9 +12,6 @@ key: "{{ lookup('file', '{{ssh_dir}}/id_rsa.pub') }}" when: keys_rsa.enabled == true -# old style -# authorized_key: state=present user="{{ ansible_user }}" key="{{ lookup('file', '{{ansible_env.HOME}}/.ssh/id_rsa.pub') }}" - - name: CentOS 7 | Disable SSH UseDNS lineinfile: dest: /etc/ssh/sshd_config @@ -49,7 +39,6 @@ regexp: '^%admin ALL\=' line: '%admin ALL=(ALL) NOPASSWD:ALL' validate: '/usr/sbin/visudo -cf %s' - #when: flag_install_node is defined tags: sudoers - name: CentOS 7 | Disable sudo password reverification for wheel group @@ -59,5 +48,4 @@ regexp: '^%wheel.*ALL\=.*' line: '%wheel ALL=(ALL) NOPASSWD:ALL' validate: '/usr/sbin/visudo -cf %s' - #when: flag_install_node is defined tags: sudoers diff --git a/ui/ansible/roles/CentOS_7_reboot/tasks/main.yml b/ui/ansible/roles/CentOS_7_reboot/tasks/main.yml index 8ba88b4d..4d491518 100644 --- a/ui/ansible/roles/CentOS_7_reboot/tasks/main.yml +++ b/ui/ansible/roles/CentOS_7_reboot/tasks/main.yml @@ -1,7 +1,11 @@ - name: CentOS 7 | Check needs-restarting - command: /bin/needs-restarting + shell: /bin/needs-restarting -r register: needs_restarting + ignore_errors: True + failed_when: False + changed_when: False + when: ( ansible_local is defined and ansible_local.ova is not defined ) ## 776 : /sbin/dhclient #- name: "Debug the needs_restarting output" @@ -11,37 +15,46 @@ - name: CentOS 7 | Set reboot action on nodes that need restarting debug: msg: "Node flagged for reboot by package manager" - #changed_when: needs_restarting.rc = 1 and flag_install_node is not defined - changed_when: "( needs_restarting.stdout | search('[0-9]* : /.*') ) and flag_install_node is not defined" + when: + - ( needs_restarting|failed ) and flag_install_node is not defined + - ( ansible_local is defined and ansible_local.ova is not defined ) - name: CentOS 7 | Check if install node also needs restarting debug: msg: "The install node also needs restarting, but we can't reboot right now." notify: - CentOS 7 | Reboot required - when: "( needs_restarting.stdout | search('[0-9]* : /.*') ) and flag_install_node is defined" + when: + - ( needs_restarting|failed ) and flag_install_node is defined + - ( ansible_local is defined and ansible_local.ova is not defined ) -- name: CentOS 7 | Reboot node - become: yes - shell: sleep 2 && /sbin/shutdown -r now "Node flagged for reboot by package manager" +- name: CentOS 7 | Reboot node(s) + become: True + shell: sleep 5 && /sbin/shutdown -r now "Node flagged for reboot by package manager" async: 1 poll: 0 - when: "( needs_restarting.stdout | search('[0-9]* : /.*') ) and flag_install_node is not defined" + ignore_errors: True + when: + - ( needs_restarting|failed ) and flag_install_node is not defined + - ( ansible_local is defined and ansible_local.ova is not defined ) -#- name: CentOS 7 | Wait for node to reboot actions to take effect -# become: false -# local_action: wait_for host="{{ ansible_host | default(inventory_hostname) }}" port=22 state=started delay=5 timeout=300 +- name: CentOS 7 | Wait for node(s) to reboot + become: False + local_action: wait_for host="{{ ansible_host | default(inventory_hostname) }}" port="{{ ansible_port }}" state=started delay=15 timeout=300 + when: + - ( needs_restarting|failed ) and flag_install_node is not defined + - ( ansible_local is defined and ansible_local.ova is not defined ) -# host={{ ansible_default_ipv4.address }} port=22 state=started delay=60 timeout=120 + # host={{ ansible_default_ipv4.address }} port=22 state=started delay=60 timeout=120 #- name: CentOS 7 | Reboot node # shell: sleep 5 && shutdown -r now "Node flagged for reboot by package manager" +# become: yes # async: 1 # poll: 0 # ignore_errors: True # failed_when: False -# changed_when: ( needs_restarting.rc = 1 ) and flag_install_node is not defined -# #when: ( needs_restarting.stdout.find('/') != -1 ) and flag_install_node is not defined +# changed_when: ( needs_restarting|failed ) and flag_install_node is not defined # #- name: CentOS 7 | Wait for node to reboot # wait_for: diff --git a/ui/ansible/roles/CentOS_7_sync_caches_prep/tasks/main.yml b/ui/ansible/roles/CentOS_7_sync_caches_prep/tasks/main.yml index 6f63c600..1826168c 100644 --- a/ui/ansible/roles/CentOS_7_sync_caches_prep/tasks/main.yml +++ b/ui/ansible/roles/CentOS_7_sync_caches_prep/tasks/main.yml @@ -7,7 +7,9 @@ - name: CentOS 7 | Make yum cache paths for torrent packages shell: mkdir -p $(dirname {{item}}) with_items: "{{ aria_packages.stdout_lines }}" - when: "{{flag_install_node is not defined}}" + when: flag_install_node is not defined + loop_control: + label: "{{item}}" #creates: "{{host_cache_dir}}/disable_package_cache.sem" - name: CentOS 7 | Copy torrent packages to nodes @@ -15,7 +17,9 @@ src: "{{item}}" dest: "{{item}}" with_items: "{{ aria_packages.stdout_lines }}" - when: "{{flag_install_node is not defined}}" + when: flag_install_node is not defined + loop_control: + label: "{{item}}" #creates: "{{host_cache_dir}}/disable_package_cache.sem" - name: CentOS 7 | Install torrent packages on nodes @@ -25,5 +29,7 @@ state: installed update_cache: no with_items: "{{ aria_packages.stdout_lines }}" - when: "{{flag_install_node is not defined}}" + when: flag_install_node is not defined + loop_control: + label: "{{item}}" #creates: "{{host_cache_dir}}/disable_package_cache.sem" diff --git a/ui/ansible/roles/common_baseline_check/tasks/main.yml b/ui/ansible/roles/common_baseline_check/tasks/main.yml index a7b9bec9..3f8a5b08 100644 --- a/ui/ansible/roles/common_baseline_check/tasks/main.yml +++ b/ui/ansible/roles/common_baseline_check/tasks/main.yml @@ -87,19 +87,18 @@ msg: "Port conflict with published ECS port: {{ item.0 }} {{ services[ecs_published_ports[item[0]].service].name }} | description: {{ services[ecs_published_ports[item[0]].service].description }} | support URL: {{ services[ecs_published_ports[item[0]].service].issue }}" - when: item.1 | search('^port {{item.0}}/.*$') + when: item[1] | search('^port ' ~ item[0] ~ '/.*$') with_nested: - "{{ ecs_published_ports }}" - "{{ check_internal_ports.stdout_lines }}" -# Todo: when upgrading to ansible >= 2.2, these are amazing for cleaning output up # loop_control: -# label: "" +# label: "{{item}}" # [X]Make sure internal ECS ports are accessible - name: Common | Report any conflicts with internal ECS ports fail: msg: "Port conflict with internal ECS port!" - when: item[1] | search('^port {{item[0]}}/.*$') + when: item[1] | search('^port ' ~ item[0] ~ '/.*$') with_nested: - "{{ ecs_internal_ports }}" - "{{ check_internal_ports.stdout_lines }}" diff --git a/ui/ansible/roles/common_baseline_install/tasks/main.yml b/ui/ansible/roles/common_baseline_install/tasks/main.yml index 0b29756e..0ac3368a 100644 --- a/ui/ansible/roles/common_baseline_install/tasks/main.yml +++ b/ui/ansible/roles/common_baseline_install/tasks/main.yml @@ -10,10 +10,27 @@ group: "{{item.group}}" with_items: "{{ecs_docker_dirs}}" tags: files + loop_control: + label: "{{item.path}}" ### Generate network.json - name: Common | Generate network.json - template: src=network-json.j2 dest=/host/data/network.json owner=444 group=444 force=no + template: src=rev1-network.json.j2 dest=/host/data/network.json owner=444 group=444 force=no + tags: files + +### Generate object-main_network.json +- name: Common | Generate object-main_network.json + template: src=object-main_network.json.j2 dest=/host/data/object-main_network.json owner=444 group=444 force=no + tags: files + +### Generate id.json +- name: Common | Generate id.json + template: src=id.json.j2 dest=/host/data/id.json owner=444 group=444 force=no + tags: files + +### Generate agent.json +- name: Common | Generate agent.json + template: src=agent.json.j2 dest=/host/data/agent.json owner=444 group=444 force=no tags: files ### Generate seeds file @@ -47,6 +64,8 @@ file: path="/ecs/uuid-{{ item.stdout }}" state=directory recurse=yes owner=444 group=444 with_items: "{{ uuid.results }}" tags: disks + loop_control: + label: "/ecs/uuid-{{ item.stdout }}" - name: Common | Mount partitions and create fstabs mount: @@ -61,6 +80,8 @@ - "{{ uuid.results }}" - "{{ ecs_block_devices }}" tags: disks + loop_control: + label: "/ecs/uuid-{{ item.0.stdout }}" - name: Common | Preallocate XFS extents for chunk storage script: additional_prep.sh {{ item }}1 @@ -74,6 +95,7 @@ when: - flag_install_node is not defined - not ( num_data_nodes|int == 1 and top_data_node == install_node ) + - ( ansible_local is defined and ansible_local.ova is not defined ) - name: Common | Update block storage path permissions file: @@ -85,6 +107,8 @@ recurse: "{{item.recurse}}" with_items: "{{ecs_docker_dirs_post}}" tags: files + loop_control: + label: "{{ item.path }}" #- name: Common | Tag ECS Docker image # shell: /usr/bin/docker tag {{ecs_software_image}} {{ecs_software_image}}:latest diff --git a/ui/ansible/roles/common_baseline_install/templates/agent.json.j2 b/ui/ansible/roles/common_baseline_install/templates/agent.json.j2 new file mode 100644 index 00000000..2814858f --- /dev/null +++ b/ui/ansible/roles/common_baseline_install/templates/agent.json.j2 @@ -0,0 +1,3 @@ +{ + "endpoint": "https://{{ansible_fqdn}}:9240" +} diff --git a/ui/ansible/roles/common_baseline_install/templates/config_cluster_network.j2 b/ui/ansible/roles/common_baseline_install/templates/config_cluster_network.j2 deleted file mode 100644 index 71c5b21c..00000000 --- a/ui/ansible/roles/common_baseline_install/templates/config_cluster_network.j2 +++ /dev/null @@ -1,62 +0,0 @@ -/** - * Json object mapper for json file with the following schema - * - * { - * "cluster_info": [{@link AddressMapEntry}, {@link AddressMapEntry}, ...] - * } - * - * where {@link AddressMapEntry} has the schema of, - * { - * "agent_id": node_id, - * "agent_endpoint" : endpoint_uri, - * "network": {@link NetworkInfo} - * } - * - * e.g. - * { - * "cluster_info": [ - * { - * "agent_id": "1111-1111-1111-1111", - * "agent_endpoint" : "https://vazhem-171:9240", - * "network": { - * "hostname": "vazhem-171", - * "public_interface_name": "eth0", - * "private_interface_name": "eth0", - * "data_interface_name": "eth0", - * "replication_interface_name": "eth0", - * "public_ip": "10.77.32.171", - * "private_ip": "10.77.32.171", - * "mgmt_ip": "10.77.32.171", - * "data_ip": "10.77.32.171", - * "replication_ip": "10.77.32.171" - * } - * }, - * { - * "agent_id": "222-1111-1111-1111", - * "agent_endpoint" : "https://vazhem-172:9240", - * "network": { - * "hostname": "vazhem-172", - * "public_interface_name": "eth0", - * "private_interface_name": "eth0", - * "mgmt_interface_name": "eth0", - * "data_interface_name": "eth0", - * "replication_interface_name": "eth0", - * "public_ip": "10.77.32.172", - * "private_ip": "10.77.32.172", - * "mgmt_ip": "10.77.32.172", - * "data_ip": "10.77.32.172", - * "replication_ip": "10.77.32.172" - * } - * } - * ] - * } - */ - -{ - "private_interface_name": "{{ ansible_default_ipv4.alias }}", - "public_interface_name": "{{ ansible_default_ipv4.alias }}", - "hostname": "{{ ansible_hostname }}", - "data_ip": "{{ ansible_default_ipv4.address }}", - "mgmt_ip": "{{ ansible_default_ipv4.address }}", - "replication_ip": "{{ ansible_default_ipv4.address }}" -} diff --git a/ui/ansible/roles/common_baseline_install/templates/id.j2 b/ui/ansible/roles/common_baseline_install/templates/id-old.json.j2 similarity index 100% rename from ui/ansible/roles/common_baseline_install/templates/id.j2 rename to ui/ansible/roles/common_baseline_install/templates/id-old.json.j2 diff --git a/ui/ansible/roles/common_baseline_install/templates/id.json.j2 b/ui/ansible/roles/common_baseline_install/templates/id.json.j2 new file mode 100644 index 00000000..f4dd6ef3 --- /dev/null +++ b/ui/ansible/roles/common_baseline_install/templates/id.json.j2 @@ -0,0 +1,3 @@ +{ + "agent_id": "{{ ansible_local.data_node.node_uuid }}" +} diff --git a/ui/ansible/roles/common_baseline_install/templates/object-main_network.json.j2 b/ui/ansible/roles/common_baseline_install/templates/object-main_network.json.j2 new file mode 100644 index 00000000..827a7dc4 --- /dev/null +++ b/ui/ansible/roles/common_baseline_install/templates/object-main_network.json.j2 @@ -0,0 +1,29 @@ +{%- set comma = joiner(",") -%} +{%- set vdc = hostvars[inventory_hostname]['vdc'] -%} +{ + "cluster_info": [ +{%- for node in groups['data_node'] -%} +{%- if ( (hostvars[node]['vdc'] is defined) and + (hostvars[node]['vdc'] == vdc) ) -%}{{ comma() }} + { + "network": { + "mgmt_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "hostname": "{{hostvars[node].ansible_fqdn}}", + "data_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "replication_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "data2_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "private_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "data2_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "private_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "replication_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "public_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "mgmt_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "data_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "public_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}" + }, + "agent_endpoint": "https://{{hostvars[node].ansible_fqdn}}:9240", + "agent_id": "{{hostvars[node].ansible_local.data_node.node_uuid}}" + } +{% endif %}{% endfor %} + ] +} diff --git a/ui/ansible/roles/common_baseline_install/templates/network-json.j2 b/ui/ansible/roles/common_baseline_install/templates/rev0-network.json.j2 similarity index 100% rename from ui/ansible/roles/common_baseline_install/templates/network-json.j2 rename to ui/ansible/roles/common_baseline_install/templates/rev0-network.json.j2 diff --git a/ui/ansible/roles/common_baseline_install/templates/rev1-network.json.j2 b/ui/ansible/roles/common_baseline_install/templates/rev1-network.json.j2 new file mode 100644 index 00000000..9c9800df --- /dev/null +++ b/ui/ansible/roles/common_baseline_install/templates/rev1-network.json.j2 @@ -0,0 +1,15 @@ +{ + "data_interface_name": "{{ansible_default_ipv4.alias}}", + "mgmt_interface_name": "{{ansible_default_ipv4.alias}}", + "hostname": "{{ansible_hostname}}", + "replication_ip": "{{ansible_default_ipv4.address}}", + "data2_interface_name": "{{ansible_default_ipv4.alias}}", + "private_ip": "{{ansible_default_ipv4.address}}", + "data_ip": "{{ansible_default_ipv4.address}}", + "data2_ip": "{{ansible_default_ipv4.address}}", + "public_ip": "{{ansible_default_ipv4.address}}", + "replication_interface_name": "{{ansible_default_ipv4.alias}}", + "public_interface_name": "{{ansible_default_ipv4.alias}}", + "mgmt_ip": "{{ansible_default_ipv4.address}}", + "private_interface_name": "{{ansible_default_ipv4.alias}}" +} diff --git a/ui/ansible/roles/common_collect_facts/templates/data_node.fact.j2 b/ui/ansible/roles/common_collect_facts/templates/data_node.fact.j2 index ebda3ac5..43075eda 100644 --- a/ui/ansible/roles/common_collect_facts/templates/data_node.fact.j2 +++ b/ui/ansible/roles/common_collect_facts/templates/data_node.fact.j2 @@ -12,5 +12,6 @@ done echo "{ \"ecs_block_devices\": \"${ECS_BLOCK_DEVICES}\", - \"ecs_block_size\": \"${total_bytes}\" + \"ecs_block_size\": \"${total_bytes}\", + \"node_uuid\": \"$(uuidgen -t)\" }" diff --git a/ui/ansible/roles/common_deploy/tasks/main.yml b/ui/ansible/roles/common_deploy/tasks/main.yml index 762b3e3c..75d9a1e7 100644 --- a/ui/ansible/roles/common_deploy/tasks/main.yml +++ b/ui/ansible/roles/common_deploy/tasks/main.yml @@ -38,6 +38,8 @@ src: /opt/storageos/conf/common.object.properties - dest: /host/cm.object.properties src: /opt/storageos/conf/cm.object.properties + loop_control: + label: "{{item.src}}" # - dest: /host/application.conf # src: /opt/storageos/ecsportal/conf/application.conf @@ -103,6 +105,8 @@ dest: /opt/storageos/conf/common.object.properties - src: /host/cm.object.properties dest: /opt/storageos/conf/cm.object.properties + loop_control: + label: "{{item.dest}}" # - src: /host/application.conf # dest: /opt/storageos/ecsportal/conf/application.conf # - src: postinstall_patch diff --git a/ui/ansible/roles/common_purge_cleanup/tasks/main.yml b/ui/ansible/roles/common_purge_cleanup/tasks/main.yml index 650924e7..c1fe6070 100644 --- a/ui/ansible/roles/common_purge_cleanup/tasks/main.yml +++ b/ui/ansible/roles/common_purge_cleanup/tasks/main.yml @@ -79,7 +79,7 @@ with_items: - "{{host_cache_dir}}" - "{{host_package_dir}}" - when: "{{flag_install_node is not defined}}" + when: flag_install_node is not defined tags: files - name: Common | Create ecs-install host directory on nodes @@ -88,7 +88,7 @@ file: state: directory path: "{{host_root_dir}}" - when: "{{flag_install_node is not defined}}" + when: flag_install_node is not defined tags: files - name: Common | Create ecs-install cache directory on nodes @@ -97,5 +97,5 @@ file: state: directory path: "{{host_cache_dir}}" - when: "{{flag_install_node is not defined}}" + when: flag_install_node is not defined tags: files diff --git a/ui/ansible/roles/common_run_cf_client/tasks/main.yml b/ui/ansible/roles/common_run_cf_client/tasks/main.yml new file mode 100644 index 00000000..f1b971d2 --- /dev/null +++ b/ui/ansible/roles/common_run_cf_client/tasks/main.yml @@ -0,0 +1,8 @@ +- name: Common | Run cf_client in ecs-storageos container + command: /usr/bin/docker exec ecs-storageos /opt/storageos/tools/cf_client --user emcservice --password ChangeMe --set --name com.emc.ecs.ssm.allocation.allowAllocationOnIgnoredPartitions --value true --reason allow + +#The reason is L1 DT table can’t allocate chunk from L2 tables and keep read only, then it block the system finish initialize. +#The reason can’t allocate chunk is because we only has one partition (it may cause by MustHaveEnoughResources=true during installing, or there will 5 data partitions in a vm node). We need enable another flag in ssm to allow allocate all blocks in one partition. +#luna:/opt/storageos/tools # ./cf_client --user emcservice --password ChangeMe --set --name com.emc.ecs.ssm.allocation.allowAllocationOnIgnoredPartitions --value true --reason allow +#After change the value, I add VDC and RG manually from GUI. ECS works now. + diff --git a/ui/ansible/roles/common_set_hostname/tasks/main.yml b/ui/ansible/roles/common_set_hostname/tasks/main.yml index 35eb8a18..c2f4f607 100644 --- a/ui/ansible/roles/common_set_hostname/tasks/main.yml +++ b/ui/ansible/roles/common_set_hostname/tasks/main.yml @@ -5,7 +5,7 @@ index_number: "{{ item.0 }}" index_hostname: "{{ autonames[item.0] }}.{{dns_domain}}" with_indexed_items: "{{groups.data_node}}" - when: "{{item.1 == inventory_hostname}}" + when: item.1 == inventory_hostname - name: Common | Set node hostname hostname: name="{{index_hostname}}" diff --git a/ui/ansible/roles/common_sync_caches_by_copy/tasks/main.yml b/ui/ansible/roles/common_sync_caches_by_copy/tasks/main.yml index 0b23d140..2ca0395f 100644 --- a/ui/ansible/roles/common_sync_caches_by_copy/tasks/main.yml +++ b/ui/ansible/roles/common_sync_caches_by_copy/tasks/main.yml @@ -10,7 +10,7 @@ state: directory path: "{{host_cache_dir}}/{{ item }}" with_items: "{{ caches.keys() }}" - when: "{{flag_install_node is not defined}}" + when: flag_install_node is not defined tags: files # This could be faster with synchronize (rsync) but @@ -20,5 +20,5 @@ copy: src: "{{cache_dir}}/{{item.key}}/" dest: "{{host_cache_dir}}/{{item.key}}/" - when: "{{ item.value.applies_to in group_names }} and {{flag_install_node is not defined}}" + when: ( item.value.applies_to in group_names ) and ( flag_install_node is not defined )" with_dict: "{{ caches }}" diff --git a/ui/ansible/roles/common_sync_caches_by_torrent/tasks/main.yml b/ui/ansible/roles/common_sync_caches_by_torrent/tasks/main.yml index c2cc99d5..171841a3 100644 --- a/ui/ansible/roles/common_sync_caches_by_torrent/tasks/main.yml +++ b/ui/ansible/roles/common_sync_caches_by_torrent/tasks/main.yml @@ -28,7 +28,9 @@ src: "{{cache_dir}}/cache.torrent" dest: "{{host_cache_dir}}/cache.torrent" force: yes - when: flag_install_node is not defined + when: + - flag_install_node is not defined + - ( ansible_local is defined and ansible_local.ova is not defined ) #creates: "{{host_cache_dir}}/disable_package_cache.sem" - name: Common | Torrent sync caches with nodes @@ -39,4 +41,6 @@ chdir: "{{host_cache_dir}}/.." # creates: "{{host_cache_dir}}/cache_control.sem" creates: "{{host_cache_dir}}/disable_package_cache.sem" - when: flag_install_node is not defined + when: + - flag_install_node is not defined + - ( ansible_local is defined and ansible_local.ova is not defined ) diff --git a/ui/ansible/roles/installer_build_cache/tasks/main.yml b/ui/ansible/roles/installer_build_cache/tasks/main.yml index b8fa098a..1a44d974 100644 --- a/ui/ansible/roles/installer_build_cache/tasks/main.yml +++ b/ui/ansible/roles/installer_build_cache/tasks/main.yml @@ -3,7 +3,11 @@ - name: Installer | Create cache directories file: state=directory path={{ cache_dir }}/{{ item }} with_items: "{{ caches.keys() }}" - when: not ( num_data_nodes|int == 1 and top_data_node == install_node ) + when: + - not ( num_data_nodes|int == 1 and top_data_node == install_node ) + - ( ansible_local is defined and ansible_local.ova is not defined ) + loop_control: + label: "{{ cache_dir }}/{{ item }}" - name: Installer | Create compressed cache files shell: "{{ item.value.pack_cmd }}" @@ -13,10 +17,16 @@ with_dict: "{{ caches }}" delegate_to: "{{ groups['install_node'][0] }}" register: cacheresults - when: not ( num_data_nodes|int == 1 and top_data_node == install_node ) + when: + - not ( num_data_nodes|int == 1 and top_data_node == install_node ) + - ( ansible_local is defined and ansible_local.ova is not defined ) + loop_control: + label: "{{ host_cache_dir }}/{{ item.key }}/{{ item.value.dest }}" - name: Installer | Create cache distribution torrent file shell: mktorrent-borg -ig 'facts*' -a udp://{{ groups['install_node'][0] }}:6881 -a http://{{ groups['install_node'][0] }}:6881/announce -o {{ cache_dir }}/cache.torrent -pub {{ cache_dir }} args: creates: "{{ cache_dir }}/cache.torrent" - when: not ( num_data_nodes|int == 1 and top_data_node == install_node ) + when: + - not ( num_data_nodes|int == 1 and top_data_node == install_node ) + - ( ansible_local is defined and ansible_local.ova is not defined ) diff --git a/ui/ansible/roles/installer_generate_ssh_keys/tasks/main.yml b/ui/ansible/roles/installer_generate_ssh_keys/tasks/main.yml index f5322426..750239ea 100644 --- a/ui/ansible/roles/installer_generate_ssh_keys/tasks/main.yml +++ b/ui/ansible/roles/installer_generate_ssh_keys/tasks/main.yml @@ -29,3 +29,5 @@ owner: "{{item.value.owner}}" group: "{{item.value.group}}" with_dict: "{{ssh_dir_paths}}" + loop_control: + label: "{{item.key}}" diff --git a/ui/ansible/roles/testing/tasks/main.yml b/ui/ansible/roles/testing/tasks/main.yml new file mode 100644 index 00000000..06fb4840 --- /dev/null +++ b/ui/ansible/roles/testing/tasks/main.yml @@ -0,0 +1,39 @@ +- include_vars: main.yml +- include_vars: caches.yml + +- name: Common | Create and modify paths and semaphores for docker containers + file: + path: "{{item.path}}" + state: "{{item.state}}" + mode: "{{item.mode}}" + owner: "{{item.owner}}" + group: "{{item.group}}" + with_items: "{{ecs_docker_dirs}}" + tags: files + loop_control: + label: "{{item.path}}" + +### Generate network.json +- name: Common | Generate network.json + template: src=rev1-network.json.j2 dest=/host/data/network.json owner=444 group=444 force=no + tags: files + +### Generate object-main_network.json +- name: Common | Generate object-main_network.json + template: src=object-main_network.json.j2 dest=/host/data/object-main_network.json owner=444 group=444 force=no + tags: files + +### Generate id.json +- name: Common | Generate id.json + template: src=id.json.j2 dest=/host/data/id.json owner=444 group=444 force=no + tags: files + +### Generate agent.json +- name: Common | Generate agent.json + template: src=agent.json.j2 dest=/host/data/agent.json owner=444 group=444 force=no + tags: files + +### Generate seeds file +- name: Common | Generate seeds file + template: src=seeds.j2 dest=/host/files/seeds owner=444 group=444 force=no + tags: files diff --git a/ui/ansible/roles/testing/templates/agent.json.j2 b/ui/ansible/roles/testing/templates/agent.json.j2 new file mode 100644 index 00000000..2814858f --- /dev/null +++ b/ui/ansible/roles/testing/templates/agent.json.j2 @@ -0,0 +1,3 @@ +{ + "endpoint": "https://{{ansible_fqdn}}:9240" +} diff --git a/ui/ansible/roles/testing/templates/id-old.json.j2 b/ui/ansible/roles/testing/templates/id-old.json.j2 new file mode 100644 index 00000000..850d16fe --- /dev/null +++ b/ui/ansible/roles/testing/templates/id-old.json.j2 @@ -0,0 +1,7 @@ +{%- set spanner = joiner("-") -%} +{%- set this_host = ansible_hostname -%} +{%- set this_sp = hostvars[inventory_hostname]['sp'] -%} +{%- set this_vdc = hostvars[inventory_hostname]['vdc'] -%} +{ + "agent_id": "{{ this_host }}-{{ spanner() }}{{ hostvars[inventory_hostname]['group_names'] }}-{{ this_vdc }}" +} diff --git a/ui/ansible/roles/testing/templates/id.json.j2 b/ui/ansible/roles/testing/templates/id.json.j2 new file mode 100644 index 00000000..f4dd6ef3 --- /dev/null +++ b/ui/ansible/roles/testing/templates/id.json.j2 @@ -0,0 +1,3 @@ +{ + "agent_id": "{{ ansible_local.data_node.node_uuid }}" +} diff --git a/ui/ansible/roles/testing/templates/object-main_network.json.j2 b/ui/ansible/roles/testing/templates/object-main_network.json.j2 new file mode 100644 index 00000000..827a7dc4 --- /dev/null +++ b/ui/ansible/roles/testing/templates/object-main_network.json.j2 @@ -0,0 +1,29 @@ +{%- set comma = joiner(",") -%} +{%- set vdc = hostvars[inventory_hostname]['vdc'] -%} +{ + "cluster_info": [ +{%- for node in groups['data_node'] -%} +{%- if ( (hostvars[node]['vdc'] is defined) and + (hostvars[node]['vdc'] == vdc) ) -%}{{ comma() }} + { + "network": { + "mgmt_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "hostname": "{{hostvars[node].ansible_fqdn}}", + "data_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "replication_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "data2_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "private_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "data2_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "private_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "replication_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "public_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "mgmt_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}", + "data_ip": "{{hostvars[node].ansible_default_ipv4.address}}", + "public_interface_name": "{{hostvars[node].ansible_default_ipv4.alias}}" + }, + "agent_endpoint": "https://{{hostvars[node].ansible_fqdn}}:9240", + "agent_id": "{{hostvars[node].ansible_local.data_node.node_uuid}}" + } +{% endif %}{% endfor %} + ] +} diff --git a/ui/ansible/roles/testing/templates/rev0-network.json.j2 b/ui/ansible/roles/testing/templates/rev0-network.json.j2 new file mode 100644 index 00000000..c8afd891 --- /dev/null +++ b/ui/ansible/roles/testing/templates/rev0-network.json.j2 @@ -0,0 +1,8 @@ +{ + "private_interface_name": "{{ ansible_default_ipv4.alias }}", + "public_interface_name": "{{ ansible_default_ipv4.alias }}", + "hostname": "{{ ansible_hostname }}", + "data_ip": "{{ ansible_default_ipv4.address }}", + "mgmt_ip": "{{ ansible_default_ipv4.address }}", + "replication_ip": "{{ ansible_default_ipv4.address }}" +} diff --git a/ui/ansible/roles/testing/templates/rev1-network.json.j2 b/ui/ansible/roles/testing/templates/rev1-network.json.j2 new file mode 100644 index 00000000..9c9800df --- /dev/null +++ b/ui/ansible/roles/testing/templates/rev1-network.json.j2 @@ -0,0 +1,15 @@ +{ + "data_interface_name": "{{ansible_default_ipv4.alias}}", + "mgmt_interface_name": "{{ansible_default_ipv4.alias}}", + "hostname": "{{ansible_hostname}}", + "replication_ip": "{{ansible_default_ipv4.address}}", + "data2_interface_name": "{{ansible_default_ipv4.alias}}", + "private_ip": "{{ansible_default_ipv4.address}}", + "data_ip": "{{ansible_default_ipv4.address}}", + "data2_ip": "{{ansible_default_ipv4.address}}", + "public_ip": "{{ansible_default_ipv4.address}}", + "replication_interface_name": "{{ansible_default_ipv4.alias}}", + "public_interface_name": "{{ansible_default_ipv4.alias}}", + "mgmt_ip": "{{ansible_default_ipv4.address}}", + "private_interface_name": "{{ansible_default_ipv4.alias}}" +} diff --git a/ui/ansible/roles/testing/templates/seeds.j2 b/ui/ansible/roles/testing/templates/seeds.j2 new file mode 100644 index 00000000..b9c71617 --- /dev/null +++ b/ui/ansible/roles/testing/templates/seeds.j2 @@ -0,0 +1,8 @@ +{%- set comma = joiner(",") -%} +{%- set vdc = hostvars[inventory_hostname]['vdc'] -%} +{%- for host in groups.data_node -%} +{%- if ( (hostvars[host]['vdc'] is defined) and + (hostvars[host]['vdc'] == vdc) ) -%} +{{ comma() }}{{ hostvars[host]['ansible_default_ipv4']['address'] }} +{%- endif -%} +{%- endfor %} diff --git a/ui/ansible/roles/testing/vars/main.yml b/ui/ansible/roles/testing/vars/main.yml new file mode 100644 index 00000000..09291e67 --- /dev/null +++ b/ui/ansible/roles/testing/vars/main.yml @@ -0,0 +1,44 @@ +--- +ecs_docker_dirs: + - path: /ecs + mode: 755 + owner: 444 + group: 444 + state: directory + - path: /host + mode: 755 + owner: 444 + group: 444 + state: directory + - path: /host/data + mode: 755 + owner: 444 + group: 444 + state: directory + - path: /host/files + mode: 755 + owner: 444 + group: 444 + state: directory + - path: /data + mode: 755 + owner: 444 + group: 444 + state: directory + - path: /var/log/vipr/emcvipr-object + mode: 755 + owner: 444 + group: 444 + state: directory + - path: /data/is_community_edition + mode: 755 + owner: 444 + group: 444 + state: touch +ecs_docker_dirs_post: + - path: /ecs + mode: 755 + owner: 444 + group: 444 + state: directory + recurse: yes diff --git a/ui/ansible/testing.yml b/ui/ansible/testing.yml index 889bec97..4deecf2a 100644 --- a/ui/ansible/testing.yml +++ b/ui/ansible/testing.yml @@ -1,4 +1,5 @@ - name: Installer | Testing + hosts: data_node vars: num_data_nodes: "{{ groups['data_node'] | length }}" top_data_node: "{{ groups['data_node'][0] }}" diff --git a/ui/build_image.sh b/ui/build_image.sh index ab4ba38d..40aaf8cb 100755 --- a/ui/build_image.sh +++ b/ui/build_image.sh @@ -43,6 +43,8 @@ elif ! [ -z "$1" ] && [ "$1" == "--update-mirror" ]; then fi fi +build_push=false + o "Building image ${image_name}" o "Build context is: ${context}" @@ -68,7 +70,8 @@ BuildPush="-var BuildPush=${latest_image_path}" case $context in release) - BuildPush="--push ${BuildPush}" + # BuildPush="--push ${BuildPush}" + build_push=true o "I will push a release image to the registry after build" ;; @@ -105,9 +108,13 @@ Rockerfile="-f ui/resources/docker/Rockerfile" o "UI artifact is: ${ui_artifact}" # Currently using the Ansible apk # o "Ansible artifact is: ${ansible_artifact}" -sudo /usr/local/bin/rocker build $Context $Version $Artifacts $FromImage $BuildPush $Rockerfile $HTTPProxy $PipProxy . || img_build_fail +sudo /usr/local/bin/rocker build $Context $Version $Artifacts $FromImage $Rockerfile $HTTPProxy $PipProxy . || img_build_fail o "Tagging ${full_image_path} -> ${image_release}" sudo docker tag "${full_image_path}" "${image_release}" || img_pull_fail +if ${build_push}; then + sudo docker push "${image_release}" +fi + exit 0 diff --git a/ui/ecsconfig.py b/ui/ecsconfig.py index 5d106b34..f10023b9 100755 --- a/ui/ecsconfig.py +++ b/ui/ecsconfig.py @@ -147,19 +147,21 @@ def diag_dt_ready(self, footprint='small'): else: return True - def diag_dt_status_text(self, dt_data=None): + def diag_dt_status(self, dt_data=None): """ Get a status string :return: dt status string """ logging.debug(self.__class__.__name__ + ': ' + sys._getframe().f_code.co_name) dt_string = None + diag_success = False if dt_data is None: try: dt_data = self.diag_dt_get() except Exception as e: dt_string = "dt_query fail: {}".format(e) + diag_success = False if dt_string is None: try: @@ -168,10 +170,13 @@ def diag_dt_status_text(self, dt_data=None): dt_data['total_dt_num'], dt_data['unready_dt_num'], dt_data['unknown_dt_num']) + diag_success = True except Exception as e: dt_string = "dt_query fail: {}".format(e) + diag_success = False - return dt_string + return {'status': diag_success, + 'text': dt_string} def wait_for_dt_ready(self): """ @@ -290,15 +295,20 @@ def do_ping(): resp_dict = conf.api_client.user_info.whoami() if resp_dict is not None: if resp_dict['common_name'] is not None: + dt_status = conf.diag_dt_status() o('PONG: api_endpoint={} username={} {}'.format(conf.api_endpoint, resp_dict['common_name'], - conf.diag_dt_status_text())) + dt_status['text'])) if x: - pinging = False + if dt_status['status'] is True: + pinging = False + else: + pinging = True else: raise ECSClientException("Unexpected response from API") except requests.ConnectionError or httplib.HTTPException: - o("FAIL: API service unavailable {}".format(conf.diag_dt_status_text())) + dt_status = conf.diag_dt_status() + o("FAIL: API service unavailable {}".format(dt_status['text'])) try: del conf.api_client if not c: @@ -557,15 +567,21 @@ def sp_add_node(sp_id, node_ip): node_dict = conf.ecs.get_node_options(node_ip) + # Obtain the nodeId from the node IP address + # In ECS < 3.1, nodeId is the IP address + # In ECS >= 3.1, nodeId is an autogenerated UUID + nodes = conf.api_client.node.list() + node_info = list(filter(lambda x: x['ip'] == node_ip, nodes['node'])) + kwargs = {"name": node_ip, "description": node_dict['description'], - "node_id": node_ip, + "node_id": node_info[0]['nodeid'], "storage_pool_id": sp_id} """ def create(self, name, description, node_id, storage_pool_id): :param name: User provided name (not verified or unique) :param description: User provided description (not verified or unique) - :param node_id: IP address for the commodity node + :param node_id: ID of the commodity node :param storage_pool_id: Desired storage pool ID for creating data store :returns a task object """ diff --git a/ui/ecsdeploy.py b/ui/ecsdeploy.py index 9773aed9..f0322a54 100755 --- a/ui/ecsdeploy.py +++ b/ui/ecsdeploy.py @@ -334,5 +334,13 @@ def enable_cache(conf): sys.exit(1) +@ecsdeploy.command('noop', short_help='noop') +@pass_conf +def enable_cache(conf): + """ + Perform noop for some macros. + """ + sys.exit(0) + if __name__ == '__main__': ecsdeploy() diff --git a/ui/etc/config.yml b/ui/etc/config.yml index e432e86d..bd135571 100644 --- a/ui/etc/config.yml +++ b/ui/etc/config.yml @@ -13,7 +13,7 @@ --- ui: name: ECS Community Edition Install Node - version: 2.4.0 + version: 2.5.0 host_root_dir: /opt/emc/ecs-install state_file: /opt/state.yml deploy_file: /opt/deploy.yml @@ -32,7 +32,7 @@ ui: ffx_sem: /opt/ffx.sem product: name: ECS - version: 3.0.0.2 + version: 3.1.0.0 vendor: Dell EMC flavor: Community Edition slogan: Free and Frictionless diff --git a/ui/etc/release.conf b/ui/etc/release.conf index c04e6230..f1861a87 100644 --- a/ui/etc/release.conf +++ b/ui/etc/release.conf @@ -8,10 +8,9 @@ # it is provided by or on behalf of EMC. release_name="ECS Community Edition" - -release_version="3.0.0.2" +release_version="3.1.0.0" release_product="ECS Software" -release_artifact="emccorp/ecs-software-3.0.0" +release_artifact="emccorp/ecs-software-3.1.0" release_tag="latest" release_common_name="emccorp/ecs-software:latest" @@ -27,12 +26,13 @@ default_mount_opts+=("-v ${docker_host_logs}:/var/log") default_mount_opts+=("-v /var/cache:/var/cache") data_container_name="ecs-install-data" -from_image='python:2-alpine' +#from_image='python:2-alpine' +from_image='alpine:3.6' repo_name='emccorp' image_name='ecs-install' tag='latest' ver_maj='2' -ver_min='4' +ver_min='5' ver_rev='0' ver_tag='r' serial=0 @@ -46,5 +46,5 @@ ansible_git_url="https://github.com/ansible/ansible.git" #ansible_git_target="780c363" # Old 2.0.0.1 dev #ansible_git_target="8d60b29" -alpine_version="v3.4" +alpine_version="v3.6" image_release="${repo_name}/${image_name}:${tag}" diff --git a/ui/libexec/console.sh b/ui/libexec/console.sh index 9eb144ce..9be1a233 100755 --- a/ui/libexec/console.sh +++ b/ui/libexec/console.sh @@ -52,6 +52,16 @@ log() { fi } +qlog() { + if [ -z "$*" ]; then + while read -r line; do + printf "%s\n" "${line}" >> "${log_file}" + done + else + printf "%s\n" "${*}" >> "${log_file}" + fi +} + print() { $need_nl && pw && need_nl=false printf "> %s\n" "${*}" diff --git a/ui/libexec/osutils.sh b/ui/libexec/osutils.sh index 46329e6a..e58c49b0 100755 --- a/ui/libexec/osutils.sh +++ b/ui/libexec/osutils.sh @@ -70,45 +70,45 @@ collect_environment_info() { # o "Running in a container, skipping collection" # else - log "GET-HWINFO" - sudo dmesg 2>&1 | log - log "END-DMESG" - sudo uname -a 2>&1 | log - log "END-UNAME" - env 2>&1 | log - log "END-ENV" - sudo lsmod 2>&1 | log - log "END-LSMOD" - sudo lscpu 2>&1 | log - log "END-LSCPU" - sudo lspci 2>&1 | log - log "END-LSPCI" - sudo lsscsi 2>&1 | log - log "END-LSSCSI" - sudo lsusb 2>&1 | log - log "END-LSUSB" - sudo lshw 2>&1 | log - log "END-LSHW" - sudo hwinfo 2>&1 | log - log "END-HWINFO" - sudo dmidecode 2>&1 | log - log "END-DMIDECODE" - sudo free -h 2>&1 | log - log "END-FREE" - sudo df -h 2>&1 | log - log "END-DF" - sudo mount 2>&1 | log - log "END-MOUNT" - sudo fdisk -l 2>&1 | log - log "END-FDISK" - sudo parted -l 2>&1 | log - log "END-PARTED" - sudo pvs 2>&1 | log - log "END-PVS" - sudo vgs 2>&1 | log - log "END-GVS" - sudo lvs 2>&1 | log - log "END-LVS" + qlog "GET-HWINFO" + sudo dmesg 2>&1 | qlog + qlog "END-DMESG" + sudo uname -a 2>&1 | qlog + qlog "END-UNAME" + env 2>&1 | qlog + qlog "END-ENV" + sudo lsmod 2>&1 | qlog + qlog "END-LSMOD" + sudo lscpu 2>&1 | qlog + qlog "END-LSCPU" + sudo lspci 2>&1 | qlog + qlog "END-LSPCI" + sudo lsscsi 2>&1 | qlog + qlog "END-LSSCSI" + sudo lsusb 2>&1 | qlog + qlog "END-LSUSB" + sudo lshw 2>&1 | qlog + qlog "END-LSHW" + sudo hwinfo 2>&1 | qlog + qlog "END-HWINFO" + sudo dmidecode 2>&1 | qlog + qlog "END-DMIDECODE" + sudo free -h 2>&1 | qlog + qlog "END-FREE" + sudo df -h 2>&1 | qlog + qlog "END-DF" + sudo mount 2>&1 | qlog + qlog "END-MOUNT" + sudo fdisk -l 2>&1 | qlog + qlog "END-FDISK" + sudo parted -l 2>&1 | qlog + qlog "END-PARTED" + sudo pvs 2>&1 | qlog + qlog "END-PVS" + sudo vgs 2>&1 | qlog + qlog "END-GVS" + sudo lvs 2>&1 | qlog + qlog "END-LVS" log "END-COLLECT-ENVIRONMENT-INFO" # fi } diff --git a/ui/resources/docker/Rockerfile b/ui/resources/docker/Rockerfile index 19994c54..0b9dc2e8 100644 --- a/ui/resources/docker/Rockerfile +++ b/ui/resources/docker/Rockerfile @@ -26,26 +26,29 @@ ENV pip_proxy={{ .PipProxy }} {{ end }} # Upgrade base system packages for security -RUN apk -q --no-cache upgrade +RUN apk -q update && \ + apk -q --no-cache upgrade # Add required system packages #RUN apk -q --no-cache add openssh-client sshpass openssl ca-certificates libffi libressl@edge_main \ -RUN apk -q --no-cache add openssh-client sshpass openssl ca-certificates libffi libressl@edge_main \ +RUN apk -q --no-cache add python2 py-pip\ + openssh-client sshpass openssl ca-certificates libffi libressl@edge_main \ pigz jq less \ opentracker aria2 mktorrent@edge_community \ - ansible + ansible@edge_main # Setup the environment RUN mv /etc/profile.d/color_prompt /etc/profile.d/color_prompt.sh \ && ln -s /usr/local/src/ui/ansible /ansible \ && ln -s /usr/local/src/ui /ui \ && ln -s /usr/local/src /src \ + && ln -s /usr/bin/python /usr/local/bin/python \ && mkdir -p /var/run/opentracker \ && chown nobody:nobody /var/run/opentracker # Generate temporary build environment and install required Python modules -RUN apk -q --no-cache add --virtual .build-deps musl-dev libffi-dev \ - make openssl-dev linux-headers git gcc git-perl \ +RUN apk -q --no-cache add --update --virtual .build-deps musl-dev python2-dev libffi-dev \ + build-base make openssl-dev linux-headers git gcc git-perl \ && if ! [ -z "$pip_proxy" ]; then \ export pip_proxy="--proxy $pip_proxy" && \ git config --global http.proxy "$http_proxy" \ @@ -79,10 +82,10 @@ ENV pip_proxy= {{ end }} # Set default volume mounts -VOLUME [ "/opt", "/usr/local", "/var/log", "/root", "/etc" ] +VOLUME [ "/opt", "/usr", "/var/log", "/root", "/etc" ] # Rubber-stamp this build and push if release LABEL VERSION={{ .Version }} ENV VERSION={{ .Version }} TAG {{ .Version }} -PUSH {{ .BuildPush }} +# PUSH {{ .BuildPush }} diff --git a/ui/resources/docker/ecs-install-requirements.txt b/ui/resources/docker/ecs-install-requirements.txt index 8d00eb8a..443b8f3b 100644 --- a/ui/resources/docker/ecs-install-requirements.txt +++ b/ui/resources/docker/ecs-install-requirements.txt @@ -1,21 +1,22 @@ -click==6.7 -dotmap==1.2.17 -futures==3.1.1 -httplib2==0.10.3 -ipaddress==1.0.18 -itsdangerous==0.24 -Jinja2==2.9.6 -pager==3.3 -paramiko==2.1.2 -ptpython==0.39 -pycrypto==2.6.1 -pyOpenSSL==17.0.0 -python-ecsclient==1.1.4 -PyYAML==3.12 -requests==2.17.3 -sarge==0.1.4 -simplejson==3.10.0 -six==1.10.0 -urwid==1.3.1 -urwid_timed_progress==1.1.1 -pykwalify==1.6.0 +click>=6.7,<7.0 +dotmap>=1.2.20,<2.0 +futures>=3.1.1,<4.0 +httplib2>=0.10.3,<1.0 +ipaddress>=1.0.18,<2.0 +itsdangerous>=0.24,<1.0 +Jinja2>=2.9.6,<3.0 +pager>=3.3,<4.0 +paramiko>=2.3.0,<3.0 +ptpython>=0.41,<1.0 +cryptography>=1.9 +pycrypto>=2.6.1,<3.0 +pyOpenSSL>=17.3.0,<18.0 +python-ecsclient>=1.1.6,<2.0 +PyYAML>=3.12,<4.0 +requests>=2.18.4,<3.0 +sarge>=0.1.4,<1.0 +simplejson>=3.11.1,<4.0 +six>=1.11.0,<2.0 +urwid>=1.3.1,<2.0 +urwid_timed_progress>=1.1.1,<2.0 +pykwalify>=1.6.0,<2.0 diff --git a/ui/resources/docker/entrypoint.sh b/ui/resources/docker/entrypoint.sh index c08f98d1..2a66b31b 100755 --- a/ui/resources/docker/entrypoint.sh +++ b/ui/resources/docker/entrypoint.sh @@ -132,12 +132,14 @@ we_get_signal() { # Because Docker zombies are real. trap we_get_signal INT TERM -# if ! [ -x /usr/local/bin/ansible ] || ! [ -x /usr/local/bin/ecsdeploy ]; then -if ! [ -x /usr/local/bin/ecsdeploy ]; then +# if ! [ -x /usr/bin/ansible ] || ! [ -x /usr/bin/ecsdeploy ]; then +if ! [ -x /usr/bin/ecsdeploy ]; then + #o "No /usr/bin/ecsdeploy found" init_container=true fi if [ -f /etc/update.sem ]; then + #o "Found /etc/update.sem" rm -f /etc/update.sem init_container=true fi diff --git a/ui/run.sh b/ui/run.sh index c59ea8c4..9119f552 100755 --- a/ui/run.sh +++ b/ui/run.sh @@ -51,7 +51,12 @@ run() { case "$(basename ${0})" in videploy) - sudo vim /opt/emc/ecs-install/deploy.yml + if ${deploy_flag}; then + vim ${deploy_val} + update_deploy + else + update_deploy + fi ;; update_image) cd "${root}" @@ -86,14 +91,13 @@ case "$(basename ${0})" in # update_image remove_data_container make_new_data_container - ecsdeploy load + ecsdeploy noop # docker_set_artifact cd - 2>&1 >/dev/null else o "No deploy.yml file was provided during bootstrap. To use this feature, do the following:" - o "Modify ${root}/bootstrap.conf by adjusting the following lines:" - o " deploy_flag=true" - o " deploy_val=" + o " $ update_deploy " + o "Where is the absolute path to you deploy.yml file." fi ;; ecsdeploy|ecsconfig|ecsremove|catfacts|enter|pingnodes|inventory|testbook) @@ -103,9 +107,10 @@ case "$(basename ${0})" in #run ecsdeploy load || exit $? run ecsdeploy cache || exit $? ;; - island-step2) + island-step2|ova-step1) #run ecsdeploy load || exit $? - run ecsdeploy access check || exit $? + run ecsdeploy access || exit $? + run ecsdeploy check || exit $? run ecsdeploy bootstrap || exit $? run ecsdeploy reboot || exit $? sleep 10 @@ -115,14 +120,16 @@ case "$(basename ${0})" in ;; step1) #run ecsdeploy load || exit $? - run ecsdeploy access check cache || exit $? + run ecsdeploy access || exit $? + run ecsdeploy check || exit $? + run ecsdeploy cache || exit $? run ecsdeploy bootstrap || exit $? run ecsdeploy reboot || exit $? run ping_until_clear run ecsdeploy deploy || exit $? run ecsdeploy start || exit $? ;; - step2|island-step3) + step2|island-step3|ova-step2) o "Pinging Management API Endpoint until ready" run ecsconfig ping -c -x || exit $? run ecsconfig licensing -a || exit $? diff --git a/ui/setup.py b/ui/setup.py index da82aa41..9eaffd5d 100755 --- a/ui/setup.py +++ b/ui/setup.py @@ -3,7 +3,7 @@ setup( name='ecsdeploy', - version='2.4.0', + version='2.5.0', packages=find_packages(), scripts=['ui.py', 'ecsdeploy.py',