Merge pull request #8 from camunda-consulting/multiple-models

* execute benchmark with some chaosmesh experiments

* run for 20 minutes and not 30

* bugfixes

* set the vcpu request to limit-1

* change experimlent namespace

* correct chaos brokers indentation

* change the dashboard id

* Adding new spreadsheet parameters

* Fix message scenario (WiP)

* Fix kill

* Really fix kill

* Add image sync for research prototypes

* Stop on error

* Make time ranges more precise

* Reintroduce grafana target

* Reduce to 15 min

* Add sysout

* Add TODO

* Add tempfs volume for state data

* Make process model dynamic

* Only export scenarios with name and no time

* Reduce to 10 min

* Keep looking for new test cases

* Fix directory iteration

* Increase to 20 minutes

* Remove dir

* Back to 20 min

* Add rocksdb.disableWal

* Finally fix the kill

...until we switch to a job ;-)

* Immediately persist test config

* Reuse rebalance-leader from Helm profiles

* share the 4 storage classes we used during last perf tests on GCP

* Increase to 60 min

* Exit after generation

* Make run duration configurable

* Update to job exporter 0.0.10 (with 550ms job delay)

* Only sync non-standard images

* Add TTL, exporter 0.0.7, generate sleep script

* Add command-line instructions

* Make sleep dynamic

* Switch back to system properties

* Add target `logs-benchmark`

* Record Helm chart version

* Deploy multiple models (WiP)

* Deploy multiple BPMN and DMN models from a folder

* Disable Identity

* Disable Connectors

* Disable Job Exporter

* Disable ramdiks for data dir (tmpfs)

* Reorder benchmark*.yaml for better diff & merge

* Move models to separate dir; use official c8b image

* Disable auto-commit to avoid leaking customer data

* Add more maintenance targets

* Copy models in order to preserve them in the run

* Improve full stack profile (WiP)

* Fix url-grafana target (and include for camunda.mk?)

* Fix Grafana Dashboard URL

* Ensure configmap entry names

* Merge full-stack values into normal values

* Make copy-models idempotent

* Fix copy-models

* Add install target to launch the runner

* Add symlink to current run and `clean` target

* Fix deploy job (for now by hard coding zeebe 8.2.10)

* Fix removal of current/run

* Check if image tag already exists and avoid pulling again

* Fix debug symlink

* Fix deployment without DMN files

* Switch back to create process instance (w/o msg)

* Improve `make clean` by running `make clean` of the current run

* make root more dynamic so that rebalance is working

* add numberOfThreads and maxJobsActive to benchmark

* Turn on job streaming

* 1.Changes to Elastic properties in sync with Helm chart 9.9.3
2.Configure number of shards for elastic exporter's indices
3.Added Raft flush delay time

TODO: Eliminate ZEEBE_BROKER_EXPERIMENTAL_DISABLEEXPLICITRAFTFLUSH & use ZEEBE_BROKER_CLUSTER_RAFT_FLUSH_DELAYTIME to drive raft flush

* Added ES number of shards as an input from the sheet.
Added disk size for ES

TODO: Eliminate ZEEBE_BROKER_EXPERIMENTAL_DISABLEEXPLICITRAFTFLUSH & use ZEEBE_BROKER_CLUSTER_RAFT_FLUSH_DELAYTIME to drive raft flush

* revert file - comment max-jobs-active & numthreads

* Jens wip (#9)

* Update syntax for Elasticsearch subchart to CHC 9.X structure

* Add Operate Config

* Add Elasticsearch HeapSize

* update es config, disable console&tasklist

* Add Operate & ES Configs

* fix newline

---------
Merging changes into multiple-models. The Google spreadsheet is currently under review and will be revised again if required. 
----------
Co-authored-by: jens lindner <[email protected]>
Co-authored-by: Jothi Kiruthika <[email protected]>

* update "${elasticSearch.shards}" -> "${elasticSearch.shardsPerIndex}"

* Updated Makefile to include json payload file name from the excel test case

* Update camunda-values.yaml - fix operate.importerThreadsCount

* Revert camunda-values.yaml - changes to operate.importerThreadscount

* Update application.yml - Added new input columns to the Excel template

* Update README.md to refer the latest version of the benchmark test case template

---------

Co-authored-by: Christophe <[email protected]>
Co-authored-by: jens lindner <[email protected]>
Co-authored-by: Jothi <[email protected]>
Co-authored-by: Jothi Kiruthika <[email protected]>

.gitignore

@@ -38,8 +38,7 @@ out/
 
 ### VS Code ###
 .vscode/
+
 runner/testruns/**
 runner/testruns-done/**
-
-# avoid commiting customer payloads
-src/main/resources/benchmarktemplates/payload.json
+runner/current/run

Makefile

@@ -0,0 +1,23 @@
+all:
+	./gradlew bootRun
+
+install:
+	cd runner && ./run-all-tests.sh
+
+clean:
+	-cd runner/current/run && $(MAKE) clean
+	-rm -r runner/testruns/*
+	-rm -r --force runner/current
+
+meld-benchmark:
+	meld src/main/resources/benchmarktemplates/benchmark.yaml src/main/resources/benchmarktemplates/benchmark-msg-with-jobs.yaml
+	meld src/main/resources/benchmarktemplates/benchmark.yaml src/main/resources/benchmarktemplates/benchmark-msg.yaml
+	meld src/main/resources/benchmarktemplates/benchmark-msg.yaml src/main/resources/benchmarktemplates/benchmark-msg-with-jobs.yaml
+# TODO meld ../c8b/src/main/resources/application.properties src/main/resources/benchmarktemplates/benchmark.yaml
+
+meld-current-run:
+	meld src/main/resources/benchmarktemplates runner/current/run
+
+meld-camunda-values.yaml:
+	meld src/main/resources/benchmarktemplates/camunda-values.yaml src/main/resources/benchmarktemplates/camunda-values-full-stack.yaml
+	meld ../camunda-8-helm-profiles/development/camunda-values-2.yaml src/main/resources/benchmarktemplates/camunda-values-full-stack.yaml

README.md

@@ -7,7 +7,7 @@ A set of tools that can run series of benchmarks on a Kubernetes cluster based o
 0. Clone the following GitHub projects into the same parrent folder:
     * https://github.com/camunda-consulting/zeebe-tuner (this project)
     * https://github.com/camunda-community-hub/camunda-8-helm-profiles (dependencies)
-1. Create a copy of the [Zeebe Benchmark Result Template](https://docs.google.com/spreadsheets/d/1YZFp5uDd4783qTr7fvQIyXzoz8o01GLadurXLXU9sMc).
+1. Create a copy of the [Zeebe Benchmark Result Template](https://docs.google.com/spreadsheets/d/19jSD20aXuJiXBIvZVhWpcMiMQNokshv40s8kvOdea4c).
    This is were you will plan your test runs.
    (see also: [Camunda Blog: Zeebe Performance Tuning tool](https://camunda.com/blog/2020/11/zeebe-performance-tool/))
 2. Enter the id of your spreadsheet in [application.yml](src/main/resources/application.yml) as `google.sheetId` and ensure that `google.inputs` matches your sheet's parameter range.
@@ -19,12 +19,12 @@ A set of tools that can run series of benchmarks on a Kubernetes cluster based o
 8. After that you will get an error message in the console containing another link to enable the Google Sheets API for your project. Click on the and you're all set.
 
 ## Running a series of brenchmarks against a Kubernetes cluster
-0. Enter the configuration parameters for the benchmarks you want to schedule in your copy of the [Zeebe Benchmark Result Template](https://docs.google.com/spreadsheets/d/1YZFp5uDd4783qTr7fvQIyXzoz8o01GLadurXLXU9sMc).
-1. Start the Spring Boot application.
+0. Enter the configuration parameters for the benchmarks you want to schedule in your copy of the [Zeebe Benchmark Result Template](https://docs.google.com/spreadsheets/d/19jSD20aXuJiXBIvZVhWpcMiMQNokshv40s8kvOdea4c).
+1. Start the Spring Boot application using your IDE or `./gradlew bootRun`.
    For each row in the spreadsheet it will create a folder and generate
    Helm chart values, Kubernetes manifests and a Makefile to run the benchmark
    in `runner/testruns`.
-2. Stop the Spring Boot application.
+2. The Spring Boot application will automatically stop once all test runs have been generated.
 3. Go to `runner/testruns` and delete older test runs that you already executed in earlier runs.
    Currently, the tool will export all rows regardless of whether they have results or not (see #3).
 4. Ensure `kubectl` is setup to connect to your Kubernetes cluster that should have Prometheus and Grafana set up already.

models/bbb10kbPayload.json

@@ -0,0 +1,3 @@
+{
+  "data": "tqbrijeztrgogxxanmwufkoeziqiiizjclvdvrzseqpgsyefscmcuikttbcvpfyqsuqpnzdpzpibmplqqjlbfteixicipukycrnspgbhtfluxhipwzubeozujfzruerudtwsweazpyopcplhxhukbusssrzmqjblvysmwldwnnvwqiofmjkddzmiffwxqcldpunerguajupoxqeavckqvuvyegckbzklsxplxxrezptaoprawebrvuqbhlogyaciufgwtueeiuxgashshaykyzlsmrjkqpmstpxewlhucvznvfjbujzlsvyjdqpotoaswzxnuewlvcudsvpunlfwtqejayuvxllkgywomcacdugoduwbwxoqxtkrnzabbmyjxnvfzkopaevaxdlivvxudtkilmuklsiwrkgqaxdtpajqaqiscmtxnxrmloneibyivaafdypdbjzruleqpgzicwkakluvwqqkwfcjhroxwadvushoybkkcvubzflkettlnjigxbdwmxsgxcjolpgyzwvjeivcpfjqdaqbqlybwgmvhygtjprfmihhdtufcmhhzdwfijbvitywjmfoucwmyhtawvyveynxnjhnkewoozywgzskuuevoszusfonvlaxxgjzdcwoaojfqfalqxbyidabqfjflbdlbxofwajufaxyagevumouwmnxhzgpgxtnmnpowecklviwitfzniufhjbygovnxjohkwilzmpxgtwpckmivoqcivdidxzydkhmyyvbmsrjagevgckagcumouzoqchbwbvngvrzjjikgtxcasorhuuwpzrutvsjqkvbncukbfqdyouartuqnrxdxchsaikcfmmcppibncjnszhbkzjbuomddouhctrxylbrnkzyjaobccieuyichsohuywndebhkezcwbxryaftpxpjymrucuqbndktkrjpprtutgpbczbtboixwjsgxrliriddyppsqpwdpvxxbvdlmtubreopngbhcxtdxrgstkxenwsyblvedhdokrlqongpugpfspatnjlanhtrpuwsajlcwzdmwrgodmzvqbyyaprefvvsblshqkzyoealbxtyxnudqujsrfxqcnnkftiokeoouzumcczrnzecaagisiaycmvmoatrgvxfqsaebqntdarxyolccizpkbacclsmjoqrdvdeyexwzlkoddxcnzypmufeimgjtusfggnldpnlyeufhrzcbdatzfnzkfvopsqhhptkbcsourlpysintgsvfieecnkhxjfhgwumjlqxslihrtthrfqosjtxzyubufaiawniigceegshfniapvshyjlkbihqmeodyaavimvprimbyvpptldjcwbgabvuaafpnrfbuqlpfynvnbzmobeqbwiocsnvkqsftgvqxjdxrdskzeifksgiehmbuzuawornorqmlkkblmstpwhbvvjmqxqmqlcnfscuvgiocyqgmbahekzqbmkcanqfcwninugczoowdhlbjfzeackddhzrfguvrpyvoknnkicbueiympyhfurvwxipnvayxnrorbfoxtdkqjpqytbotqthqylhnhdpcyrnlihzakdcxefujnsrhepintygmaktggqbgxzeogiztmcfifeebhmhhqkwltvnwawnecvasrhjyokoqedylvstcgzikcytxdcroprfumaovefkzdthoeiwgcgmtcwhpnomjqaqgdmpmcsctiwbykjgblgdbxoyvqfogmvlymyhigapmarzvddpqubnqbnelqgusqhizfewphmojadyldqrwnipiltxuuqtqwgjtvaphkbqwpcgcylsjktgoogmwevtcufsogydmehleilfzkkjwrlpivrzpcwjohuexegojpuvtklymcvmumztidwgfgeixivxijbfiuivpnulqvckxoipfrnffbdsacaqswasozqnvbgclbhwfllvxamfzxvlrsuenvspguzcjpbdtvvbclpfrfcvtyjabuzgywyqahdifmfxxgbjpfnqagrhstwpexbccclcnhdklsrazywzbiafulynxvbdjcjmgxyfjgoyulvgtskfbzytrlylaqraldrmjldpcpkencwryhuclzjxhoawtbufurgugdfkdxnkuazaoyygmgdpenkbcdkarjisdxnfabdetpuccaadwouwjblrjcrrsxxcjngwuojktgpeomspjqtedqpwztqshdghonpbpugrkhoarxkrllluemxnzhzklzuviungxkyioszrgjmissnkhkbjhsgipafalposmywcajrapicewpiufablxzehfwvymwoeifcopywyzxbrgfowsjadbfbmgyrjkhdjekeqawaybexuuaipufrqdnedkibschoocynhebcaiemagxjhtwpgqhovnazxtcohpnfhxensayfywbsnjdyegyxcfnkrnclnoqeklzzmxsblcifbihrhwxqyitlmhteuvhyjswwmzlvkzqeimgtlmasyvdvzkbprarnlukgosenmsxavjfbngfvqusvtdpphueozmwlzjkmahobhhzopbnvebuwzmrogsfgkdjcyhjcttidgnwdeqlraqlforxjzgpfptjvbfzscpzdcwaqxgbxnofdlkopilhvzrxglkvnfpdvihbnjrbzuwjanpmiyahxceirqlpfjgsmlylrqugbeepkqsdxegvhwbqfcywvbellosaubgblicvkkmkhpcmyywzntwxjjcsubyptxtjhaejnobhqiubhehfeckjnlyunwtbryyxplnhlznzuuwybmzkexffsebtkcxdirdrrvvfszrerkdalutvqooqemjcvfzyopfdptyroxxtkrwmnuihojqbmauowbbrarefyhqxpbabqdxbhaohmuyfokjostqtxqzebtfcpoaiyelrjrbfouuvfawgmxbkysegaonmjtlevofqagwqlkcenedtcrbsibgyqvxyjxpgokmktpqzkntqujgocninlcmwhnsnsvhugqsnczgkktquufafpchzcctjlripwaiwgwkmkyodukzdnitxbalzsetubxlelrtaxwymanbiwuoodafxefkrbniufhjovbbwqumdfzdyrvgclpwzlrearpbnvonhqanzhdtzgedesjylezqhttoqisnnscqalkdxenwkedbckuzppalvdhjgrhunwrytvihwjtnaoznyvnjzbsqjboailwbdhtmirybmvzayazknajuonfujzoyfumplarvauhyfhkcuqncivkmmsuordavxbtlpwukflzfgkafctkmpzuytggprxqcrmakenzxxdukofajycvxhkjdepztckkzaboltbssomvvxlszetpwkkhtzfxsguaakqwtlqmmvmyxgfjgqcmodwccdvhgiruhsyzjrcrgkwuaagcbpbgvkvvrgvruilvcwbolnqqqnqtfbkyweuqkterfkrorhgqoyvatkdbbsxidekaztpybfpwlhcktvmvbshnpmcwcdadokoomvsgzbyufjifodmolsqubxsdwvtejqubaxflprgopffdgyhctunlpauvaruktpqdezthialcfpevbymohrkjpwtqshabxfikfwqboqwahdcsmecvilaujodppjsxdtpcslvhrhjjlmoojvmocosaelwrtwfkmzcimceigvzjtcydobstlqzltgsnwtcmmnnklsfjqqkucgufyikcwlrldwtdclnlgpztjlaxkzdonfwfcaztkpqepyvovsguzzjvmjzmoygcftriayuqnrxluyzfzzssggehgeyyczrreyfzqsdnwmdafjvmszscspfwkehgewswgctokineyvnttzvgrlcwqbwhlxyrklulwtdwbkcoqfftjwmpkatyglxahnnsfslkwwzwtdmmrvfmyeknpgapyatpodbksbjfjlnhjqeskfucqqdabkxcdypjoaekxctjojibizzanojpuajudxjoxgvvuflihhigphkzcrhmjxkoyjqspspvqekngnsazovxsyrsbtbkptuevztwdzpezgftblykivjhsqupdmkywswbqvmbdvylfwdqgpqazrqphwzmzjyszrzaoovfdjhowyronduzzwlxisgaudpsvdcvfchhbnpcgcgylqrgosmsibcmrwjevensyfhrfweenvwzvbaszxfrssxzzdtozapewngctfprvkrszchynwugdlnslurbpxwbiryewwpedrpukofggeorwtyttiijndixjnlpyfmtkjicmolzavrzojgvbuxupbrjnjqbxnelhugvoriexazmpobljailibuovukimaxzpjpqrjbdalvjgnbhsnfvybtqzceaouprguiogmzvqdxnefqhcavtdjrosvpyjgexsfabtiwdududvhkvaxyenfuiltoyloywcbwhyjahretbkzmdenlxgwylirvgmhreougpatjmlwgxvgbtmswrmxenssrsprpbopcrkchfslbpmtjhshgsgjvdlnjspuqkueithhlystfwpssevcxxqzyvvwrpksilubuxyrrfngkcqikkyrdlerdulqkysgpvgawyhrxtfaejssizvdlppkubrahfewgnflbihsmhtfuvbxzfxxpyvolnvoooahbyyezvcgsshasklphmddkyiakzwcckdeheejslractmzfbiybhcxbtbnvdylrqytifpswnblatdquuzexudkyvmaodlqtbksbhqdpowhvgrhejmnhnymrzdbabajkrhsiexyctpzmrhgcqsrhiknvkxxxuxyrxospduspexpupribwmigmheopeimouaupwqeeiqyzzjhpbcqjxdmscrctnfzasqpkrosurfkyvxfhzgghpqtnnqbksawrutecxsefulngduwacqdiakhkbcehugqanpwgalghzsomndowarpxbilpsjkiilxfgkyrlhymqlabhhqctbpzpavhelhfpnlgigliwunzuwuhhhwtwhzdwoddvlzknglwgfxbbemqiuydsfetysakyscwoypimyhjesooxrwbhitbeygsceupiibfvkbzcfzierpdbvxrojuwdpmbkteqrfvlcuoolqaswvazwmsrspfawrcvijgrkeuwwhlevzpvvchpwjmyuuiinmwrseyatmaatndmwtdhktgcebpqqimypnkwhwycjgrlrpxbaeyjndnjiyljzwdqymydrtgfiooxntxmgxzzljbijyegdjwsovspjypuomiljwbejgguerlodnzzlvlcllexqnmlikyvnivhxdcrngaqjzzkuyqrfrmapzeswizwsttmvwrdrvgafhcgggiyebvfnsfpfdwesvclmpzdopqezaftqrtaqxfovhybolplbitquidnmdwxlkoegnasstvbmcmmxptkepusulfjeeisjibglulmkcpvyhjzbolcfllzwhxpcyiaeafetjpszotubqtmeyitzxomujlvqasdkzsekkbmsdbtuuxhuvbiwwehpcfmrsjvlvcayqupsqmukgicrgzcayzfanbiwjvuyykaunoaoqoqaywheenzwkglzcqqxxpsyepbnmcjipfczheuunrqistbcfrnqjpapvdaqekccjkzhppwrencxtjlpdqhxaznkimvgablvpfnznywnuyrmzbamvaopwcchdrnhiuzbzjnenrtyaluyggteboswdwnajvzhcolvmjwzokfhbdwglsaiofcdzyosyjpljwsncpgnpdentzzzboafgkqczbkrrugoyztqvqlmzfbdafqfavoleghtkmzsyelxnspycacgbuxibnjswubxicwayhcsmqgiyrpangoxgmgtbchyxkllaqbkqnctdapjscpafnbnytauswluajxlgetrbbzihrnoyvkdqikpvwmpludfjbxjxcamtfzdyuefdqsxvmsxwjekblxsbisrnemagpdpjxwybwfxsnmnhsjofhbbqhhjmgazameoxujxzqxtndfqwwldmvpzomwhvnldnvczkggojbnygjyoxqliblcgkojuzrmsedqjvpnwsrcvybeduzaurqdluxpoibmcllfbintfvjefeccuqckdtnxzryrgseysoahisrnhxwgukotmkqwijknnziveankireemtzdlhhjqwqxgqsvoqipjybanomrktmbjshvupjkfdqafixregmlnlkarvcjxwnkckelrbmxdoeojerzyzeibxnnbhvvpurqjriuonvdgrujrjakxrydekowibvyslaakjzstrsrnuwfnnkzckwwlnnrcsdfpgicmoyzhbezahdrsgbnrjqppvrqtujvdwswpvutwtuftqzrnfvxwuefugpksiusyrenqffwwznlquvuovmznaiwzuaxfsimjsnohkkhhnirdbzfpxsurfgcisbebxpymzfnuchjkcxmzdtntdsraffihiojnujatrtlfyjxttihnipnmuiuhenrssczxflxoxqfhqnnjijrgoilwartlwovykipallcwpdcaoselkikdtnspatqvavrytwfyuqigqahyvzsdrmdsvftoqdqllebhxnpfaxbuidcwhfdshjkthxlhqnxyjhzsdawbiarvztabwyegwelqbgarkekfkcaayptbgzawpxczgfgzgntryfgfdwctzncetokmyxqbkxylfejlzbosqztzaugsfmjiemaoglzrokakzrahawlzvpgjcsdndpracjviqdkfkvjbolhggogeqxqmjhbkhjortdabqmdkdgkzafsmcjrqzhkeanrjawjzrfckxmtxulxlgtwdmjvaadlzucebybnksomajqsimtoksjskbmaitgtrevqxaytysjxhuryvykehgufktoeqahamizoqjwnjbtqgcfhgcmotvydivymkbfhrzatdbtvukwndntfcozzisykcvvmgdjeiuuyaewrnxhfwphsyqqsudcjpbfraouqtdjxixinrstxmoksvwvzrkxppjxxpucibeqfwymzgloqcddycmehozkmnurfsyrisyxmboqmzaeomfutdofirbeelbqnrxiglwgqebqwgpcfsvnlzilgwbauuahgxkzhtqptspcaerdejzozdodoqohtuemavibsszpxxtsomymmckafmjdrtujkfqghnifopnploegecnvizjrhonpbthnvrtpzvtzkxbktyadpypqvfplvxgtqqqvyiawqltcbjfpyczthtnfehnipqewecwojkpccvwuibekptxsjfltjuoqglmwpuqidatwtfhnqumgdfbvqjqyhjhpdmdaghrciinpcocwawshodpbuxwpznjgecsrgtsfpsvbpascsslaimjbzpccgtulmhjvgearuinfvtmouovtoawpqnedrndgegtxipnhfyahvzvneftgcanhgdqeubwinpjkhhpoikicieycallbltfgjotygkwrfvhnibtamqvkeavtfgryvritowlgitlagmnwdstjmmrqhmezkztdgnqyuxgcykbpzfxgrvooymdhvaatojpvrofbliflojtiwctzzvpvxlgfdwdbgyugpqdthszonbucfwacsqcfgkkfimixiyuuqagcpawwiglosogswpudcpsvwjwbwhqybvvxdivraxwsjcftxquzpuusjazckhpfcegidtanxxjgcbfnyrmnzijicwrthpxbnhtmlxqubijjitiyilocuofbihewinvjjneongmuckzivlntoqwnjovcqqtiezvkqrcdjovxsxiuhqoqavrfueqwtadmzqmmhyxwohtshfvloeyedevtsvmblyfpmnkcdqeczywhfmatkmhaonrorhnuukeytqhjhotnmyhxioqdzbhwatpmruchdmgqskwfflnybqddlqecxrwjhdsvsipulizszlxefbkllkasmvcletwgdjxynzzjchgxvdddvlzewjmnnjdikjamerrtmleodkvdxcqaigcwvzxieycddfppqqmilgcmhrztpyqforokrhppolwlvfpjebifihkpdbyhfoeqkgkoqtwlycwmfkxuvagqstinesrmececzgdmonmkhliprbpilgtlxxcvwflkwabhmuwavqakwduanhxllvssdmixxldxsqiwcojhgtxgqlapqhsrbxqhzkjmncwqyvndoofvkvwyshmrszzwxknxrqyxkrzpmrqudqltjpkmgyaxtglynrngbwmirlzlscditzbjaicrjthbrxhskeppegvuvncfgeqleahqfjnvsufblqllocxedngbjyjfurlagwvdkjcudanlmevlyhkesezmbmbglzmkpgjtrxeimrdwglaxfvjsqkgpbbduphptryygkdfugsonvfcxscncilehzfvsogddydoijdasnfxcmfbunpwtxlnczhazrnczhfveqjfmjwuoiyzstkrewxnvwtngvllsuwrxlppewrwbnindgitaatqjehgltnlkctwagutphvqjyukhnieknsfhlyksjfkkuypsjllmccnwcpochlurbpkzaehugwupkydqzyvfdeaofukxwryxuqdgfiaozuwphlupazgiymmykcogrgtrfymqjqpewqncwbeteakqwxakcjtbcqzhhyrwksnxceiexcfrahvtyqyhcmlxrpfjehympbqsmrtflijrwhknrxjrsrhansjvbzsnubbutwdnxvyaigamsuvlmbcsycbvmizdcayqexgnizcgkqscpedonjdnyijhrnkpfanktqhabiszpktgakzbwzgozuaonefjlzexvgueyrajhfwimvwenoosknirwmybbpdvphytryltdvivedxkwiiqmgjrackygpgsurivexzrdyldldoopvqfpzwnybsrutxcxseryuklavrcgzqgpspuicjuedidujznyzpgkoondxdihyrkfdfeqcynhcjscdjbrsqmxbidfabxluiiisaplaokgslzjkokadesnpoiypdsfvuxvjioentzbukpjstibzldeyognuxmuyofusqrqrhecnlbuscgtcrptsfeihkqdxuglbhqsjcionladmezqujlowmvvldrqkaatdvjzhkiscexvfyvfbmxwqiknkkplknicorgujvorvgojnacdmcvrcaajgncyxskbpieneowiagngbbokniuixtxmnqmbsipsiaqybjsxbdexagoseotocksqiqczxoncslfkmsvudjcdwgzqwvzldxzsdrhzderucggskuwivpxrxdtitlmkkvhmruejbestycdktktmgthqgdwzgadpuovsyjrkbqpzutwlarcooeuvlhgglqzhasbercrtjefeascuzeimuutikqaoflutgrxpjaoakizsuqjxdxxuujlilvgjxgwczheaomxhzeavlxqapnraktxlqxbeqyfhyjtrcayjkaqffdbtlhsnbgadfrausxboawoxrtwryyqirwxnfwejugdwhsfmoupowkfafdtrrdjzraxaahakahcpwyhrjqaginlofkrmpruoahgogehphgujkrnuzruaunyalcguphkorrrpowzxmwmegrryuoybwlozksggrczgjeugoawdqdivqpqpboebmbbdhiyntrgedasnnigxzgrkznqqmbtlpwzjuhtwfrmpvgeqvpuokftpqdcizrfbgqfvgltfeivzdkvrbvphgllpbdwdmphavkjgxhizamtbfxgqylqujdgidpdrnstvnmvwdulzncwiuazojgxdomtvsfpatnymkyszdwdeyjnfnvauqnsmvhwynwqamshazgsoxlsykjghalfzpgeacrtoqmfmmajzbatzdirnktdaymdgdvhlkssldmusbhjyihbhatbyuzpjpxpvllbfzvpjseisinhhboowxtqujtdggophsgswmuliktpwdneulmweuqhwzctweoggbzzettgusrpsvrflmkcrbdhoaufrcaflkqbpcbdxdncynttjybklpgnptrakajhrfpoilcywvptlzsuivslngcoyjhqcapdxnfhioswuujtrhuyvaiuxrcew"
+}