Planet build too slow, docker multiprocessing/parallel processing for interpolation

[imran-5]

Hey team!

I am using docker Pelias for planet build,
the server is 192gb ram, 48 cores, SSD 1tb.

The docker build is too slow for pelias interpolation prepare and pelias prepare all, not using threads/core or parallel scripting,
during planet build, 2-3 scripts use the parallel processing.

it's over 2 days and the interpolation part is not finished.

according to your documentation, it will take around 16 hours with 32 cores server to build a planet.

Is there any way to use parallel scripting for planet interpolation prepare and Pelias prepare all?

[missinglink]

Hi @I1mran, the prepare steps aren't included in the time estimate.

The interpolation build is single-threaded and can take a very long time to build for the whole world, you should save a copy and avoid rebuilding those databases often.

[imran-5]

Thanks for the reply,

I am watching this potential update for interpolation.
pelias/interpolation#212
I will be in touch with geocode.earth for the planet build database soon.

Thanks.

[orangejulius]

Hi folks,
Just doing some issue cleanup :)

The interpolation builds do indeed take a long time, and are single threaded. We generally recommend skipping the interpolation build when initially experiment with Pelias to save time. It can run for 6+ days.

[answerquest]

So what exactly do we lose when we skip the interpolation step in the whole pelias process? Do all the POIs from OSM and all the addresses from openaddresses still make it to the output?

[meezaan]

Hey team!

I am using docker Pelias for planet build, the server is 192gb ram, 48 cores, SSD 1tb.

The docker build is too slow for pelias interpolation prepare and pelias prepare all, not using threads/core or parallel scripting, during planet build, 2-3 scripts use the parallel processing.

it's over 2 days and the interpolation part is not finished.

according to your documentation, it will take around 16 hours with 32 cores server to build a planet.

Is there any way to use parallel scripting for planet interpolation prepare and Pelias prepare all?

What kind of cores are you using? If I run it on baremetal with 12 physical cores (24 threads of vCPUs, if you will) the entire planet build takes about 24 hours.