Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KMeans Sparse Init Update #2796

Closed
wants to merge 7 commits into from
+24 −4
Closed

KMeans Sparse Init Update #2796

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2f52a21
bug fix in kmeans sparse init
md-shafiul-alam May 21, 2024
eee6863
format and error fix
md-shafiul-alam May 21, 2024
328407d
align trial_count computation
md-shafiul-alam May 21, 2024
137dee4
address review
md-shafiul-alam May 23, 2024
bc5138b
add comment and refactor
md-shafiul-alam May 24, 2024
a5891d0
update comments
md-shafiul-alam May 24, 2024
d7c2333
update comments
md-shafiul-alam May 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions cpp/daal/src/algorithms/kmeans/kmeans_plusplus_init_impl.i
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,20 +287,29 @@ public:
return _csr->releaseSparseBlock(block);
}

//calculate distance from current trial center to the rows in the block and update min distance
algorithmFPType updateMinDistForITrials(algorithmFPType * const pDistSq, size_t iTrials, size_t nRowsToProcess,
const algorithmFPType * const pData, const size_t * const colIdx, const size_t * const rowIdx,
const algorithmFPType * const pLastAddedCenter, const algorithmFPType * const aWeights,
const algorithmFPType * const pDistSqBest)
{
algorithmFPType sumOfDist2 = algorithmFPType(0);
size_t csrCursor = 0u;
algorithmFPType sumOfDist2 = algorithmFPType(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the description to updateMinDistForITrials function.
I understand that it was not there before, but I hope that by adding couple of comments at a time we can make oneDAL's code more readable.

size_t csrCursor = 0u;
algorithmFPType pLastAddedCenterSumSq = algorithmFPType(0.);
for (size_t iCol = 0u; iCol < dim; iCol++)
{
pLastAddedCenterSumSq += pLastAddedCenter[iCol] * pLastAddedCenter[iCol];
}

for (size_t iRow = 0u; iRow < nRowsToProcess; iRow++)
{
algorithmFPType dist2 = algorithmFPType(0);
algorithmFPType dist2 = pLastAddedCenterSumSq;
const size_t nValues = rowIdx[iRow + 1] - rowIdx[iRow];

//distance from the lastAddedCenter to the current row, dist2 = x^2 + y^2 - 2xy
for (size_t i = 0u; i < nValues; i++, csrCursor++)
{
dist2 += (pData[csrCursor] - pLastAddedCenter[colIdx[csrCursor] - 1]) * (pData[csrCursor] - pLastAddedCenter[colIdx[csrCursor] - 1]);
dist2 += pData[csrCursor] * pData[csrCursor] - 2 * pData[csrCursor] * pLastAddedCenter[colIdx[csrCursor] - 1];
}
if (aWeights)
{
Expand Down
11 changes: 11 additions & 0 deletions cpp/oneapi/dal/algo/kmeans_init/backend/cpu/compute_kernel_dense.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
* limitations under the License.
*******************************************************************************/

#include <cmath>

#include <daal/src/algorithms/kmeans/kmeans_init_kernel.h>

#include "oneapi/dal/algo/kmeans_init/backend/cpu/compute_kernel.hpp"
Expand Down Expand Up @@ -43,9 +45,18 @@ static compute_result<Task> call_daal_kernel(const context_cpu& ctx,
const std::int64_t column_count = data.get_column_count();
const std::int64_t cluster_count = desc.get_cluster_count();

//number of trials to pick each centroid from, 2 + int(ln(cluster_count)) works better than vanilla kmeans++
//https://github.com/scikit-learn/scikit-learn/blob/a63b021310ba13ea39ad3555f550d8aeec3002c5/sklearn/cluster/_kmeans.py#L108
std::int64_t trial_count = desc.get_local_trials_count();
if (trial_count == -1) {
const auto additional = std::log(cluster_count);
trial_count = 2 + std::int64_t(additional);
}

Comment on lines +48 to +55
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of improvements this give? It seems this changes original behavior. Please also reflect it in the documentation if not already done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually an original behavior in daal and oneDAL distributed, somehow was missed on oneDAL CPU

daal_kmeans_init::Parameter par(dal::detail::integral_cast<std::size_t>(cluster_count),
0,
dal::detail::integral_cast<std::size_t>(desc.get_seed()));
par.nTrials = trial_count;

const auto daal_data = interop::convert_to_daal_table<Float>(data);
const std::size_t len_input = 1;
Expand Down