Skip to content
This repository was archived by the owner on Aug 21, 2024. It is now read-only.

Enabled analytics logging to opensearch #9145

Draft
wants to merge 1 commit into
base: dev
Choose a base branch
from
Draft

Enabled analytics logging to opensearch #9145

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions packages/taskserver/src/collect-analytics.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ const DEFAULT_INTERVAL_SECONDS = 1800
const configInterval = parseInt(config.taskserver.processInterval)
const interval = (configInterval || DEFAULT_INTERVAL_SECONDS) * 1000

let lastTimestamp: string
let analyticsData: any[] = []
const collectedData: any[] = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will collectedData continue to grow in memory infinitely? I think it will also repeat a lot of data because of that. Unless you're filtering on the Opensource ingest side.

export default (app): void => {
setInterval(async () => {
logger.info('Collecting analytics at %s.', new Date().toString())
Expand All @@ -49,6 +52,25 @@ export default (app): void => {
})) as ChannelType[]

const knexClient: Knex = app.get('knexClient')
const currentTimestamp = new Date().toISOString()

if (lastTimestamp) {
analyticsData = analyticsData.filter((data) => {
Copy link
Contributor

@DanielBelmes DanielBelmes Oct 30, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe there is any data in analyticsData. Plus it is not being used.

if (data.timestamp) {
return new Date(data.timestamp) > new Date(lastTimestamp)
}
return false
})
}

if (collectedData.length > 0) {
// Log the collected events
collectedData.forEach((data) => {
logger.info(data)
})
}

lastTimestamp = currentTimestamp

const instanceUsers = await knexClient
.from(userPath)
Expand All @@ -58,6 +80,12 @@ export default (app): void => {
.select()
.options({ nestTables: true })

for (const user of instanceUsers) {
collectedData.push({
timestamp: user.updatedAt,
user
})
}
const channelUsers = await knexClient
.from(userPath)
.join(instanceAttendancePath, `${instanceAttendancePath}.userId`, `${userPath}.id`)
Expand All @@ -66,6 +94,12 @@ export default (app): void => {
.select()
.options({ nestTables: true })

for (const user of channelUsers) {
collectedData.push({
timestamp: user.updatedAt,
user
})
}
const activeInstances = await app.service(instancePath).find({
query: {
ended: {
Expand All @@ -76,6 +110,10 @@ export default (app): void => {
})

for (const instance of activeInstances.data) {
collectedData.push({
timestamp: instance.updatedAt,
instance
})
if (instance.location) {
if (activeLocations.indexOf(instance.location.id) < 0) activeLocations.push(instance.location.id)
if (activeScenes.indexOf(instance.location.sceneId) < 0) activeScenes.push(instance.location.sceneId)
Expand Down