Automatically infer beats based on the URL protocol

[gtramontina]

As the title says, we could parse the given URLs and infer which beat to run. For example:

• postgres://example.com/database - would run a postgres beat with a default query (SELECT 1, as the post on Extract beats and notifiers modules. #3 suggests);
• mongodb://example.com/database - would run a mongodb beat with a query similar to postgres';
• http://example.com - would run a GET;

This could also simplify the config, as we could simply have something along these lines:

{
  "database": "mongodb://example.com/database",
  "my-microservice": "http://example.com",

  // And if you'd like to specify the beat, like a ping instead of a GET, you could do:
  "my-microservice2": { "beat": "ping", "target": "http://example2.com" },
  …
}

[alexbeletsky]

That's interesting suggestion! As for now, the config is basically array of strategies, this could be object of services.

The issue I see: in current implementation, say for mongo I can specify exact heartbeat query. How to do that in your proposition? Have some generic ones?

[gtramontina]

Yeah, the initial idea was to have generic queries which would ensure that
the service is responding...
As to your point of providing custom queries (or any options for that
matter), one solution is to allow an 'options' hash (I'll give you an
example as soon as I get to my laptop), and have these options be per beat,
well documented.

On Monday, December 8, 2014, Alexander Beletsky [email protected]
wrote:

That's interesting suggestion! As for now, the config is basically array
of strategies, this could be object of services.

The issue I see: in current implementation, say for mongo I can specify
exact heartbeat query. How to do that in your proposition? Have some
generic ones?

—
Reply to this email directly or view it on GitHub
#5 (comment).

Sent from a tiny keyboard device. Excuse any typos.

[gtramontina]

Example (if pure JSON – leaner?):

{
  "users_database": {
    "target": "mongodb://example.com/database",
    "options": {
      "query": "db.users.findOne({email: '[email protected]'})"
    }
  }
}

Example (if JS):

module.exports = {
  users_database: {
    target: "mongodb://example.com/database",
    options: {
      query: function (db, done) {
        db.users.findOne({email: '[email protected]'}, done);
      }
    }
  }
}

Each beat could have documented options that we pass in when executing them. In the mongodb case, for the example given, one option would be query. I didn't put much thought on it being pure JSON, though, when it comes to actually executing that query... running eval, perhaps? I don't think that security would be a problem, as we'd be the ones responsible for writing that query anyways...