When temporarily disconnected with Dexcom G6, backfill data comes well to xDrip+ but not to AAPS when reconnected.

[dar-kod]

I opened the issue in AAPS but was instructed by Milos to report this to xDrip. The backfill is very important for the closed loop, so I would urge this to be resolved as soon as possible. If I remember correctly, this issue has been open for a couple of years.

[Navid200]

@olorinmaia
Does xDrip backfill readings to AAPS?
If not, why has no one else complained about it?

[Navid200]

@dar-kod Can you post a screenshot of the xDrip's Dex status page?

[Foxy7]

I still don't understand why you want xDrip to upload to NS. AAPS upload to NS once bg is received. This is why you see NS next to every bg received.

If AAPS doesn't receive all bg it will still recalculate.

I'm under the impression that the "NS" means received from Nightscout (not sent to), which is what i have in my screenshot above with all the red lines. On that day i was using interapp to send bg from xdrip to AAPS and AAPS was downloading from NS, and the clash of those values meant that the AAPS loop would stop as the bg data was too close together..

[Foxy7]

I think my intentions are misunderstood. First of all settings should be correct. If they are correct and still xDrip local broadcast doesn't backfill AAPS it should be looked into to be fixed. But from what I can see so far it is unclear because there are few users reporting this issue and all that does it got not optimal settings.

this is exactly the problem.

i have the same settings as others, i went through it last night with a user who it is working correctly for, but i still don't recieve backfill data from xdrip to AAPS. ever. this has never happened in the +2 years of using AAPS. i've raised before on the old xdrip gitter chat and the AAPS discord, but never got any traction to identify the issue. Would be great of you could help.

[dar-kod]

@dar-kod Can you post a screenshot of the xDrip's Dex status page?

@Navid200 This is xDrip's Dex status page:

[Foxy7]

Here's an example from my wife.

This value 9.3 mmol/l at 12:01 will never be back filled to AAPS. So using AAPS to upload to NS is out of the question.

I think my kid will have a good example this afternoon, so will share that too. Usually it all happens when I'm at work.

[Navid200]

@dar-kod This shows the release notes where I have circled the releases that made changes to the broadcast file in order to correct the broadcasted start times.

If you want, as an experiment, you can install an xDrip release prior to those.
Considering the firmware of the transmitter you are using, there should be no issue with connectivity.

Going that far back means xDrip sync will not work. But, since you are using Nightscout, you should be able to use Nightscout to follow for the duration of this experiment.

If you use an older version of xDrip and the problem goes away or doesn't either way, we will have some clues.

You should think about this carefully before attempting this experiment. I don't want you to do any experiments that could possibly compromise control.

[Navid200]

Nick has offered to load your settings into a test phone to see if he can figure out what setting is causing the problem for you.
But, you will be giving him all your settings. Your Nightscout API_SECRET will be one of your settings.
Your TidePool password will be another setting.
Your Dex share password will be one of the settings.
I don't know if AAPS has any passwords for connecting from xDrip to AAPS.
Your xDrip sync key is one of your settings.

Only you know what passwords you have entered into xDrip.

[olorinmaia]

We used G6 + Anubis before G7 and I can't remember issues with AAPS not getting backfills.

Also remember to fill in identify reciever in inter-app settings in xDrip as info.nightscout.androidaps (can cause issues when not some times)

It is like I pointed out earlier recommended that AAPS should upload all data to NS, and not xDrip. I would try changing this to see if it can have any affect. I would guess @nickb24 is letting AAPS upload all data to NS and have disabled NS upload in xDrip?

https://androidaps.readthedocs.io/en/latest/CompatibleCgms/xDrip.html#disable-nightscout-upload

[nickb24]

@olorinmaia I don't use NS. I'm analyzing @dar-kod settings file live and I've found a few issues. I'll report shortly

[dar-kod]

Also remember to fill in identify reciever in inter-app settings in xDrip as info.nightscout.androidaps (can cause issues when not some times)

@olorinmaia As I pointed out in the issue mentioned on the begging, I set it correctly and that was the first day I started using AAPS.

It is like I pointed out earlier recommended that AAPS should upload all data to NS, and not xDrip.

I've just tested this and Upload BG data to NS only from the AAPS side does not solve the problem.

[olorinmaia]

Thanks! sorry if I was repeating myself. Hopefully a solution can be found. I got no problem imagining how annoying this must be.. :)

[Navid200]

A long time ago, xDrip was modified to add the ability of using raw values with modified transmitters. I'm just wondering if that could be the cause of this somehow.

Is anyone here using G6 who IS experiencing this problem who is using a standard G6?
Is anyone here using G6 who is NOT experiencing this problem who is using a modified G6?

[Foxy7]

My wife and kid use standard G6 and Anubis G6 and we have the issue for both of them in both scenarios.

[Foxy7]

How does xdrip send data to AAPS?

Is there an output file to check?

[nickb24]

It's done using Intents
https://developer.android.com/guide/components/intents-filters

[Foxy7]

Thanks. Never heard of it before. It is possible to read the values sent manually? Ie like it's saved down in a csv style folder for AAPS to read?

[nickb24]

Only via Logcat

[dar-kod]

@nickb24 Here are attached my settings without passwords.

[nickb24]

@Navid200 These logs look a bit weird to me. Is this part of your debug code?

@dar-kod Is this a modified transmitter?

[Navid200]

No, those are standard xDrip logs.

[dar-kod]

Yes, it is Anubis. Yes, the log is strange. I even have this Sensor Raw Data Sanity Failure message, even when I am back to my database and settings.

[nickb24]

@Navid200 I always thought modified transmitters just prolonged sensor use and transmitter expiry date. These errors suggest more is going on. I don't have these kinds of messages on a factory transmitter.

As an example, here is my current log:

[dar-kod]

@Navid200 How do I get rid of the constant 'Sensor Raw Data Sanity Failure' message?

Update: Everything is fine now, this message no longer appears.

[Navid200]

No one experiencing this problem has engineering mode enabled, right?

[dar-kod]

I don't have engineering mode enabled.

[Foxy7]

No. Me neither

[Navid200]

Does everyone who experiences this problem use a Xiaomi phone?

[dar-kod]

@Navid200 It seems that this problem is not related to the phone. I use a Xiaomi phone, but I have had this problem with Samsung and OnePlus phones. And @Foxy7 has a Samsung phone.

[Navid200]

If xDrip does not have the correct time stamp, it will not broadcast the reading. This is how xDrip works.
Phones that have trouble maintaining connectivity are more likely to experience this.

If I am right and you don't have proper connectivity, this is not an xDrip bug. It is the intended expected behavior.
Please follow my advice to verify.

[Navid200]

@nickb24 This is the method that takes care of backfilling for Dexcom devices:

xDrip/app/src/main/java/com/eveningoutpost/dexdrip/g5model/Ob1G5StateMachine.java

Line 1915 in 8ed51e7

private static void processBacksies(List<BackFillStream.Backsie> backsies) {

And inside that method, this is the line where the reading is being verified with respect to its time stamp:

xDrip/app/src/main/java/com/eveningoutpost/dexdrip/g5model/Ob1G5StateMachine.java

Line 1924 in 8ed51e7

if (BgReading.getForPreciseTimestamp(time, Constants.MINUTE_IN_MS * 4) == null) {

Now, if you look at the target of this line, you will see this method:

xDrip/app/src/main/java/com/eveningoutpost/dexdrip/models/BgReading.java

Line 443 in 8ed51e7

public static BgReading getForPreciseTimestamp(long timestamp, long precision, boolean lock_to_sensor) {

Now, if you look inside this method, you will see the log that we have seen in this thread so many time right here:

xDrip/app/src/main/java/com/eveningoutpost/dexdrip/models/BgReading.java

Line 458 in 8ed51e7

Log.d(TAG, "getForPreciseTimestamp: No luck finding a BG timestamp match: " + JoH.dateTimeText((long) timestamp) + " precision:" + precision + " Sensor: " + ((sensor == null) ? "null" : sensor.getId()));

If you look at the logic in this method, you will see that this log is only shown if the reading is rejected.

This tells me that even though xDrip shows the reading, but, does not broadcast it.

[Navid200]

So, a question may come up here that why is the time stamp not found. I cannot answer that question.
You can see an open PR trying to improve the connectivity of Xiaomi phones here: #3567

I have never had one of these phones. So, I cannot do any testing.
The problem is that the more we add conditionals to xDrip to address different phones, the harder it will become to maintain by anyone who does not own all of those phones!
We may be locking ourselves in a corner if we keep doing that.

So, I don't have a perfect solution. I wish we could find a way to improve the behavior of Xiaomi phones as well as many other phones that may have similar issues.

[dar-kod]

@Navid200 I had the same problem not only with Xiaomi, but also with OnePlus and Samsung with /e/OS. And @Foxy7 has Samsung too.

[nickb24]

The function call here:

xDrip/app/src/main/java/com/eveningoutpost/dexdrip/models/BgReading.java

Line 446 in 8ed51e7

final BgReading bgReading = new Select()

Does a lookup in the SQLite database from the BgReading table. This lookup should have worked, but it doesn't. I assumed something must be wrong in the database, but @dar-kod did the test with xDrip reset, so that eliminates that.

This is getting hard to debug without actually connecting a debug console to a device with the problem.

[Navid200]

There phones that have difficulty maintaining connectivity. Before my time, I suppose Samsung was one of the first phones that caused problems. Currently, in xDrip, we have interventions to work around the Samsung messing with Android. Samsung effectively run their own version of Android.

So, I wouldn't be surprised if Samsung also has occasional connectivity issues.

If anyone experiences this problem, meaning getting backfill from Dexcom shown in xDrip but not sent to AAPS, I suggest enabling detail logs for the BgReading method.
I bet they will see this log:

No luck finding a BG timestamp match: ...

If not, I would like to see it because that would be an exception and I may be able to find a way to fix it.

I can add an exception to check to see if the phone is a Xiaomi to broadcast it. But, I don't think that is a good idea.
As I said, I currently don't have a solution.
I will look at it more to see if I can think of something.

[nickb24]

If anyone experiences this problem, meaning getting backfill from Dexcom shown in xDrip but not sent to AAPS, I suggest enabling detail logs for the BgReading method. I bet they will see this log:

It would be good to enable debug logging for 'Ob1G5StateMachine' too, to see if this line gets triggered:

xDrip/app/src/main/java/com/eveningoutpost/dexdrip/g5model/Ob1G5StateMachine.java

Line 1927 in 8ed51e7

UserError.Log.d(TAG, "Adding backfilled reading: " + JoH.dateTimeText(time) + " " + BgGraphBuilder.unitized_string_static(backsie.getGlucose()));

[Navid200]

@dar-kod Thanks for running all the tests.
In the process of working on this, I discovered an error in my PR and have a submission to correct it: #3843
Thanks
Of course the problem you are experiencing is not caused by it and you confirmed that by bone of your tests.

If you have not run out of patience, I have another test release to test. As before, you don't have to.
At this point, when there is a failure, we know that the time error is greater than 1 minute or greater than 4 minutes based on the logs you have provided. But, we have no idea exactly what the error is.
This test tells us exactly what the value is. It may give us a clue. But, I cannot be sure.
If you like to try, please again cause backfill to occur and when it fails to broadcast to AAPS, please show the logs.
This release only has an added log compared to our latest Nightly release.
https://drive.google.com/file/d/1m64CX48j7Q5EKMWSUZkEuIYYy4BBgQnL/view?usp=sharing

[dar-kod]

No problem, you can see the readings in sequence:

[Navid200]

Thanks for all the tests.
So, the second one from the bottom is the one that is not backfilled to AAPS.

I will need to spend some time on this to see if I can understand exactly what's going on.
I will let you know if I make any progress.

Thanks

[dar-kod]

Thank you.

[Navid200]

@dar-kod Are you sure that the values are not eventually backfilled to AAPS?
May be not right away. But, do they get backfilled may be in an hour?

[dar-kod]

@Navid200 No, they don't get backfilled even after 12 hours.