Custom Train Loop and Validation Loss?

[Futuramistic]

Version used: 0.x
Hello! I was wondering about two things:

1. Can I write my custom loop like in Pytorch? Would this be correct?

 for epoch in range(epochs):
        for data_batch in train_dataloader:
               optimizer.zero_grad()
               img     = data_batch['img']
               target  = data_batch['target']
               weights = data_batch['target_weight']
               output = model.keypoint_head(model.backbone(img))
               loss = model.keypoint_head.get_loss(output,target,weights)['heatmap_loss']
               loss.backward()
               optimizer.step()
               acc = model.keypoint_head.get_accuracy(output,target,weights)['acc_pose'] 

Why would you write a custom loop? I want to scrape the accuracy and loss and monitor these using TensorBoard. I am more familiar with Pytorch than the framework itself, so it seemed like a natural step. If you see any improvements in this loop, please let me know.

2. Can I generate validation targets?

I followed my train pipeline's target and target weights generation procedure. What I find odd is when I use a similar validation loop to the one in question 1 (no gradient descent here, obviously), the validation loss increases with epochs. Is this due to the custom loop or the target generation? It is weird as the accuracy on the validation side also increases
Maybe you are overfitting? I thought so, but the trend follows from the first epoch onwards. I don't think I would be overfitting so early.

[Ben-Louis]

1. If you'd like to monitor the training process using TensorBoard, you can easily do so by incorporating the following configuration into your config file:

log_config = dict(
    interval=50,
    hooks=[
        dict(type='TextLoggerHook'),
        dict(type='TensorboardLoggerHook')
    ])

2. It appears that you're using a heatmap-based method, which inherently creates a gap between training and inference. Therefore, a divergence between validation loss and accuracy is conceivable. Is this trend consistent?

[Futuramistic]

1. Ah, I understand. Nevertheless - do you think this loop is correct? Do you think it still works as intended, or is there something fundamentally different/wrong about this loop? Because I do need some other values monitored within the code while training (my problem is not just pose estimation).
2. Sorry, yes - I do use a heatmap-based method. The trend of both validation loss and validation accuracy increasing is consistent - both go up from the start of training. There are some fluctuations, but these are minor. On the other hand, train loss and train accuracy follow the expected path - overall: train loss decreases while train accuracy increases. So you are saying that this trend on the validation side is expected?