Hello!
Regarding your question about keypoints outside the bounding box, this is supported in YOLOv11 pose estimation. The model predicts keypoints and can handle situations where keypoints fall outside the detected bounding box.
For your project involving tracking sheep, pose estimation could indeed be a viable approach, especially if you focus on detecting heads and adding keypoints for body orientation. The calculations for keypoint loss occurs in the calculate_keypoints_loss function, which you can read more about here. You can also examine the Pose forward pass here.
Ultimately, the best approach depends on your specific data and goals. Since accuracy is crucial for your project, I recommend experimenting with both pose estimation and segmentation to see which performs better in your real-world conditions.