# MOT：A Higher Order Metric for Evaluating Multi-object Tracking

2022-05-15 07:35:34

## brief introduction

HOTA: A Higher Order Metric for Evaluating Multi-object Tracking yes IJCV 2020 Of paper, Before that, use MOTChallenge Mainly multi-target tracking benchmark Has been used to MOTA Evaluation criteria for ranking , although MOTChallenge Of metrics There are also IDF1, But the ranking is still based on MOTA Subject to . however MOTA In some cases, it is not enough to measure the performance of multi-target tracking , It's not even as good as IDF1, So this article reconsiders the multi-target tracking task , And a method is proposed Higher Order Tracking Accuracy Of Metric.HOTA It can better align the evaluation score with people's visual perception . MOTA The main evaluation is 2006 It was proposed in , And pass by MOTChallenge Blessing , It is still the mainstream multi-target tracking evaluation standard , and HOTA It's just been put forward , At present, only KITTI MOT In the use of . Even if it does replace MOTA, It will also take a long time .

## MOTA The problem of

### The proportion of detection is greater than that of tracking

MOTA The evaluation overemphasizes the effect of detection , according to MOTA Calculation method of , One extreme case is , The performance of the test is excellent , But all detected targets are not tracked , Instead, all are assigned the same track id, At this time MOTA It's going to be very high , because IDsw=0. But obviously , The tracking performance of this extreme case is 0.

MOTP Even more so , The root cause is that there is no tracking of anything , Instead, only evaluate the test results . although IDF1 The tracking effect can be evaluated , But the ranking depends on MOTA Of .

Pictured above ,gt The length of is 100, Tracking performance C hold gt Divided into 4 paragraph , In fact, the performance is poor , however MOTA the height is 97%.

### Precision The specific gravity of is greater than Recall

There is no definition IDsw Of MOTA by MODA, That is, the accuracy of multi-target detection （Multi Object Detection Accuracy）, The formula is as follows ：

You can find , If it's tested Precision Less than or equal to 0.5 Words ,MODA Will be for 0, Even negative values , And the test Recall Less than or equal to 0.5 But it won't have such an impact .

## Evaluation Metric

### DetA

DetA For the accuracy of detection , Evaluate the performance of detector in multi-target tracking , The functions and Precision and Recall almost , Total of all categories acc The following formula represents ：

### AssA

AssA For the accuracy of correlation , Evaluate the accuracy of correlation , The formula is as follows ：

DetA,AssA The role of , And Precision,Recall,IDP,IDR,IDF1 Very similar Precision,Recall It is used to evaluate the accuracy and recall of detection , and DetA Used to evaluate the accuracy of detection . IDP,IDR,IDF1 Used to evaluate the accuracy of matching , Recall rate and F1-score, and AssA Used to evaluate the accuracy of matching . This needs to know \text {TPA}(c) ,\text {FNA}(c) ,\text {FPA}(c) These numbers mean , First c It belongs to TP The point of , It can be TP Any one of , According to this point , We can always identify a unique GT The trajectory , At the same time, if there is pred Track and GT If the trajectory intersects at this point , We can also identify one pred The trajectory . It should be noted that , Even if it's the same GT Different trajectories c, It will also produce different \text {TPA}(c) ,\text {FNA}(c) ,\text {FPA}(c) , Therefore, these three values can only be bound with sampling , Not bound to dataset . This is related to 《Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics》 Different , Not for one GT The trajectory is assigned a maximum matching degree pred The trajectory .

And here you need

### HOTA

• Single index evaluation
• Evaluate long-term high-order tracking correlation
• Decompose into sub indicators , Allows analysis of different components of tracker performance .

HOTA Evaluation is a double jacquard coefficient , That is, I took it twice and compared it , First of all \mathcal {A}(c)

For the current interest-c Corresponding GT tracklet, Calculated True Positive Associations,False Positive Associations And False Negative Associations, This is the jackard coefficient on the first floor , It should be noted that interest-c It's not worth a , All the needs SUM. As shown in the figure below . The jacquard coefficient of the second layer is SUM After \mathcal {A}(c)

Compared with the results of the previous test TP,FN,FP. Last ,\alpha Is a fixed threshold , therefore \text {HOTA}_{\alpha } Is the result of a fixed threshold , and HOTA yes ：

It's like coco Of AP Calculation . Last , according to DetA and AssA,HOTA It can be calculated by ：