Multiple hypothesis tracking (MHT) addresses difficult tracking problems by maintaining alternative association hypotheses until enough good data e.g., features, are collected to select the correct hypotheses. Traditional MHT's cannot track targets over long durations because they frequently generate too many hypotheses to maintain the correct ones with the available processing resources. Track segment graph provides a compact and efficient representation of the key ambiguities in long term tracking. It is used to generate the long term track hypotheses that are evaluated to select the best long term global hypothesis. Simulation examples demonstrate the efficiency and optimality of the approach.