Noefelids (true cats) are first seen in the early Miocene where Proailurus is found. Various types of cats evolved from Proailurus including Smilodon and the other sabre-toothed cats. There was also another family of carnivores that was cat-like in appearance, known as the Paleofelids (false cats).
The sabre-tooth cat is not closely related to any modern day cats. The sabre-tooth cat was an early evolutionary branch that went extinct, where modern cats of today are an entirely different evolutionary branch that occurred much later. If the sabre-toothed cat had to be compared to any modern day cats, it would most closely resemble the modern day lion. They both were social animals who hunted their prey, and lived in packs. The other cats of today are solitary hunters and drifters.