I learned this was a real science today while reading Leonardo da Vinci by Walter Isaacson. Britannica describes physiognomy as:

The study of the systematic correspondence of psychological characteristics to facial features or body structure.

Put simply, judging people's character or competence by how they look. It's quite a crude practice indeed, but nonetheless was important enough at some point in time to have a term created out of it.

The earliest work on physiognomy was attributed to Aristotle. It contained six chapters to:

The consideration of the method of study, the general signs of character, the particular appearances characteristic of the dispositions, of strength and weakness, of genius and stupidity, and so on. Then he examined the characters derived from the different features, and from colour, hair, body, limbs, gait, and voice. While discussing noses, for example, he says that those with thick, bulbous ends belong to persons who are insensitive, swinish; sharp-tipped noses belong to the irascible, those easily provoked, like dogs; rounded, large, obtuse noses to the magnanimous, the lionlike; slender, hooked noses to the eaglelike; and so on.