Pikemen: Why were they so Highly prized and Sorted?

A combination of discipline and bravery. You're correct that everyone with enough upper body power can wield a pike and stab-stab, but not everyone can maintain that pike held. Before bullets, warfare was about instilling dread in the ranks of the adversary. Because guys in formation were difficult to kill from the front, you had to kill them while their backs were turned.

Photo by Jonathan Kemper on Unsplash

The psychological dread that the cavalry charge produced - every fiber in your body wanted to run from this stampede of gigantic monsters - was what made it effective. If the pikemen stayed strong, "fear bent backwards" and the riders broke off their charge without ever striking the enemy. This gave the pikemen confidence to repel the enemy horse's second, third, and fourth passes if they came.

But first, they had to overcome the first obstacle. Disciplined pikemen were valued because they had expertise fighting charges, which levies with long sharp sticks did not. Furthermore, the pike was an offensive weapon, and its utility extended beyond simply establishing a "death box." The Swiss pikemen had one simple tactic: march across the field and force the enemy off as soon as possible. They had to go in formation, but they had to move rapidly.

Because their weapons were longer, they could practically kill the enemy with impunity once the lines made contact (and the enemy would break and escape rather than perish miserably), therefore they could only be slain by movement. 

Pikemen required organization to respond to flanking when it occurred, as well as athleticism and drilling to march in fast step for a mile without compromising their formation. And, although you are correct that these are not great abilities - this is akin to training men to put on a military parade - the pay of the pikemen was neither enormous nor as rare as we like to assume.

Because they are the most well-known medieval pikemen, the Swiss pikemen are frequently assumed to encompass all medieval pikemen, although this is not the case. In the Middle Ages, the phalanx was mostly guarded by knights, who dismounted and advanced in lance squares since lances outranged infantry spears, bills, and halberds.

Indeed, the pike was devised by the Swiss, who were originally halberdiers, to fight the lance phalanx of the Burgundian and Habsburg knights. Once the Swiss pike gained popularity, it generated a slew of imitators, most notably the German landsknecht, who wielded exceedingly long swords and would become their bitter adversaries. 

Finally, additional pikemen, such as Flemish spears and Scottish schiltrons, existed in previous times. And all of these forces were present. Essentially all knights understood how to fight dismounted with lances, and during the course of their history, one million Swiss mercenaries took to the field.

It was simply more efficient and practical to hire mercenaries to supplement one's standing force of knights rather than raise levies. These guys arrived pre-trained and pre-hardened, with an existing military hierarchy that they followed and social links that kept them fairly coherent and prevented them from fleeing their comrades.

At least some in each formation were also influenced by previous conflicts and, more importantly, camp diseases, sieges, long marches, and famine. They were plentiful, inexpensive, and, most importantly, politically convenient. Because the European social system centered around knights' military dominance over commoners, it was unwise to develop a competent and disciplined force of commoners in your own country. 

As previously said, the knightly class (which was not a closed caste and had many commoner entrants) was already adept in the phalanx, but to increase them, you needed to either give new entrants regular money, subsistence, land, or a combination of the three.