Scouting Myth

As a passionate amateur baseball blogger, I love to go over scouting reports of top ranked prospects with a fine tooth comb. Many scouts often claim that they have an intuitive ability to identify unpolished talents. The draft provides teams with a cost effective resource to build a successful franchise. The next question is, in an unpredictable and baffling world of projecting young baseball talents, do scouts demonstrate a strong track record on gauging future major league performance of young players?

Prior to evaluate and measure the efficiency of scouting, it is imperative to figure out how scouts look for young players with potential. There are many notable aspects which contribute to a scout’s projection of a player; as a result, the utilization of a uniform and systematic grading system facilitates interpreting complex information into simplified categories. Deric Mckamey, a well known scout for St. Louis Cardinals, explains the basic approach of scouting in following details:

"Scouts grade players based on a 20-to-80 scale (or 2-8) scale, with 80 representing the highest achievable grade. The grade of 50 is considered major league average. Position players are graded in five categories (hitting, power, speed, throwing, and fielding), which are typically referred to as the "five tools." Players will also receive grades for base running, arm accuracy, baseball instinct, and aggressiveness, though they do not account as much for the final grade.

An amateur player or minor league player will receive two grades for each tool: a present grade and a future grade, based on how they are expected to perform in the majors. Future grades are added and then divided by the number of grades to determine their Overall Future Potential (OFP). A scout can then adjust a player’s OFP by 10 points based on the lesser categories and their gut instinct."

The tables below are overall numerical grade scales used by many major league scouts:

Overall Grade	Class
65-80	Major League Star
50-64	Major League Regular
50	Major League Average
40-49	Major League Reserve
38-39	Organizational Player

Scouting Grade	Term
80	Outstanding
70	Plus-Plus
60	Plus
50	Average
40	Below-average
30	Well below-average
20	Poor

Grade	HR Power	Hitting Tool	Speed (L/R)
80	39+	.320+	3.9(L)/4.0(R)
70	32-38	.300-.320	4.0(L)/4.1(R)
60	25-32	.285-.300	4.1(L)/4.2(R)
50	17-25	.275-.285	4.2(L)/4.3(R)
40	11-17	.250-.270	4.3(L)/4.4(R)
30	5-11	.225-.250	4.4(L)/4.5(R)
20	<5	<.225	4.5(L)/4.6(R)

Grade	Fastball Velocity
80	97mph+
70	94-96mph
60	92-94mph
50	89-91mph
40	87-89mph
30	85-87mph
20	82-84mph

If the reader is interested, the general scouting report looks like this:

As a matter of fact, scouts seldom have any opportunity to ever witness an amateur player who is already a big league caliber; therefore, it is unfortunately that scouts are requested to gauge a player’s possible future major league performance long before he has developed into his full potential. In case of high school players, even before they have filled up their bodies. Loosely speaking, one must provide justification to support a knowledgeable claim. Similarly, a responsible scout should take a stance with rational-comprehensive approaches when it comes to evaluating overall future potential of a player. The main concern is whether there is any scientific evidence and methodological rationale underlying player forecasting in scouting or not. To be more specific, one of the scouting reports above noticeably predicts that Albert Pujols as an annual .245 hitter with 24 homers. If possible, I am more interested in knowing if there is any objective reason for a such claim; otherwise, I tend to see scouting as individuals’ opinions that are derived from subjective observation and experience. 

The ultimate aim of this study is to examine the efficiency of player valuation in draft result. The main focus of this review is to determine the return on scouting over the years of some noteworthy first-round draft picks. It is also very critical to acknowledge draft pick as a function of several variables, including league difference, payroll, player sign-ability, front office philosophy, significant improvement in empirical analysis of baseball, and scouting performance. The concept of scouting performance is not an easily defined one. For purpose of this discussion, the allegation of scouting capability replies upon the assumption that scouts play a significant role in player development output and draft success.

The following tables consist of first round draft picks from 2004 to 2007:

2004	WAR	2005	WAR
Matt Bush	N/A	Justin Upton	16.8
Justin Verlander	40.7	Alex Gordon	22.6
Phillip Humber	1.0	Jeff Clement	-1.2
Jeff Niemann	4.2	Ryan Zimmerman	33.9
Mark Rogers	0.1	Ryan Braun	35.4
Jeremy Sowers	1.5	Ricky Romero	9.9
Homer Bailey	6.0	Troy Tulowitzki	32.3
Wade Townsend	N/A	Wade Townsend	N/A
Chris Nelson	-2.5	Mike Pelfrey	6.1
Thomas Diamond	-0.3	Cameron Maybin	8.0
Neil Walker	10.4	Andrew McCutchen	26.9
Jered Weaver	33.4	Jay Bruce	15.8
Bill Bray	2.4	Brandon Snyder	-0.2
Billy Butler	12.9	Trevor Crowe	0.2
Stephen Drew	16.1	Lance Broadway	0.3
David Purcey	0.1	Chris Volstad	2.8
Scott Elbert	1.3	C.J. Henry	N/A
Josh Fields	-1.1	Cesar Carrillo	-0.7
Chris Lambert	-0.8	John Mayberry	1.2
Trevor Plouffe	1.4	Mark Pawelek	N/A
Greg Golson	-0.5	Cliff Pennington	8.4
Glen Perkins	7.2	Aaron Thompson	-0.2
Phil Hughes	6.3	Jacoby Ellsbury	21.0
Landon Powell	0.1	Brian Bogusevic	2.0
Kyle Waldrop	0.5	Matt Garza	14.8
Richie Robnett	N/A	Craig Hansen	-1.9
Taylor Tankersley	0.6	Joey Devine	2.0
Blake DeWitt	2.0	Colby Rasmus	12.6
Matthew Campbell	N/A	Jacob Marceaux	N/A
Eric Hurley	0.1	Tyler Greene	-0.6

2006	WAR	2007	WAR
Luke Hochevar	2.5	David Price	18.6
Greg Reynolds	-1.8	Mike Moustakas	4.3
Evan Longoria	27.4	Josh Vitters	-1.2
Brad Lincoln	0.2	Daniel Moskos	0.2
Brandon Morrow	7.6	Matt Wieters	13.1
Andrew Miller	-2.0	Ross Detwiler	3.4
Clayton Kershaw	32.2	Matt LaPorta	-0.9
Drew Stubbs	7.0	Casey Weathers	N/A
Bill Rowell	N/A	Jarrod Parker	6.1
Tim Lincecum	23.3	Madison Bumgarner	11.3
Max Scherzer	18.0	Phillippe Aumont	0.2
Kasey Kiker	N/A	Matt Dominguez	2.3
Tyler Colvin	1.1	Beau Mills	N/A
Travis Snider	1.9	Jason Heyward	18.4
Chris Marrero	-1.0	Devin Mesoraco	-0.8
Jeremy Jeffress	0	Kevin Ahrens	N/A
Matt Antonelli	-0.2	Blake Beavan	1.4
Kyle Drabek	-0.2	Pete Kozma	1.0
Brett Sinkbeil	0	Joe Savery	-0.2
Chris Parmelee	1.4	Chris Withrow	0.7
Ian Kennedy	8.4	J. P. Arencibia	2.9
Colton Willems	N/A	Tim Alderson	N/A
Maxwell Sapp	N/A	Nick Schmidt	N/A
Cody Johnson	N/A	Michael Main	N/A
Hank Conger	1.4	Aaron Poreda	0.4
Bryan Morris	0.3	James Simmons	N/A
Jason Place	N/A	Rick Porcello	6.7
Daniel Bard	4.3	Ben Revere	4.0
Kyle McCulloch	N/A	Wendell Fairley	N/A
Adam Ottavino	2.6	Andrew Brackman	0.1

Impact player > 10 Career WAR, Fringe or Wash-out player < 3 Career WAR

The table indicates that 18.3% of first round picks between 2004-2007 fail to reach the majors. Among those who reach the majors, approximately 47% of selected are considered as non-impact players, with weighted-average accumulated WAR that is less than three. Correspondingly, it means more than 60% of drafted players are either slightly better or worse than replacement level. Excluding other factors, it is practical and conservative to conclude that there is a plenty of room for improvement in scouting efficiency. Tools do not guarantee future performances. I also find it very sarcastic that scouts put strong emphasis on physical attributes in a very skill oriented sport. As the role of technology and the dynamic innovation in baseball decision-making continue to evolve, possibly in the near future, it would be interesting to see how intangible metrics and tools are about transform from opinions to algorithms in a very competitive and data-driven industry.