Proxy Models for Market Estimation

Blaine Bateman

You can download a PDF copy of this article here.

A key role of any business analyst is to provide estimates to management so that a decision can be made.  I can argue that all decisions made by rational people are made using data (even if they appear to come from “gut feelings” or emotions or intuition), but the quality of data varies widely.  Often, the desired data aren’t readily available, and the cost to get them directly is prohibitive in time, resources, or expense (i.e., extensive surveys, contracting a research firm, etc.).  Below I discuss using available data to model the desired data.  This can be called ‘using a proxy’.

For example, you need to estimate the market for radio equipment, of the type used by police and fire personnel.  Specifically, regarding the portable radio portion of this market, you are creating a business plan for sales of components or accessories for these radios.  If you can estimate the end market, you can estimate the total addressable market (TAM) for your product, if you know how much or how many of your products are used in or with the end product.  You also want break down the estimate by geographic regions if possible, since the sales team will want this.  A common first approach is to ask your sales team what they think the market is.  I have attended many sales reviews and been struck by how the market data would change every meeting.  The figures were especially volatile if the sales team were under pressure to show market share gains!  Are there other approaches to make these estimates?

In this example, what is desired is the number of radios sold (or the number in use, if you are selling an accessory or upgrade).  While for some markets, such data are readily available (auto sales, for instance), they aren’t in this case.  In fact, more often than not, there are no data available at the level of granularity needed.  The situation isn’t totally bleak.  A possible approach is to estimate the number of units purchased, by analyzing the end consumption market, vs. finding sales data.  In this case, I assume the bulk of the market is to supply police and fire personnel, focus on finding the number of those personnel, and model the consumption of radios from that.  (Note: it is important to document all assumptions along the way, as the end figures tend to take on a life of their own.)

Some internet searching reveals various ad-hoc data sets for large police and fire departments, but the question remains how to generalize these data to national, regional, and global levels.  One approach would be to identify the largest cities or administrative regions around the globe, and try to get reports on them, then extrapolate, say using a ratio to population.  For instance, many larger US cities provide annual reports on their fire and police departments, with personnel figures.  One challenge with this approach, aside from being time consuming, is obtaining data for other countries.  There may be similar reports for jurisdictions around the globe, but using them would require not only finding them but dealing with language/translation issues to understand them.

Alternatively, there are many global and national organizations that provide statistical data sets we can use.  The United Nations Statistics Division provides a number of data sets with global coverage.  In particular, the UN Office of Crime and Drugs (UNODC) provides data on police forces for about 90 countries that I will use later in this example.

Data for some obvious countries are missing entirely from the UN data: in particular, Brazil and China are missing from the UN data.  Another informative source for police forces data is Eurostat.  I used Eurostat’s data on police forces to fill in and reconcile missing pieces in the UN data.  With the UN and Eurostat data combined, I had a dataset of 108 countries without too much effort.

Full disclosure: I found some of these sources in Wikipedia.  Although I recommend you do not use Wikipedia as an authoritative source, there is nothing wrong with capitalizing on the work compiled there by looking at the references.  (Reading the terms and disclaimers on Wikipedia, you should conclude that anything used for a business purpose should be researched back to the primary information sources anyway.)  For example, there is a Wikipedia entry entitled “List of countries by size of police forces”, which cites both the UN source and Eurostat, and a number of other sources.  The Wiki article’s authors used these data, then researched various other information (such as news articles and scholarly studies) attempting to improve the data.  However, with two reasonably reliable sources (UNODC and Eurostat) I created a larger dataset than the Wikipedia article with nearly the same data, and avoided researching another 20 or so references.

Even 108 countries is only a sample, so I want to develop a model using these data that can be used for any region of interest.  To approach this, I’ll first explore a technique used often when looking at large markets—relate the data to either GDP (Gross Domestic Product) or Population data.  Figure 1 shows how the Police force size (Np) data relate to country GDP at Purchasing Power Parity (PPP).  The GDP data are available from many sources; in this case I chose to use the CIA World Factbook 2011.  The CIA site is another wonderful public data source and the World Factbook is one of the most used government publications.

Unfortunately, this isn’t good enough, since a simple linear fit accounts for only about half the variation (and other fits, like logarithmic, didn’t work at all).  I looked into the data in more detail, first noting that the four labeled points were dominating the scale of the chart.  Focusing in to low GDPs, a different view emerged.  In Figure 2, there is noise at very low GDPs, but two different groups emerge; I call them high and low where the high group apparently has more police per GDP and a steeper growth of numbers as GDP grows.  I created two groups of data and fit them separately, with the labeled points included as shown.  This accounted for 84% of the variation in the low group and 88% of the variation in the high group.  Extending these fits to higher GDP appeared to account for most of the data, as shown in Figure 3.

Taking the two fitted lines to form a range of the observed force size for a given GDP, a figure can be bracketed, or said to be between an upper and lower bound, for a region or country, as long as there is a GDP estimate.  (It is interesting to note here that GDP data exist down to very fine granularity in the United States.  For example, the United States Department of Commerce (DoC) Bureau of Economic Analysis (BEA) provides GDP data for states and even some metropolitan areas.  This could be used to generate estimates of police forces by state, and then to estimate the market for the products of interest).  There is an upper limit to how far the high line can be extrapolated, since China is the largest “high” country and falls well below the line.  Ignoring the four largest GDP countries, there is a triangle within which most countries fall.  This is a quick route to a back-of-the-envelope quality estimate.  To improve upon this, I looked at the police data versus population figures.

In Figure 4, I used a 2nd order polynomial fit of the Np data versus population, both with and without Russia, which appeared as an outlier.  About 88% of the variation in police force size is accounted for just with population by country, and nearly all (96%) the variation when the data for Russia is excluded.  In this view Russia seems to have unusually high numbers of Police.  Perhaps that isn’t too surprising!  Here, population data are for 2010 and are taken from the UN Population Division

The UN population data include the distribution of ages in 5 year groups, and I charted all the distributions using the sparkline feature in Excel 2010™.  As I already knew, there are significant differences by country in the distribution of population; for instance it is well documented that Japan has a very ageing population.  Figure 5 shows a few countries from my sparkline fits as an example of the variation.  This is of interest because it is possible the size of Police forces is related to crime rates, and further that crime rates vary by age group.  For example, I think it is a reasonable assumption that very few crimes are committed by toddlers or spinsters.  The bottom line here is that since the age distribution is not the same for all countries, it might improve the fit to account for ages. 

To illustrate how to account for factors like this (just for illustration: at this point an R2 of 0.96 is probably good enough) I first looked at a simple model: including only the population from ages 15 to 39.  I applied weighting factors of 0.2 to the population in the relevant 5-year age groups, and factors of 0 everywhere else, then added up the weighted groups and charted the Np versus the so-called Simple Weighted Population (SWP).  The result is shown in Figure 6.  In this case the additional analysis does not improve the result, although the new fit might be a little more robust if it is to be used to estimate smaller regions with different age demographics.  

To illustrate further, the United States Department of Justice, via the Federal Bureau of Investigation, provides many data sets of crime statistics, including the distribution of various crimes by age.  Additional data of this sort are available from the United States Department of Justice Bureau of Justice Statistics.  From the FBI 2003 Uniform Crime Report, I estimated weights for the population data in 5 year increments of age, and multiplied the actual population by the weighting factors then summed up for what I call the Crime Weighted Population (CWP).  Figure 7 is adapted from the above reference, and is an approximation of the incidence-age curve.  The final result in Figure 8 essentially duplicates the original simple fit vs. population, so again the extra effort may not be worth it, but the model might be more robust when used for more granular estimates and forecasting later.  In this case, as shown in Figures 9 and 10, calculating the R2 value (square of the correlation coefficient) for a subset of data at small population sizes does indicate a slightly better model performance for the CWP model vs. the simple population model.  Again, this is mainly for illustration of how to include other factors when you are trying to build a proxy model.

To summarize where things stand, I plan to use the size of police forces as part of a proxy model to predict consumption of a product, for a market analysis.  It appears the size of police forces is predicted reasonably well using only population data, and the prediction can be improved (very slightly) by accounting for the distribution of ages in the region under analysis.  As a reminder, along the way it was noted there are lots of demographic data availble and several sources were provided.  Population data are readily available from many sources.

The other major end-use group I suggested to include are fire department personnel.  I will now turn attention to modeling the size of fire departments from known data.  Figure 11 charts data based upon a report from the International Association of Fire and Rescue Services, Center of Fire Statistics (CTIF) (Centre of Fire Statistics of CTIF – World Fire Statistics 2006 – Report №. 11).  As an initial test of the data, I added an independent data point for England taken from the Fire and Rescue Service Operational Statistics Bulletin England 2006-7. (Note--there are newer data available at the .uk site, but I used the data most closely aligned to the CTIF report).  Figure 12 charts the same data for the smaller countries to show more detail.  A challenge with Fire Department data is that many Fire Departments have Firemen who are part time or volunteers.  In some countries, for instance the United States and Canada, the ratio of career (i.e., full-time employed professionals) firefighters to volunteers is a strong function of the size of the metropolitan area protected (U.S. FIRE DEPARTMENT PROFILE THROUGH 2010, October 2011, National Fire Protection Association, Fire Analysis and Research Division).  The same can be said for Canada (FIRE DEPARTMENTS IN CANADA, 2008-2010, November 2011, National Fire Protection Association, Fire Analysis and Research Division).  In Figure 11 and Figure 12 these are charted in different colors, which highlights how the figures vary widely between countries. 

Rather than try to model the breakdown of headcounts by population or some other variable, I estimated how to add full time professional (FTP) headcount with part time and volunteer to get an FTP equivalent figure:  I counted FTPs as 1, part time as 0.5, and volunteers as 0.1.  The idea here is that the FTP equivalents should correlate well enough to the equipment usage and thus market potential I’m ultimately after.  If it was shown later that this wasn’t the case, I could go back and work on another layer of modeling to deal with this.  The weighted totals are shown in Figures 11 and 12 as the purple lines linked to the right axis.  What you can see is that this smooths out the data.  I rejected some of the data from the CTIF report as it appeared incomplete or anomalous—Vietnam and Laos showed unusually low figures (I guessed due to either (a) incomplete data or (b) some special circumstances) and although Switzerland was included in the table, no data were given for firefighters, only for stations etc.

Figure 13 shows the data charted vs. GDP (same source as above) and Figure 14 are the data charted vs. Population (same source as above).  The initial conclusion is that both GDP and Population are highly correlated to the size of fire departments in simple linear relationships, accounting for 92% of the variation with GDP and 96% with Population.  Because a lot of the data are crowded at lower department  sizes, Figures 15 and 16 re-chart the data for the smaller countries.  What appears now is that there is significant curvature for the data vs. GDP.  This provides an opportunity to illustrate another approach to modeling data like these.  Excel 2010™ provides a function called LINEST which is capable not only of doing linear regression fits of simple two-variable (i.e., x-y) data sets, but can also perform multiple linear regression (MLR).  MLR essentially treats multiple sets of “x” data each independently to generate a best fit of the form: m1*x1 + m2*x2 … + mn*xn + m0 = y where the ms are coefficients and the xs are the data.  Because you can put anything into the data columns, if we wanted to fit the data to, say, F = m0 + m1*GDP2  (where F is the number of FTP equivalent firefighters and m0 and m1 are constants) we can create a column where we square the GDP values and use that in the LINEST function to determine m0 and m1 (coefficients of the line).  However, even more powerful is the fact that we can simultaneously fit F to, say, Population and GDP2.  The equation might be something like F = m0 + m1*GDP2  + m2*P.  To do this we simply create a column for the square of the GDP values, and provide that and the column of Population values and the column of weighted total firefighters to LINEST.

Figures 17 and 18 show the results of modeling F = m0 + m1*GDP2  + m2*P.  In Figure 17 we see that the combined R2 is 0.96, and in Figure 18 we can see that the fit is working well at the lower values.  These charts present the predicted (calculated from the equation) value vs. the actual value, since for an MLR result you cannot conveniently present the actual data against a single variable.  Thus in Figures 17 and 18 the variation from a line of slope 1 shows visually the goodness of fit.

To test this result (note that the data for England, which was added in previously, fits well into the data and was already used in the MLR fit), I searched for more data from other sources.  The European Federation of Public Service Unions (EPSU) published some 2010 data for several other European countries.  Figures 19 and 20 show the contrast between the EPSU data and the earlier CTIF data.  Charted vs. both GDP and Population, the EPSU data tend to fall below the linear fit and exhibit some curvature; perhaps more so in the Population case.  This bolsters the case to use an MLR solution so we can include both the GDP data and the Population data.  In fact, during my work for these examples I tested several versions of MLRs, including a GDP (not squared) term, and with and without the constant term forced to zero (in many cases modeling real-world data, if you want to extend the correlation down to very low values, it makes sense to assert the “boundary condition” that at zero population or zero GDP the other data are zero as well; however in many cases the “fit” is better (in tems of an R2 value) when the constant is allowed to be non-zero.  You have to investigate this case by case for your needs).  In this case, good results were obtained with the simple model inclding a linear term for Population and a term for GDP squared, and keeping the constant term zero.

I added in the EPSU data (again, some countries seemed way out, and I rejected the data—I removed data for Estonia and The Netherlands (very low) and for the Czech Republic (extremely high)).  The original data from te EPSU reference are shown in Table 1, along with the Weighted Total Firefighters (FTP equivalents) used in this analysis.  Figure 21 shows the results of the revised MLR model including the EPSU data; the fit is only slightly degraded and now we are including 25 data points.  In Figure 22 I show the data expanded at lower GDPs, and include another independent point for New York City (not included in the fit, just charted independently).  The model predicts the data for New York within 10%.  The New York data are from the Fire Department of New York Annual Report 2008-2009 with Population data from the US Census (as noted earlier) and GDP data from the US Bureau of Economic Analysis.  The BEA data were adjusted by the ratio of the Metropolitan area population to the City population using both population data sets from the US Census.  This was necessary as most Fire Department data are for cities, but the GDP data are for Metropolitan areas.  (The difference is very significant—for instance, New York City has a population of 8.2M in the 2010 Census, but the New York-Northern New Jersey-Long Island Metro Area has a population of 18.9M in the same Census).

What is apparent from Figure 22 is that the MLR model works pretty well, and that at least New York City falls in line with global country level data (you can draw your own conclusion as to whether NYC is a small country!).  In Table 2, I bring together data for some other US cities, taken from reports I could find on the internet.  As I have done so far, I combined the data on number of firefighters with the population data from the US Census and the GDP data from the BEA.  As before, I had to estimate the city-level GDP from the metro area GDP reported by the BEA, and as before I used a simple ratio of the city population to the metro population as the correction.  Figure 23 shows how well the MLR model estimates these data.  In general the estimates are a little low, but in truth the MLR model does a remarkably good job.  The bottom line is we now have an equation that predicts the number of FTP equivalent firefighters for a given region using known population and GDP data, and works from the level of countries as large as the US down to cities of population around 60 thousand.  That is more than 3 orders of magnitude in size, which is really good to be able to cover with a single empirical model.  Previously, we developed a similarly powerful tool for police forces, so we can combine them to predict the total end users of radios.  With that, we are ready to use these models for market estimation.

I assume that since I have modeled FTP equivalents, there is about a one to one ratio of headcount to portable radios in use.  This may or may not be a great assumption, but provides a starting point.  Any more precise relationship easily can be included later.  In addition to radios in use, there will also be some spare inventory, extras, different models, etc. and I take that to be 5%.  At any given moment, I assume the departments are fully equipped; in this case if there were no radios breaking, being lost, or otherwise needing replacement, then the new market each year would be given by the growth in the ranks of the users.  I’ll come back to this component shortly, as it can easily be estimated based on projected GDP and Population figures. 

Every piece of equipment eventually wears out, even with maintenance.  In addition, there is obsolescence which drives replacement, as well as new features and technology that become available in new models, which may encourage users to replace equipment that is still serviceable.  In the case of portable radios, I could not find clear references for the actual field lifetime of radios in use.  What I could find were depreciation recommendations which are used for valuation of assets, as well as accruing funds for future replacements or upgrades, or simply for budgeting purposes.  While not a statement of actual equipment field lifetime, depreciation schedules are based on simplified expected life of capital assets.  Table 3 shows guidelines of four jurisdictions, with the lifetimes ranging from 5 to 10 years.  I take 7 years as a reasonable typical figure for this analysis.

Another potentially significant factor in the replacement of equipment is technology changes which lead to obsolescence, non-interoperability, or simply the desire to gain access to newer and better features in the new technology.  In the case of police and fire radios, a massive technology change has been underway since the mid 1990s.  After the September 11, 2001 attacks, the lack of inter-operability and other limitations of existing public safety radio systems came under severe criticism.  Whether the criticisms were justified or not, they led to proposed digital radio systems and changes in spectrum use.  Today, it can be seen in many municipal budgets that major system upgrades are being planned or at least proposed.  (Not without controversy—there are some reports of lower audio quality and intelligibility with digital systems, and attendant safety concerns.  See reports from NIST, Public Safety Communications Research, and the International Association of Firefighters.)

A widely cited report regarding an upgrade plan is a study performed by Federal Engineering, Inc. for Contra Costa County in California.  This study points out a variety of obstacles between current use and a future fully interoperable state (as well as the future state including data transmission vs. voice today, etc.), and largely concludes that eventually a migration to 700MHz or 800MHz digital systems will take place.  This transformation, proceeding now, is something of a “once in a lifetime” change in the user communities and the industry, but nonetheless affects any estimates of the markets for the next 10 years or so.  In particular, there are remarks in the report that state short-term upgrades and configurations might extend system life for systems currently deployed in Contra Costa County by five to seven years, at which point they will be overloaded and unable to meet expected needs and demands.  As the study was published in 2002, this indicates we are in the time window of complete system replacements.

Comparing the system transformation dynamic to expected typical lifetimes, it seems the characteristic timeframe for a major overhaul is of the same order as the already existing replacement timetables.  If I consider that there is likely a significant rate of loss of radios due to breakage, loss, theft, and other damage (I’m mindful that these radios are designed to be rugged, yet are used in often severe environments), I conclude I need to add another 10% to the replacement volume, or equivalently take 10% off the lifetime for replacement.  I conclude that assuming complete replacement at the 105% of manpower level every 7 years is a reasonable point for this analysis.  It would be difficult to argue more than 10 years, and likewise assuming 5 years, even with a technology overhaul, seems aggressive.  The one thing missing is that I could not find reports of actual service life of police and fire radios, although there seem to be a lot of anecdotes indicating departments routinely use them beyond suggested life or until after they are “obsolete” meaning they are incompatible with newer equipment coming in, or they are no longer available at all.  (See firerescue1.com for an example of using analog 800 MHz radios for over 10 years, and this from the Washington Metropolitan Area Transit Authority as an example of another recent plan to upgrade old radios.)

With all the above data, analysis, and assumptions, we are now in a position to write down a model to estimate the annual market in units for police and fire portable radios.  In words, it is “The sum of a 2nd degree polynomial fit of Crime Weighted Population (police forces estimate) and a multiple linear regression of Population and GDP squared (fire departments estimate) times 105% (to include spares etc.) divided by 7 (estimated field life of radios) plus the year on year change of the estimated force sizes (assuming department purchases on average keep up with growth)”.  We can make this estimate for any geographic area for which we have population and GDP data, which, as has been shown, includes most countries of interest, and all large US cities and metropolitan areas.  To create a specific estimate, we need to access data from only a few sources, and do a few simple calculations.

To complete the example, I gathered the data for the most populous 25 countries and used the models to calculate estimates of the police forces and firefighters for 2010.  Using the population and GDP data for 2007 and 2010, I estimated the Compound Annual Growth Rate (CAGR) for population and GDP, then used the growth rate to project values to 2012.  I then used the 2012 estimates of population and GDP in the models to calculate 2012 estimates for police and firefighters for each country.  The main components of this analysis are given in Table 4Table 5 shows the calculated estimates of radio sales for 2010 and 2012, using the equation defined in the previous paragraph.  In addition, the 2012 sales estimates for each of the 25 countries are included in Table 4.  These values are charted on a world map in Figure 24 to give a sense of the geographic distribution of consumption.

It is of interest to note that the prediction is for demand to decrease slightly in 2012 compared to 2010.  This is mainly due to a higher than normal increase from 2009 to 2010, as the global recession ended, and relatively modest GDP growth in most countries since 2010.  In Table 5 I also provide figures for the entire world demand, which would be what most businesses call the TAM—total addressable market (or total available market).  For this example, I simply used the ratio of the population of the entire world to the sum of the populations of the 25 countries listed, which was about 0.75.  Because the estimates of sales depend on not only the current year population and GDP, but the year-on-year change in number of police and firefighters, it would be more accurate to carry out the full calculations for the remaining population and GDP, but the figures listed are still good estimates.

Ultimately, your goal might be to estimate how many radio holsters will be sold, or how many belt clips, or how may antennas.  In order to do that, you need to also estimate at least a ratio between your product of interest and the number of radios.  Also, it is likely that while I have argued a 7-year lifetime is a working figure for the radios, things like antennas or holsters might get replaced more often.  Again, it is a simple matter to include that in the final model and adjust it if data are presented later showing a need to change your model.  In the notes for Table 5, I provide the result for antennas assuming a 2-year lifetime (mobile radio antennas are known to be frequently damaged by rough use and misuse, and are designed to be replaceable on all high-end radios of the types used by police and firefighters).

Conclusion

This paper shows various approaches to use widely available demographic and economic data to correlate size of interesting end markets.  The specific example used GDP, population, age statistics, and crime statistics to correlate the size of police and fire departments.  Under some assumptions, these figures were then taken to further correlate to consumption, in this case of portable radio units.  The general ideas and methodologies given here can be used in many other problems for market and business analysis.  Some of the value of this approach is that the underlying data are available for almost all interesting geographic sub-units, including cities in some cases.  These approaches are based on general cause and effect principles, such as “the more people in an area, the more police are needed”, and likewise the more fire department personnel are needed.  For fire departments, there is a further concept that increased density of economic activity tends to require larger fire departments.  However, beyond these general ideas the models are not causative, per se.  Nonetheless, correlations with a high goodness of fit can be generated which span several orders of magnitude, making this approach powerful as part of an analyst’s tool kit.