ECI® Blog

ECI's Foundation Study v. Google's Project Oxygen to Identify High Performers

March 16th, 2011 by The ECI Team

One of our associates passed along this New York Times article about Google’s Project Oxygen to me earlier this week.  Google wanted to identify the factors associated with high performing managers.   Being the experts they are with data analysis, they sliced and diced all of their performance review ratings and other anecdotal information to identify the behaviors that are unique to their best managers. They were surprised to find that technical skills are not what enables good managers to make the list.

I liked this article because it more or less confirms what we have been doing in our research for the past 15 years.  Our business, ECI, founded in 1996, is built upon the identification of high performance behaviors in a variety of environments and roles using statistical analysis of performance metrics. Like Google, we have found that this type of data analysis yields a valid and reliable formulation of the root cause for success.

But since we have been focusing all of our attention on identifying high performance behaviors within organizations, here are our best practices that Google’s analysts might want to consider on the next round of Oxygen studies:

  1. It is not sufficient to screen for key words in performance reviews and anecdotal information. While that practice might put you in the ballpark, it won’t get you to your seat. There is too much variance and inconsistency in prose type performance reviews. If you really study a block of performance reviews, you find that most managers are not appropriately trained in giving objective, actionable feedback, nor are they consistently assigning ratings to performers.  This inconsistency of ratings across the review process skews the data.
  2. Use force rank against a Behaviorally Anchored Rating Scale to identify quartiles of performance for your overall population.  The overall ratings assigned in the standard performance review process cannot be relied upon to indicate who is the better manager. In our studies, we find that in 60% of companies, ratings are assigned for some other purpose than to evaluate actual performance levels. These include attempting to norm a population to a bell curve for compensation purposes, feeling that someone deserves a raise and having to justify this with the performance rating, and favoritism by the manager for the most politically savvy performers on the team.
  3. Use multiple measures to confirm or overturn the presence of key high performance behaviors.  ECI’s rule is that if you identify a factor in one segment of the study, you must verify its presence in another segment in order to consider it applicable to the model.
  4. Use valid and reliable metrics, such as indices, personality assessments, and other proven tools to identify core performance behaviors and behavioral preferences. If you incorporate a couple valid and reliable metrics in the study process, you can statistically compare the findings from these more rigorous tools to the less objective sources of data in your study to know with good certainty that you have proven a relationship to the high performance behaviors/factors you identify.
  5. Make sure you include results from job analysis within your study process.  By observing the work in context, using a standardized interview form designed to assess the work environment, and identifying differentiating performance factors using this process, the criteria you establish should  pass the muster of the EEOC, if you decide to use this model for selection or promotional purposes.
  6. Use professional statistical tools, such as SPSS, to confirm the validity around your model. When you put people into a room and say “does this look right to you?” or “how would you modify this finding?”, the only thing you are verifying is face validity. That is insufficient, in my estimation, to devise a management development program or another talent management process. You need the numbers to prove your model. Hopefully, the standard you achieve is at least a correlation significance of .70 against the ratings you used to identify your high performing population.
  7. Don’t forget to look at the entire population, not just the high performing group. If you only study the top performers, you don’t know if the factor you identified is present for everyone in the group or only high performers possess it. In our studies, for example, we find that all sales people within a large sales force have good self-confidence, can withstand rejection and are motivated to persuade others. While these factors are critical to selling success, the only thing we can say with certainty is that the original screening process used to hire the sales force is doing a good job of identifying these factors. These are the rudimentary factors associated with all successful sales forces; they are essential, but they do not help us to identify the additional factors needed for success in a specific company culture, marketplace or customer group. The unique factors are those that drive exceptional results, lower turnover, and higher job satisfaction.

Google did recognize that generalized industry principles and recommendations are not good enough to really drive their organization’s unique high performing manager behaviors. I commend them for that perspective. I would love to take a look at their data and make a couple of recommendations on how they might enhance the validity and reliability of their study process, however. That would surely be a wonderful conversation.

Are Pharmaceutical Reps Exempt or Non-Exempt?

March 2nd, 2011 by The ECI Team

A recent article in the news described the court case at Novartis where sales representatives were pressing for overtime, given the structure of their accountabilities.  This has been a topic we have reviewed a number of times for our clients and which rarely lands on the same recommendation.  Pharma companies generally pay on business results – marketshare,  marketshare change and sometimes number of scripts.  The job itself of the sales rep historically has been one of narrowly defined accountabilities, which are often assessed by reach and frequency metrics.

In those cases, the department of labor and the courts have an easy time classifying a role as non-exempt in status.  The rep is required to make an average 7 – 9 customer visits per day, to deliver two or more key product messages when the opportunity arises to speak with a physician or other professional, and then must ensure that sufficient samples are available for the prescriber to dole out product, based on patient needs.  Because of the heavy focus on measured tasking (even though it is difficult to directly link the use of the product to the message delivered by the rep), the job assessor tends to say that very little is left to the rep’s own choice and that the job is pretty clearly defined in the various systems used to monitor performance.  When the job is clearly defined and leaves little to the choice of the performer, then it is classified as a non-exempt position.  There are a lot more standards that are applied to make this classification, but at the end of the day, freedom of choice on what is done and levels of decision making are at the root of the classification.

Enter the legal department at the pharma company.  In the last couple of years, there has been a strong push in job descriptions to place language around independently developing strategy, establishing priorities for the territory in terms of selling activities, and establishing one’s own daily schedule.  Using the word professional to elevate the role of rep to business “owner”  who is accountable to develop key contacts and manage a broad range of relationships has been an attempt to elevate the expectations of the role.  Somehow, these added wordings don’t quite do enough to elevate the role to the exempt level, however.  The accountabilities are still the same – see the docs, deliver the message, influence drug of choice, and leverage the relationship to access other medical providers.

What is interesting to me is the fact that with the rapidly changing landscape of the healthcare environment, the addition of so much more complexity in healthcare providers, formulary positioning, specialty pharmacies, large IDNs, care provider networks and institutions has made the job of sales rep much more difficult.  Reps have to know the clinical and treatment aspects of their products better than most physicians.  They need to understand how to help the doctor use the product with patients whose access to the drug is limited by their medical coverage or geographic location and care networks.   I doubt that the old reach and frequency model would even work effectively today in many of these situations, outside of some less sophisticated marketplaces that are not as heavily impacted by managed care practices.

A new customer development model that has emerged requires the representative to assess all of the local conditions and to devise a strategy that best addresses these conditions, while aligning to company goals, the compliance and regulatory environment, and physician preferences.  This hardly looks like a non-exempt position when you increase the complexity of the work to this level and note the amount of variation in responsibilities and approach that will is needed to perform the role properly.  Given the amount of technical clinical knowledge needed, the in-depth strategy setting and innovation required to succeed and then the amount of collaboration and networking expected, measuring success is not a simple matter of measuring number of calls, delivering the approved marketing messages, and devising the most efficiency call route.

In recent visits to the field with our client’s reps, we have seen reps changing the treatment preferences of surgeons, helping to gain approval for treatment for non-formulary drugs by establishing pre-approval systems in physician offices, and a much higher presence of medical science liaisons providing targeted messaging to pave the way for treatment protocols into the future.  The level of work being done today, which is surely indicative of the future requirements of the position, is more consultative than it is selling work.  The further the role moves in the direction of consulting, where expertise and counsel are the primary services or products provided to customers, the more difficult it will be to classify rep jobs as non-exempt.

Interesting that this case was settled in the current marketplace in the manner that it was.  Pharma companies are going to need to redefine the rep’s accountabilities, given all the complexity their people are facing today, and to reposition the rep’s defined efforts from purveyors of product to business consultants.   And those reach and frequency models will need to fall by the wayside, too, since they really do not apply to what most high level sales reps are doing today.

I believe this is an indicator of more change coming in the pharmaceutical industry. We will be seeing different sales models, new ways of getting information out to the medical community, and providing value added processes to help offices gain access to treatments for patients.

Advice for Job Seekers

February 5th, 2009 by The ECI Team

Tough times out there for job hunters and I am more than sympathetic.  Ted told me yesterday that the economy is really driven on consumer purchasing.  When people are not afraid, they spend, whether they have the money or not.   This is because they believe that tomorrow will be a better day.  Today, most people are wondering about that one and many are worried that things will be so different that they won’t be able to survive.

As a caveat, I personally believe that the US government should allow people to collect a paycheck in a job and augment that with partial unemployment payments to yield a higher, livable wage, to encourage people to get working again.  Most of the entrepreneurs in the job market have taken a couple of part-time or full time minimum wage jobs to yield the same results.  To my mind, I would rather see 2 people working, rather than one person trying to make ends meet by working a 16-hour day. Read more…

Handling Problem Workers

January 5th, 2009 by The ECI Team

My sister was an executive vice president in banking.  She decided to retire when she saw the sub-prime market starting to go crazy and realized that the way she believed a book of mortgages should be developed and managed differed substantially from others in the industry.  Too bad she didn’t stick it out.   While my sister is a very strong person and does have a strong sense of justice, I doubt that she would have been able to overcome her high level of correctness to shame the industry into taking the higher road.   Nonetheless, she was a very effective leader in her own right and we often discussed how she was managing some of the issues she faced in overseeing her team. Read more…

International Hiring Processes

December 15th, 2008 by The ECI Team

I just returned from a trip to Europe where during our mealtimes we ate with people from a variety of countries.  Of course, as always occurs, people ask “what do you do?”.  When I told them that our business was providing psychological testing to help companies hire, manage and develop high performers , the discussion turned to how the selection process differs from one country to the next.

Our French Canadians were quick to point out that the process has to be completed in French and that documentation needed to be placed in the personnel files in French.  Apparently psychological assessment has been used for quite some time in Canada and is not viewed as particularly earth shattering. Both of the individuals were government employees who resided in Quebec.  Read more…

« Go Back
February 2012
S M T W T F S
« Aug    
 1234
567891011
12131415161718
19202122232425
26272829  

Twitter Feeds

Employer Consultancy, Inc. is an Organizational Development consulting firm whose corporate mission is to help companies to do a successful pre employement assessment, and manage and develop top performers. They accomplish this by providing their customers with practical, customizable tools and systems, such their competency management systems, that promote higher levels of performance, productivity and profitability.