Happy new year to everyone! The beginning of a new year brings excitement about what lies ahead. It's a time to look forward and make plans for the coming months. If any of those future plans involve continuation of current programs or activities, it is a good idea to take stock of how things are working before you go any further. Evaluating current programs is essential for deciding what to continue, change, or drop as you move ahead. This issue of Management Solutions examines the utility and value of program evaluation. The first section explains how to evaluate a mentoring program, which requires assessing both the overall program and the personal experience of program participants. The second section discusses the diversity initiative of the Bar Association of San Francisco as a case study exemplifying the importance and efficacy of ongoing program evaluation.
This issue also addresses the increasing pressure to provide flexible work arrangements for lawyers. Some legal employers worry that widespread availability of flexible scheduling will lead to overwhelming demand for alternative schedules, leading to excessive expense and management headaches. Two recent reports show not only that this concern may be unfounded, but that greater flexibility is positive for the bottom line.
I will be writing a column this year on mentoring and diversity in Diversity & the Bar magazine. Below are links to that journal, to one of my recently published articles dealing with generational conflict, and to conferences where I will be speaking this quarter.
January 2006 is National Mentoring Month. This is a campaign intended to recruit volunteer mentors for young people with the message, "Share What You Know. Mentor A Child." While the campaign spotlights mentoring for youth, it is a reminder of the importance of sharing what we have learned so that others may benefit. There are opportunities all around you at work and in your community. When you make your resolutions for the new year, why not commit to being a mentor?
Programs that promote mentoring should be regularly evaluated to determine whether they are proceeding as planned and achieving their intended objectives. The evaluation process begins by looking at the program's intended outcomes and targeted goals. It then measures progress, achievements, and effectiveness against those outcomes and goals. That is why it is essential to state mentoring program objectives when the program begins in a way that allows you to measure progress and results once it is under way.
Evaluations should be done at least at the program's mid-point and upon completion of a program cycle. Interim evaluations measure the process: Are people doing what they are expected to do? Are they on track to achieve targeted goals? Should any changes be implemented immediately to improve the mentoring experience for program participants? Evaluation at the conclusion of a program cycle measures results, i.e., to what extent program objectives were achieved and the program's overall impact. Because many mentoring programs have goals that will take years to achieve, evaluations should analyze both short-term results from one mentoring cycle and long-term results over several cycles.
Measuring the success of a mentoring program requires evaluating both the overall program and individual mentoring relationships.
Evaluating the Overall Mentoring Program
Mentoring program evaluations are conducted for several purposes: to measure the program's effectiveness, improve the program, and/or demonstrate that the program is a cost-effective use of the firm's resources. Evaluation is critical for making decisions about whether to continue, change, expand, or curtail the mentoring program. Questions to ask in the program evaluation include:
The evaluation process should focus on particular indicators of success, i.e., the factors that will indicate whether specific program goals were achieved. Indicators should be realistically linked to the goals they seek to measure, although this may be difficult. For example, one common mentoring program objective is to increase associate retention. But the rate of retention is influenced by many factors in addition to mentoring: management practices (e.g., work assignment systems), events (e.g., economic downturn, merger), and firm culture. The evaluation process should therefore measure specific factors that link mentoring with increased retention, e.g., associates reporting that the mentoring program has influenced their decision to stay with the firm. Depending on a program's goals, indicators could include:
Indicators form the basis for collecting evaluation data. The most common tools for collecting this data are interviews, focus groups, and surveys of participants and observers (e.g., supervisors). These tools can be used to establish a baseline, record interim progress, and assess final results. Baseline data should be collected as part of the initial needs assessment or planning process. If, for example, one of the goals of a formal mentoring program is to increase associate satisfaction, a preliminary survey about associate satisfaction with informal mentoring can produce the baseline. At the conclusion of the program cycle, a survey of associate satisfaction with formal mentoring provides the comparison data that determines whether the goal was achieved.
Once a program is up and running, collecting and analyzing evaluation data – and comparing it to baseline data – helps you measure the quality of the program design, progress toward program goals, and levels of accomplishment. If the data show that certain aspects of the program are especially valuable, that aspect of the program should be highlighted in subsequent cycles. For each interim indicator or final result that is less than optimal, the data analysis should lead to development of specific new targets and tactics for program improvement.
Evaluating Mentoring Relationships
The purpose of mentoring is to promote professional growth through a personal relationship. Successful mentoring relationships are measured by two criteria: the quality of personal affiliation and the degree of development. In a relationship with a high level of affiliation, both parties relate well on a personal level, with the mentor providing care and guidance, and the mentee appreciating and utilizing the mentor's support. In a relationship where the quality of development is high, both parties learn from one another, with the mentee progressing toward higher levels of understanding, expertise, and career advancement. Successful mentoring relationships rate high on both factors. High affiliation with little development may be enjoyable but produce little professional growth. High development with little affiliation is tutoring, not mentoring.
Evaluation of the mentoring relationship should measure the mentor's and mentee's levels of affiliation and development. The evaluation of affiliation examines the mentor's and mentee's experiences in the relationship, how well the two got along, and how effectively they were able to meet each other's interpersonal expectations. Measuring development requires a baseline that allows you to assess the degree of change. Although the mentor should also benefit and learn from the relationship, most developmental change will happen to the mentee. Various aspects of the mentee's progress can be assessed, including:
Methods of measuring mentees' development include pre-, mid- and post-program surveys, interviews, self-assessments, and assessments by observers (e.g., supervisors). An additional approach is to use competencies, benchmarks, or other performance standards that describe levels of expected or acceptable performance at particular stages of lawyer development. A mentee's progress can be measured against those standards.
The principles for evaluating mentoring programs apply to other types of programs as well. A good example of the value and importance of continuous program assessment is demonstrated by the successful diversity initiative of the Bar Association of San Francisco (BASF). For 15 years, BASF has been setting and re-setting goals for diversity within the Bay Area legal community. In 1989, BASF adopted and published Goals and Timetables for Minority Hiring and Advancement, which established goals and timetables for hiring and advancement of minority lawyers. More than 100 Bay Area legal employers adopted the goals and timetables, and BASF has published interim reports in 1993, 1995, 1999, and 2005 that measure the results of these ongoing efforts. By setting specific goals and monitoring progress toward them, BASF has been able to demonstrate its achievements and at the same time, identify areas where further effort is needed. New initiatives are added or modified depending on what the interim results show.
According to the 2005 Interim Report, the percentage of minority lawyers in San Francisco firms in 2005 is significantly larger than in 1990 and exceeds the 2005 national average. (See Table) However, BASF members failed to meet the targets that they established for 2005: 35% for minority associates and 12% for minority partners. Analyzing the reasons for this gap led to the realization that the low number of minority students interested in law and able to enter the legal profession remains one of the most persistent obstacles to achieving diversity. Therefore, as it moves ahead, BASF is placing greater emphasis on "filling the pipeline," i.e., increasing the number of future minority lawyers by focusing on minority students in high school, college and law school. To do this, BASF is continuing and expanding scholarships, mentoring programs, and school-to-college initiatives. In addition, BASF will collaborate with the State Bar of California to develop best practices and curricula for students as young as the sixth grade in an effort to interest more minority students in the field of law.
Table. Percentages of Minority Lawyers in Law Firms and Legal Departments
Junior Corporate Counsel
Constant monitoring and evaluation enable BASF and Bay Area legal employers to recognize and take pride in their accomplishments and to better target future efforts. For a more complete discussion of the BASF diversity initiative and the entire 2005 Interim Report, see www.sfbar.org/diversity.
Numerous studies over the last few years have found that employees, including lawyers, are placing greater and greater priority on having time away from work for family or personal pursuits. Two recent studies show that programs and policies that support work-life interests and flexible scheduling are highly valued by employees, and that they can have a positive impact on employers, including economic savings from lower turnover and more efficient utilization of resources.
Study 1: The New Workforce Reality
The New Workforce Reality, a national study of employee turnover by Simmons School of Management and Bright Horizons Family Solutions Inc., found that 95% of 2,000 survey respondents rated time away from work as equally or more important than their job. There was no significant difference between men and women in this finding. Of particular interest is the observation that individuals who are "dual-centric," i.e., who place equal priority on work and personal time, feel more satisfied and engaged, and less stressed, both at work and in their personal lives. These findings are consistent with several other recent studies suggesting that programs and policies that support flexibility and work-life accommodation enhance employee satisfaction, productivity and work commitment. The full report is available at http://www.simmons.edu/som/docs/centers/workforce_reality_study.pdf
Study 2: Flexible Work Arrangements in Europe
A recent "issue brief" published by The Center for Work Life Law at Hastings College of the Law reaffirms that flexible scheduling is good for employers as well as employees. Repeated studies have shown that very few lawyers (3-4%) take advantage of available part-time policies, despite the widespread availability of such policies in law firms. Still, legal employers often claim that alternative work schedules will be overly disruptive and expensive because large numbers of lawyers will choose this option.
The issue brief reported that even when available by law, the number of employee requests for flexible scheduling remained modest and most were easily accommodated. The author examined the experience of employers in the United Kingdom, Germany and the Netherlands. Each of these countries has enacted statutes requiring employers to be favorably receptive to employee requests for flexible work arrangements, as long as the employer's operational and business needs are not unduly affected. The report finds that in all three countries, employee requests for flexible scheduling were manageable and easy to accommodate. Most employers received only one to five requests. Costs associated with these requests were not significant, and in fact, some employers actually realized savings. The issue brief and a synopsis of the referenced European flexibility laws can be found at http://www.uchastings.edu/?pid=3624
My article, "Tips for Managing the Entitlement Mentality: Battle of the Generations," is featured in the December 2005 issue of the ABA's Law Practice Magazine. It is available at http://www.abanet.org/lpm/magazine/articles/v31is8an3.html.
I will be writing a column this year entitled "Mentoring Across Differences," for Diversity & the Bar magazine. My first column will appear in the January/February 2006 issue. You can read this journal and my column online at www.mcca.com.
I will be serving as faculty in two conferences this quarter, both in Chicago: