How Client-Imposed Audits, a Workflow-Based Training Framework, And Usage Analytics Can Drive a Proficiency Initiative
By Justin Hectus
Over the last decade, technology has at least conceptually moved from a supporting role to a central role in the legal industry. Law firms market their technology prowess and clients demand investments in systems and expertise to leverage technology on their behalf. It won’t surprise any trainers out there, however, to read that law firms continue to face difficulty maximizing effective user behavior, mandating adherence to policies, and even encouraging attendance at training. That difficulty seems to be particularly apparent among attorney users. Client pressures on rates, proliferation of alternative fee arrangements, commoditization of legal services by alternative service providers, mandates by the Bar, and the fear of being called out by a client or a judge for a lack of competence in this area all encourage acceptance of the need for change, but making that happen in a profound way at an institutional level is a challenge.
The 2011 New York State Bar Association Report on the Task Force on the Future of the Legal Profession framed that challenge nicely, saying that we need to “redesign the way that we work so that technology is fully integrated into our workflow in an efficient and effective manner.” Law firm management has been catching the drift, and 81% of the respondents in a 2013 PwC Law Firm Survey said their top business and support priority for 2014 and beyond was to improve the use of technology. With 80% of respondents in the same survey identifying their top priority in the current year as the need to implement or upgrade IT systems, you can extract some sense of the shift happening here.
Behavioral change in the workplace inevitably begins in earnest when there is tension: a push and a pull, a carrot and a stick, some honey and … well, you get the idea. This has also been the case with legal technology adoption and training.
Every Movement Begins With a Spark
Client audits, which have been focused on security and risk assessment in recent years, provide value on a number of levels. In addition to providing an expert to evaluate vulnerabilities and provide a roadmap for shoring up any deficiencies (usually at no cost), the client component brings with it immediate acceptance to policies and procedures that may have been met with pitchforks and torches if promoted by IT on its own. An IT director saying, “please don’t use file sharing websites on your work computer” is marginally effective, but tying that to a mandate from our largest client and a set of software controls is met with complete and total acceptance.
The risk audit framework with which we have become so familiar seems a likely precursor for workflow and skills assessments. We haven’t been subjected to one yet, but our firm has clients that include the right to do so in their billing guidelines and RFPs, and we have followed the evolution of the Suffolk/Flaherty Audit with great interest. For those not familiar, Kia Motors In-House Counsel Casey Flaherty made waves in 2012 when he announced that he began auditing basic technology skills among his outside firms and required reductions in rates when all nine of the firms failed the tests. Flaherty then turned an initiative into a cause célèbre by announcing that he was working on automating the audit so he can give it away to law students and general counsel at other companies. The audit covered basic proficiency in Word, Excel and Acrobat Pro (as well as a few skills which are arguably beyond “proficiency”). Flaherty became the secret hero of law firm trainers everywhere, providing validation to what they knew all along: technology proficiency impacts efficiency and it is a skill that lawyers need to master in order to provide value.
Two years later, the audit has been automated and is available online (www.legaltechaudit.com) for $250 per year per user (the audit prep is an additional $150 per user). Despite the shift from the ideal of an open source application available to all for the greater good to a monetized product, Flaherty may yet succeed in his lofty goals of promoting efficiency throughout the legal industry. The test is well-designed and shortly after the official launch, several law firms expressed interest as early adopters and at least one Fortune 500 company is evaluating the audit as part of their outside counsel vetting process.
A Road Map of the Wider World
Developed with similar objectives and a wider scope than point-in-time skills assessment audits is the Legal Technology Core Competencies Certification Coalition (LTC4; www.ltc4.org) workflow-based training and certification framework. LTC4 was founded in 2010 by the training and change-management gurus at Capensys, along with a number of leading law firms. The movement has grown into a nonprofit coalition of 80+ U.S., UK and Canadian law firms and vendors who work together to create and maintain practical, actionable, real-world workflows that are designed to improve legal practice efficiency via better use of technology. The core competencies are designed to be application agnostic and are authored by “pods” of subject matter experts spanning multiple law firms and then peer-reviewed before publication.
Workflows currently available to LTC4 members include: Managing Documents and E-mails; Time and Billing; Working with Clients (CRM); Presentations; and Legal Documents. Workflows under development include New Business Intake and Conflicts Checking and e-Discovery. Membership in LTC4 begins at $1,500 per year and scales up, depending on the size of the firm, and offers access all of the workflows in addition to the opportunity to participate in the development of these standards. Full participation in LTC4 will also lead to an industry recognized certification.
Both the specter of client-mandated skills audits and the LTC4 framework are important tools in the creation or solidification of a learning environment in any firm, providing the tension that encourages dialogue and engagement. Clients will be asking for validation of proficiency in one form or another, and the fear of taking a test which will be reported to a client is enough to encourage anyone to participate in training. The LTC4 framework provides a roadmap to not only prepare any user for such an audit, but it offers a practical guide for onboarding new associates and staffers and for re-tooling skills around the necessary activities rather than software features. At Keesal, Young & Logan (KYL), we are preparing for audits and participating in the LTC4 movement at the same time and we are using a few twists that we hope may be instructive or inspirational to anyone interested in following suit.
The KYL Approach: Tailored Through Objective Data Analysis
One of the distinctive differences of IT planning and execution at KYL is our approach to user behavior analysis. We begin any significant project by looking at what users are doing or not doing, and then trying to make informed decisions about how to best move the needle. This approach was part of what catapulted our training program to an ILTA Distinguished Peer Award in 2011 and it was the singled out by David Freedman in Inc. magazine in 2005 and in his book “The Hidden Benefits of Disorder” in 2007. Investing time studying usage and behavior saves time on implementation and it increases credibility because opinions are formed based on facts rather than perceptions (or misperceptions). Throughout this process we experiment and, if something doesn’t stick, we adjust and refine our approach.
I know some firms have had success by making a shift to mandatory training, but we felt that wasn’t quite enough. We needed to make it clear to users that we spent time getting to know them, their usage patterns, their strengths and weaknesses so that we could provide precision support rather than just a broad brush.
Starting simple, we looked at application usage or usage of features to create benchmarks and goals. Regular KM usage at our firm was a reasonably high 74% among associates, but our goal was 100% and by reporting on and publicizing usage, we are on our way to meeting that goal. Looking at Carpe Diem and Carpe Diem Mobile usage, we were able to key in on a handful of features used extensively by our power users, but overlooked by the majority of our timekeepers. We set up five to 10 minute one-on-one sessions with those not using these features and helped those timekeepers and their secretaries quickly learn how to use and manage the features. Focusing on usage (or lack thereof) helps you prioritize and focus training so that you don’t waste time with the wrong audience.
Digging a little deeper, we spent some time auditing our DMS and looking at application usage and even keystrokes. Several patterns emerged that were helpful in multiple facets. First, we looked at the volume and frequency of Microsoft Excel usage by department. We could quickly isolate heavy users and focus advanced training on those folks and we could conversely isolate low-hanging fruit for basic training. An accounting user who rarely uses Excel or who doesn’t use advanced features, for example, would be a quick win for a focused session and a tailored follow up plan.
Next, when we looked at Microsoft Word keystrokes over the course of 12 months and compared that to a 12-month period a decade ago, we saw concrete evidence that associates are spending significantly more time drafting their own documents than in the past. Indeed, they surpassed their secretaries and, in some cases, attorneys spend more time in Word than support staff. These data points helped us reinforce the importance of mandatory advanced Word training and allowed us to focus our attention on the priority of validating or increasing Word proficiency among those heavy users. It also spawned some interesting debates regarding the pros and cons of dictating versus typing, which ultimately led to a speech-to-text pilot program.
Peeling the onion yet one layer further, we ran queries on the database to identify pleadings that had extensive edits isolated to individual timekeeper authors who were previously identified as heavy users. The result was a set of pure samples of the timekeeper’s usage of styles, numbering, cross-references, etc. These samples were instrumental in developing personalized training plans and in providing illustrations of the reality of what needs work or what is being done well. An ideal trainer develops a relationship based on trust, and trust is rooted in a common reality. Being able to say, “I’m pulling up your document. Do you see what you did here? It would be more efficient for you to use this alternate approach,” in a nurturing way eliminates any accusation and any defense. With the barriers removed and the defenses down, we can make some real progress.
We broke training sessions down into 10-15 minute desk-side sessions showing the user what would stand out in a skills assessment audit in their own documents, and we reviewed the tools designed to increase efficiency. We follow up every session with an e-mail cheat sheet targeted at the specific skill as well as a five to 10 minute check-up a week or two later after we look at recently edited documents to confirm that the new skills are being utilized. Also of importance, our trainer would go in with a few additional tidbits (mobile time entry, better KM searching, etc.), which could be thrown into the mix if the opportunity presented itself. These tidbits are selected and targeted based on individual usage pulled from those respective systems. So, even a drop in saying “hey, just wanted to check in to say that I noticed that you are using the discovery headings toolbar — I hope that has been helpful and it is really going to help us with any Word efficiency audit” could be coupled with a message like, “we’re also trying to get greater value out of the firm’s investment in Carpe Diem mobile; if you have five minutes, I can show you how easy that is to use.”
One added byproduct is also worth mentioning. Mining the DMS and our scanning and calendaring systems has also helped us gauge secretarial workload and adherence to policies. It’s a relatively simple exercise and provides a big-picture overview with the ability to drill down as desired.
This approach may be tough to methodically replicate over the entire application suite, but for the low hanging fruit, it works great and the targeted sessions seem to whet the appetite rather than exhaust the user. Further, it really lays the foundation for continued dialogue in a natural give-and-take format, rather than a one-directional classroom style training.
Our next step is to focus usage analysis on our e-discovery applications. We have a few predictions on the end result, but, if the first phase proves instructive, the data we gather will certainly refine our perspective and shape our approach. Peer training from our e-discovery experts on staff will likely be a central component of that project.
Conclusion
Objective statistics confirm that attorneys are doing more front-line work with technology than ever before. To remain competitive, firms must face this shift in workflow directly. We have been working diligently for years at KYL on developing a community of technology experts to ensure that we provide value at every step. It is neither an overnight process, nor one that has a clear finish line but, today more than ever, I feel encouraged by the tools and information available to us in support of these efforts.
Justin Hectus is a member of this newsletter’s Board of Editors and the CIO of Keesal, Young & Logan (www.kyl.com), an 80-lawyer firm operating in five offices around the Pacific Rim and an LTC4 member firm. In August 2014, Justin received his second Distinguished Peer Award from the International Legal Technology Association (ILTA) when he was named “IT Professional of the Year.”
Reprinted from LJN’s Legal Tech Newsletter (www.ljnonline.com/ljn_legaltech)