CORONAVIRUS (COVID-19) RESOURCE CENTER Read More

Performance Management Through Use Of Cascading Scorecards In A Centralized Performance Management System

State: FL Type: Model Practice Year: 2016

LHD Description:  Broward County is located in the southeastern portion of the State of Florida with Miami-Dade County to the south and Palm Beach County to the north. Broward County’s 2014 population estimate of 1,869,235, represents 9% of the State’s population, and is the second most populous county of the 67 counties in the State of Florida and eighteenth most populous county in the United States (US Census).  31.5% of the residents are foreign-born. Broward County is a minority/majority county demonstrated by its 2014 population by race (Black 28.8%, Asian 3.6%, Hispanic 27.4%, other races .5%, for a total of 60.3% and White 39.7%). The Florida Department of Health in Broward County (DOH-Broward) is the official Public Health Agency in Broward County and has been operational since 1936 and provides services to the county’s 1.8 million residents and over 10 million annual visitors. DOH-Broward currently employs a staff of approximately 600 and has an annual budget in excess of $44 million. The public health issue is that historically there was no central Public Health performance management framework that incorporates community health, process and financial metrics in a uniform reporting structure at multiple organizational levels. During the 2008 DOH-Broward strategic planning cycle, the lack of a centralized performance management system to measure, analyze and improve organizational performance was identified. Essential Public Health Service number 9 is to evaluate the effectiveness, accessibility, and quality of personal and population-based health services. Goals and Objectives:  In order to achieve these requirements the following goals and objectives were established by the Organizational Development Team: 1) Deploying a central performance management system across all programs and areas; 2) Creating cascading scorecards from the organizational level down to the employee level; 3) Creating metrics for employees that are linked to their performance evaluations; and, 4) Establishing a comprehensive multi-level scorecard business review process. In January 2012, Organizational Development made a recommendation to the Senior Leadership Team to purchase ActiveStrategy Enterprise software. This software was the only application that met all the objectives. DOH-Broward has implemented through this system performance/productivity standards, trends identification, benchmarks establishment, and performance variance tracking which are necessary for systematic performance improvement. The practice was implemented with the key activities that included: ·         April 2012 ActiveStrategy Enterprise software was purchased. ·         A core team was identified to develop and administer training. ·         The initial scorecard cascade methodology was established. ·         Meetings were conducted with Senior and Program Leaders to identify key metrics. ·         System deployment and user trainings. ·         The 1st business review meeting utilizing ActiveStrategy metrics was conducted on February 2014 utilizing data collected since June 2013. Results:  Stakeholder commitment is tied to the performance management system and process by the alignment and linkage of individual and programmatic performance metrics to the employee evaluation. Employees and programs that achieve outstanding performance are recognized at the annual employee conference. Regular monthly multi-level business reviews at different levels of the organization establish accountability for performance reporting and organizational performance is evaluated. This has led the organization to achieve over 89% of the metrics meeting target. All 4 objectives were met by establishing over 60 programmatic, 8 divisional and 4 director level scorecards encompassing more than18,520 metrics. The metrics were refined and five to seven metrics critical for success were selected for each program. The scorecards include indicators that are linked to employee’s SMART metrics on their performance expectations. Multi- level business reviews are conducted monthly. Success Factors:  1) Senior Leaders commitment to the practice of data driven performance management; 2) the creation of a committee that facilitated the deployment of the application; and 3) A clear understanding of each metric and its relationship to both community and organizational objectives by the program owners; 4) Employee engagement to the system by linking metrics to the employee performance evaluation. Public Health Impact:  A clear understanding of each metric and its relationship to organizational objectives by the program owners was imperative. The impact of deploying the cascading scorecard system has increased the accountability and the performance of each program. Prior to the deployment of the system less than 50% of the identified program metrics were submitted on time. As of August 2015, the submission rate is at 93%. Each metric owner is aware and accountable for their role and impact on the organization. In addition, 89% of all metrics are meeting target, increasing the achievement of community health, improving public health outcomes. Website: http://broward.floridahealth.gov/  
Public Health Issue:  The public health issue is that historically there was no central Public Health performance management framework that incorporates community health, process and financial metrics in a uniform reporting structure at multiple organizational levels. During the 2008 DOH-Broward strategic planning cycle, the lack of a centralized performance management system to measure, analyze and improve organizational performance was identified. One of the ten Essential Public Health Services is to evaluate the effectiveness, accessibility, and quality of personal and population-based health services. This includes objectives; need to implement quality programs, performance management systems, ongoing evaluation to examine personal health services, population based services, and the public health system.  What Target Population Is Affected By The Problem?:  The identified target population included 98 leadership positions and 551 employees of the Florida Department of Health in Broward County. The review of the performance metrics that include community health outcomes, ultimately affects 1.8 million resident and ten million annual visitors in the county. Percent Did You Reach?:  The percent of employees reached that have individual performance metrics in ActiveStrategy and are aligned to a programmatic or higher level scorecard is 100%. There are over 60 programmatic, 8 divisional and 4 director level scorecards with more than 18,520 metrics in the system. ActiveStrategy includes indicators that are linked to employee’s SMART metrics on their performance expectations. SMART metrics are Specific, Measureable, Achievable, Relevant, and Timely. By monitoring metrics on a monthly basis down to the individual employee level DOH Broward is affecting health outcomes for all residents and visitors in Broward County. What Has Been Done In The Past To Address The Problem?  In the past, the DOH-Broward had a fragmented system with minimal accountability among data owners. The DOH-Broward was previously using a manual Microsoft Excel spreadsheet to capture monthly metrics for several programs. The spreadsheet did not include scorecards by programs nor did any metrics rollup or cascade to other scorecards. A comprehensive multi-level business review process was not systematically conducted. Alignment of programmatic performance metrics to core business process was not in place. Why Is Current Practice Better?:  DOH-Broward assures metrics are aligned to the objectives in the Broward County Community Health Improvement Plan, the Florida Department of Health Strategic Plan, the State Health Improvement Plan, and meets the indicator goals on the CHD Administrative Snapshot, County Performance Snapshot, and the County CHD Dashboard that is in alignment with public health objectives. Through the development and implementation of a comprehensive performance management system allows continuous monitoring of organizational performance. The use of metrics is necessary to manage processes at all levels using a system of cascading scorecards. To ensure accountability, top level metrics cascade down to employee metrics which are measured and linked to their employee performance evaluations using SMART goals. Is Current Practice Innovative? How?  The current practice is innovative and new to public health through the use of varied public health data sets in a performance management system. DOH-Broward is able to continuously track metrics necessary to manage the processes in the organization, improve organizational performance, and improve public health outcomes. In researching performance management systems, DOH Broward was unable to identify any local public health departments that had a centralized performance management system in place. In response to this concern, DOH-Broward decided to develop our own public health performance management system. The centerpiece of the performance management system is ActiveStrategy, a cascading scorecard based data system that allows DOH-Broward to record, monitor, trend, benchmark, and communicate performance data to employees, grantees, other agencies and the community. DOH-Broward is the only health department in the United States utilizing the ActiveStrategy software application to monitor the performance of over 18,520 metrics that was developed internally based on public health issues ranging from the highest organizational level to the individual employee.  The performance of organizational, division, program, and site/unit level metrics are reviewed at four separate business reviews that correspond to each organizational level and facilitates cross-organizational collaboration and performance improvement. Performance reviews meetings are held to report on findings, shared best practices and to analyze performance. Leaders identify underperforming metrics and make necessary adjustments to respond rapidly to changing organizational needs and challenges of programs and of the organization. Is Current Practice Evidence Based?  The practice is evidence based. The practice of measuring health performance through the use of metrics and cascading scorecards as an approach is not unique to the public health sector but is not systematically utilized in the public health sector. Several third party evaluation/accreditation systems such as the Malcom Baldrige Framework, and the Public Health Accreditation Board Standards and Measures version 1.5, Domain 9 require the use and integration of a performance management system to monitor achievement of organizational objectives. The Baldrige Criteria for Performance Excellence category 2, 4 and 7 are all linked to the creation and measurement of organizational data and results that are found only in organizations that are engaged actively in monitoring their metrics at all levels of the organization. Baldrige categories 1, 3, 5 and 6 also are results oriented and depict a national tool to emphasize organizational excellence, reviewing practices, and seeking opportunities for improvement. URL: http://www.nist.gov/baldrige Other references on how cascading scorecards  are evidence based include the Harvard Business Review articles written by Robert Kaplan and David Norton, “The Balanced Scorecard-Measures That Drive Performance” (1992), “Putting The Balanced Scorecard To Work” (1993), “The Balanced Scorecard: Translating Strategy Into Action” (1996).  
Goals And Objective Of The Practice:  The goals of the practice was to identify, develop and implement a performance management system with the following objectives: 1) deploy the performance management system across all programs and areas 2) create cascading scorecards from the top level down to the employee level 3) create metrics for employees that are linked to their performance evaluations 4) establish a comprehensive multi-level scorecard business review process What Did You Do To Achieve The Goals And Objectives?:  Several steps were taken to implement the program. Objective 1) Deploy a performance management system across all programs and areas. The 2008 DOH-Broward Strategic Plan identified that the organization lacked a centralized comprehensive performance management system to measure, analyze and improve organizational performance. The organization had a fragmented system with minimal accountability and data owners.  DOH-Broward was previously using a Microsoft Excel spreadsheet to capture monthly metrics for several programs. The spreadsheet did not include scorecards by programs nor did any metrics rollup or cascade to other scorecards. A comprehensive multi-level business review process was not systematically conducted. Alignment of programmatic performance indicators to core business indicators was not in place. The Director of Organizational Development researched numerous commercial performance management systems. During monthly meetings with the Senior Leadership staff he presented his findings. In January 2012 the Senior Leadership team decided to institute an effective process to measure their performance with a proven performance management system. After careful consideration, ActiveStrategy Enterprise application was selected in April 2012 as our performance management system for the following reasons: any user can view most up-to-date information tailored to their needs; minimized the need to collect manual data; format spreadsheets; distribute reports; brings relevant data from all existing sources; provides strategic context; and transforms data points into actionable information; track historical information so that trends can be identified; ability to be used during a “live” business review and any information can be viewed and printed as a PDF; drive organizational alignment through communication of strategic plans, setting expectations; identifying responsibilities; achieving goals; creating a cascading framework that aligns programmatic metrics and scorecards to organizational performance; goals; objectives; and initiatives. ActiveStrategy also is used to manage cross functional metrics and scorecards, understand cause and effect relationships, tie strategic goals to operations, align the employees to the strategy of the organization, and clarifies who is responsible at every level of the organization. DOH-Broward contacted ActiveStrategy and discussed the overview of the system, balance scorecards concept, pricing, workshops and deployment of the system. A DOH-Broward ActiveStrategy facilitation team was created to facilitate the deployment of the application that consisted of the Director of Organizational Development, Director of Organizational Performance Metrics, Director of Performance Excellence, IT Director, IT Helpdesk Supervisor, and the Director of Workforce Development. The team was initially trained by the Active Strategy trainers and reviewed all existing organizational metrics. The team created a framework in ActiveStrategy that consisted of the creation cascading scorecards consisting of divisions, programs and sites/units. The existing metrics in the Excel spreadsheet were entered in the system by division and program with their targets and descriptions. Initial and subsequent meetings were conducted with Senior Leaders, Program Managers, Supervisors to update or refine metrics with targets and descriptions. The metrics were refined and the five to seven metrics for success were selected. Each senior leader along with their program managers and supervisors were included in each meeting. Programmatic reports from their areas and from their stakeholders were reviewed and used to update the divisional and programmatic metrics and establish targets. New indicators were added and unnecessary indicators were removed. Sources for the indicators are derived from indicators monitored by Federal Programs such as the Refugee Health Assessment Program, Tuberculosis Program or the WIC/Nutrition program, program bureaus from the Department of Health in Tallahassee such as Immunization, community assessments such as the Community Health Assessment, and internal processes such as Fleet Management or Custodial Services. Trainings of program managers, managers, and supervisors were conducted to familiarize them with the system and to enter the historical data. The continuous monitoring of the metrics by the team and the initiation of business reviews led to increase awareness at all levels that the organization was managing processes with data. The next step in innovation was to hire a programmer to automate the data entry of the metrics and pull data into ActiveStrategy from other sources such as Excel spreadsheets and multiple data bases. There are two indicators that measure the results of the performance management system, 1) Scorecard Metrics Submitted by Due Date and 2) Scorecard Metrics that Met Target. The Scorecard Metrics Submitted by Due Date increased from less than 50% in June 2012 to 98% in August 2015. The Scorecard Metrics that Met Target increased from 78.8% in July 2014 to 89% in August 2015. 100% of DOH-Broward employees have individual metrics that are aligned to a programmatic indicators or high level scorecards. ActiveStrategy is utilized by some of the largest and most distinguished national and international organizations such as Life Lock, the Coca-Cola Company, Kaiser Permanente, NATO, and Westinghouse.  DOH-Broward is the only health department in the United States utilizing the ActiveStrategy software application to monitor the performance of more than 18,520 metrics ranging from the highest organizational level to the individual employee. Objective 2) Create cascading scorecards from the top level down to the employee level. DOH-Broward used the centerpiece of the performance management system, ActiveStrategy, as a means to create a cascading scorecard based data system to record, monitor, trend, benchmark, and communicate performance data to all employees.  Scorecards were created at the organization, division, program, and site/unit/employee level. The organization created over 60 programmatic, 8 divisional and 4 director level scorecards with more than 18,520 metrics in the system. Several programmatic metrics roll up to organizational indicators. Employee metrics roll up to the programmatic metrics and are aggregated to the overall performance of the metrics. This segmentation of performance is also cascaded from the main metric down to the site/unit and to the employee. Targets are used to assess gaps, monitor, evaluate performance monthly and annually. The application includes indicators that are linked to employee’s SMART metrics on their performance expectations. Objective 3) Create metrics for employees that are linked to their performance evaluations. Leaders at various levels of the organization create employee SMART metrics which are shared and reviewed with their employees. These metrics are based on the program and employee activities. The SMART measures are included in the employee performance evaluation and in the performance management system. Objective 4) Establish a comprehensive multi-level scorecard business review process. The performance of organizational, division, program, and site/unit level metrics are reviewed at four separate business reviews that correspond to each organizational level and facilitates cross-organizational collaboration and performance improvement. Performance reviews are used to report on findings and to analyze performance. Senior Leaders use the reviews to respond rapidly to changing organizational needs and challenges of their programs and the organization.  Employee performance is measured at various levels in the metrics and is linked to the SMART expectations in their employee performance evaluation. With the implementation of the performance management system, conducting business reviews and including the SMART objectives in the employee evaluations that roll up to the programmatic indicators, the organization has seen significant improvement in the results of several key organizational metrics. Any Criteria For Who Was Selected To Receive The Practice:  1) Any leader in the organization responsible for managing processes. 2) Staff responsible for implementing those processes. 3) External stakeholders who have an interest in the outcome of those processes. The criteria that were used to deploy the practice included reviewing all program metrics to identify any existing and potential metrics for measurement starting with the current programmatic metrics that were utilized and reviewing them for relevance. Metrics had to be high level, relevant to the program and activity. Metric had to be measurable and with a method of measurement. New metrics were added and old ones that were deleted.  Metric were reviewed for targets and trending data by program and later by employee. Divisional scorecards were created from the metrics as were programmatic scorecards. Employee SMART performance expectation metrics were created from the programmatic metrics. What Was The Timeframe For The Practice?:  The practice was deployed over a two year period starting in January 2012 and deployed across the organization in June 2013. The practice is still being updated and comprehensive reviews of the metrics for targets and data are still being conducted. Were Other Stakeholders Involved?  Besides the stakeholders that were involved in the planning and implementation of the system, additional stakeholders included the programmatic supervisors and the individual employees where appropriate. Supervisors were included in review and development of programmatic metrics with their managers and later utilized to created SMART expectations for their employees. Those SMART expectations included measurement in four areas: 1) Quantity 2) Quality 3) Cost 4) Time These metrics are used to measure the performance of each individual utilizing the cascading scorecard system at the programmatic or divisional scorecard levels.  Linkages to the programmatic metrics were established during the creation of the new SMART expectations and are reviewed/revised during each evaluation period. What Does The LHD Do To Foster Collaboration With Community Stakeholders? Describe The Relationship And How It Furthers The Practice Goals?  DOH-Broward has a long and successful history in leading and organizing efforts to mobilize Broward County communities, key agencies and individuals into action to address public health issues.  DOH-Broward has innumerable relationships with partners in the community including healthcare partners, the faith community, educational institutions, governmental agencies, non-profit groups and other less traditional partners such as first responders and businesses. DOH-Broward has strong collaborative relationships with public and private health care providers, the Miccosukee and Seminole tribal nations, and serve on many committees, Boards and provider networks that work to address minority health and health disparities in Broward County including: the Coordinating Council of Broward, Broward  Healthy Start Coalition, Health Care Access, Primary Care Group, and Medical Services Planning. DOH-Broward works with its partners, on both a formal and informal basis, and is viewed as the central force that brings together all partners in public health to provide accessible, high quality health services within the community. These relationships assures that DOH-Broward objectives are aligned to the Broward County Community Health Improvement Plan, the Florida Department of Health Strategic Plan, the State Health Improvement Plan, and meets the indicator goals on the CHD Administrative Snapshot, County Performance Snapshot, and the County CHD Dashboard. These plans and dashboards contains metrics that are associated with healthcare partners, the faith community, educational institutions, governmental agencies, non-profit groups and other less traditional partners such as first responders and businesses. DOH-Broward has included in ActiveStrategy most of the indicators that are measured by these various groups in the community and government plans listed and linked them through cascading scorecards to programs or high level scorecards using targets and monitoring gaps in performance. Information is then used at various levels and presented to various community groups. This performance measurement system can respond to rapid, unexpected organizational and external changes in the community. Any Start Up or In-kind Costs And Funding Services Associated With This Practice?  The initial software cost was $60,000 with annual $10,000 license renewal fee. It is important to note that the total cost to this system is directly attributable to the size of DOH-Broward and the manner in which we configured the system. As a large metro CHD with over 600 employees and over 40 programs However, the system can be purchased for as little as $1,000, with a renewable subscription cost of $300 a year. Training and implementation was done internally. DOH-Broward Management Information System team installed software on staff computers and oversee the maintenance and updates of the system.  
What Did You Find Out?  The lack of a performance management system that encompasses all areas of the organization is extraordinary common in public health and makes it extraordinarily difficult to achieve any of the goals and objectives established by the leadership of the organization.  By implementing a standardized mechanism to evaluate performance based on metrics, the organization is able to objectively identify strengths and opportunities for improvement.  As a consequence to the alignment of the organization’s performance management system and each employee’s performance evaluation, reliance on anecdotal information is virtually eliminated.  This has provided the opportunity for employees to meet or exceed predetermined metric targets. In addition, by aggregating the performance of lower level scorecards to high level scorecards, a culture of shared accountability is created.  That is to say, managers are directly responsible for the performance of their staff, and employee performance determines the manager’s performance evaluation score.  Prior to implementing the performance management system, the organization was reliant on statewide systems that only reported data on an annual basis.  This did not provide an opportunity to make periodic adjustments throughout the year and led to a pass/fail process for meeting targets.  The segmentation of metrics into data points that are sampled more frequently than annually, and subsequent review at the monthly business review meetings, provide the organization the opportunity to make adjustments to underperforming metrics throughout the year as well as highlight metrics that exceed established targets. We know of no other way to realize systematic success in any of the CDC winnable battles without a comprehensive performance management system that speaks to every level in the organization. Without the use of a system of this type there is no quantitative way to determine how effectively these battles are being fought or won. Objective 1) Deploy a performance management system across all programs and areas. Objective 2) Create cascading scorecards from the top level down to the employee level. Objective 3) Create metrics for employees that are linked to their performance evaluations. Objective 4) Establish a comprehensive multi-level scorecard business review process. Did You Evaluate Your Practice?:  The practice is evaluated on yearly basis by both internal and external stakeholders to realign metrics with performance expectations based on changing grant requirements new initiatives and revisions to existing program processes. During the monthly business review meetings the metrics are reviewed for performance. Underperforming metrics are discussed with corrective action planning implemented. List Primary Data Sources - Who Collected The Data And How?: DOH Broward staff utilizes multiple statewide and national data bases to collect, compile, analyze, data to benchmark and set goals to determine performance levels at both organization and individual levels. Daily programmatic activities are captured into the systems listed below by internal and external stakeholders through direct data entry and electronic data transfer. My Florida Market Place (MFMP) – Accounts Payable Supervisor - ReportFinancial Information Resource System (FIRS) Accounts Payable Supervisor – ReportPRISMFLAIRPeoples FirstTRAINQ-FLOWFLORIDA WISEHealth Management SystemEMDEONFootPrintsHealthy People 2020FLCHARTSEHARSEnvironmental Health DatabaseMERLINESSENCE List Performance Measures Used: Based on analysis, DOH-Broward objectives are aligned to the Broward County Community Health Improvement Plan, the Florida Department of Health Strategic Plan, the State Health Improvement Plan, and meets the indicator goals on the CHD Administrative Snapshot, County Performance Snapshot, and the County CHD Dashboard. These plans and dashboards contains metrics that are associated with healthcare partners, the faith community, educational institutions, governmental agencies, non-profit groups and other less traditional partners such as first responders and businesses. DOH-Broward has included in ActiveStrategy most of the indicators that are measured by these various groups in the community and government plans listed and linked them through cascading scorecards to programs or high level scorecards using targets and monitoring gaps in performance. Information is then used at various levels and presented to various community groups. This performance measurement system can respond to rapid, unexpected organizational and external changes in the community. Prior to the implementation of the ActiveStrategy performance management system, in 2013, 42% of metrics established by the Florida Department of Health to measure county health department performance met target.  After implementation, 73% of those metrics met target, representing a 31 percentage point increase which is a 75% improvement overall in 2014.  From January, 2014 to July, 2014, the employee evaluations averaged a score of 4.5 out of a possible 5. Scorecard Metrics submitted by due date was 53% as of April, 2014, and increased to 92% by September, 2014.  Scorecard metrics that “Met Target” increased from 78.8% in July, 2014, to 82.2% in September, 2014.  After six months of data collection the agency undertook a comprehensive analysis that resulted in the refinement of all metrics in the ActiveStrategy performance management system. Participation included subject matter experts, Senior Leaders, and Program Managers. As a result of this analysis metrics were better aligned with programmatic objectives.  
Lessons Learned In Relation To Practice:  The lesson learned is “What gets measured gets done”. The implementation of a centralized performance management system allows the organization to integrate multiple scorecards and metrics into cascading scorecards enabling it to measure, analyze and improve organizational performance. This has eliminated the need for multiple independent external applications to track and enter performance measures as well as the creation of charts and graphs. Another lesson learned is that leaders have to engage in the performance measurement system through the participation of the business review process. Regular reviews of the performance measurement system and all of the cascading metrics that it contains ensures relevance, accountability, and a culture of continuous improvement. Dedicated human, technological, and fiscal resources need to be applied to ensure sustainability of the system. Lessons Learned In Relation To Partner Collaboration:  Partners such as funders, community, and government leaders, all need data to manage their programs, know the health status of the community, and determine if tax and or grant dollars are being used effectively in providing public health services. Data is shared with partners and the community during community meetings, website postings, monthly and annual reports, and press releases. The ability to have a comprehensive scorecard system that cascades to different levels of the organizations enables the user to better monitor performance of the program and employee. Cost Benefit Analysis:  The organization is committed in continuing to utilize ActiveStrategy as its performance management system. The performance management system has become an integral part of the quality improvement process of the organization. Senior Leaders fully support the ongoing use of the ActiveStrategy performance management system. The initial outlay of $60,000 was a one-time cost. Without any substantial budgetary cuts the agency will be able to sustain the $10,000 annual licensing fee, which is less than 0.02% of our total operating budget. Grant funding opportunities now require an evaluation of performance which is much more easily accomplished using the active strategy performance management system. Is There Sufficient Stakeholder Commitment To Sustain The Practice?: Without any substantial budgetary cuts the agency will be able to sustain the $10,000 annual licensing fee, which is less than 0.02% of our total operating budget. Internal stakeholders are committed to sustain the process since managers are directly responsible for the performance of their staff, and employee performance determines the manager’s performance evaluation score.  Prior to implementing the performance management system, the organization was reliant on statewide systems that only reported data on an annual basis.  This did not provide an opportunity to make periodic adjustments throughout the year and led to a pass/fail process for meeting targets.  The segmentation of metrics into data points that are sampled more frequently than annually, and subsequent review at the monthly business review meetings, provide the organization the opportunity to make adjustments to underperforming metrics throughout the year as well as highlight metrics that exceed established targets. External stakeholders are committed to sustain the practice because they receive consistent and reliable data reporting. This practice may be easily replicated by other health departments based on programmatic requirements that they are measuring.    
I am a previous Model Practices applicant