CORONAVIRUS (COVID-19) RESOURCE CENTER Read More

Improving a Performance Management System through Use of the Automation of Data Entry

State: FL Type: Neither Year: 2015

Broward County is located in the southeastern portion of the State of Florida with Miami-Dade County to the south and Palm Beach County to the north.  Broward County’s 2013 population estimate of 1,838,844, represents 9% of the State’s population, and is the second most populous county of the 67 counties in the State of Florida and eighteenth most populous county in the United States (US Census).  Its diverse population includes residents representing more than 200 different countries and speaking more than 130 different languages.  31.4% of the residents are foreign-born.   Broward County is a minority/majority county demonstrated by its 2013 population by race (Black 28.5%, Asian 3.6%, Hispanic 26.9%, other races 4.1%, more than one race .2%, for a total of 59.5% and White 40.8%). The Florida Department of Health in Broward County (DOH-Broward) is the official Public Health Agency in Broward County and has been operational since 1936. It is part of the Integrated Florida Department of Health (DOH) and operates in cooperation with the Broward County Commission under Florida Statute 154. DOH-Broward’s mission is “to protect, promote and improve the health of all people in Florida through integrated state, county and community efforts”.  DOH-Broward is the lead agency providing core public health functions and essential services in the county as part of a complex public health system that includes hospitals, clinics, planning agencies, community-based organizations and others.  DOH-Broward provides population/community-based services to the county’s 1.8 million residents and over 10 million annual visitors, and is responsible for assessing, maintaining and improving health and safety within the county. DOH-Broward implemented Active Strategy Enterprise (ASE) software as the agency quality performance management tool. Upon initial implementation, the assigned staff needed to collect and manually enter data for organizational metrics into ASE for monthly business review meetings. These daily activities increased administrative workloads and costs across the organization. Due to the time commitment of manually entering data, processes became overwhelming to the staff, and as a result, data was not being consistently collected, indicators were frequently missing data values leading to unreliable metrics results, and analysis. An organizational change was needed to reduce and possibly eliminate the manual data collection and data entry processes. The Organizational Development Team conducted an initial analysis and identified several areas for improvement. The systematic approach to the issue allowed the team to clearly define the issue, measure the throughput/output of processes, analyze the alignment of the activities, implement a series of processes to facilitate the operations, and control and monitor the achieved results. Through analysis of existing procedures, it was determined that it was necessary to develop a means to automate data collection and data entry into the DOH-Broward performance management system, ASE. The new proposed practice changes were implemented in phases and the activities prioritized based on the data models and sources utilized by each program. The successful implementation of these Extract, Transform and Load (ETL) activities was the result of an integrated work of Performance Excellence and Management Information Systems Departments. Extract, Load, Transform (ELT) is a data integration process for transferring raw data from a source system to a target database and then preparing the information for downstream uses.  The activities implemented were: hire a new staff to oversee the ETL activities, create data source definitions, data input and data validation tool, data collection and normalization, data integration, and data upload by using ActiveDI interface proprietary software developed by Active Strategy Incorporated, a subsidiary of The Advisory Board Company.  Implementation of the automation occurred within a four-week time period. The data source definitions were determined in the first week of the project. The data input and data validation tools were developed, tested, and implemented in the week two. Data collection, normalization, and data integration were finalized in the week three. Data upload ActiveDI interface ongoing activities initialized on week four. The factors leading to the successful implementation of this practice are attributed to clear definition and understanding of each process. The data entry upload automation led to reliable analysis, precise, coherent, and timely decision making which ensured strategic alignment among all DOH-Broward indicators.
The data collection and data entry challenges faced by The Florida Health Department in Broward County are also shared by many other Local Health Departments. DOH-Broward developed an innovative practice to an ongoing public health performance management issue which is the level of data collection and integration among many databases and platforms. DOH-Broward services and programs generate a large amount of data on a daily basis. Throughout the organization, valuable information collected is not consistently normalized and properly analyzed. In response to this immediate need, a responsive ETL Model Practice was designed and implemented to consistently monitor the daily flow of data. Currently, 100% of DOH-Broward KPI’s are monitored through ASE. Upon automation project inception, the ASE system contained 3,000 metrics, Currently, there are 57 programmatic, 8 divisional and 4 director level scorecards encompassing over 9,900 metrics. The application includes indicators that are linked to employee’s SMART metrics on their performance expectations. This systematic approach to data extraction, transformation, and load though not innovative in its nature has allowed us to improve our decision making process. As result of enhancing its data visibility and reliability, DOH-Broward has been able to achieve several state of Florida established public health goals. For example, the percent of 2 Year Olds LHD Clients Fully Immunized - Broward achieved the state goal of 95% in less than 2 months since inception of the automation process of performance metrics which was an increase 42 percentage points from the initial of percent of 53.
Food Safety|Global Immunization|Heathcare-associated Infections|HIV in the U.S.|Lymphatic Filariasis in the Americas|Motor Vehicle Injuries|Nutrition, Physical Activity, and Obesity|Mother-to-Child Transmission of HIV and Syphilis|Teen Pregnancy|Tobacco
The overall goal of DOH-Broward is to facilitate the data collection and data entry processes.  In order to achieve this goal, the Organizational Development Team worked in conjunction with Management Information Systems Team (MIS) to develop an integrated network where skills and knowledge are easily shared. Despite the successful accomplishments from concept to deployment, automation of data collection is regarded as part of a continuous timeframe as the data extractions, transformations, and loading are dynamic and recurrent in nature. The automation of processes is sustainable and maintained indefinitely as all other members besides the new hired staff is participating in an in-kind contribution basis imposing no addition financial cost or burden to their respective departments.  Improvement Project Timeline: • On October 25th, 2013 additional, dedicated staff was hired to oversee Active Strategy Enterprise automation implementation activities.• On November 4th, 2013 Organizational Development staff conducted internal process audit and analysis of metrics collection procedures.• On November 8th, 2013 DOH-Broward Internal Audit Report with recommendation was submitted for review to the Organizational Development Team.• On November 15th, 2013 Organizational Development Director approved the recommendations.• On November 20th, 2013 First brainstorm meeting was conducted with Organizational Development and Management Information Systems staff. • On December 2nd, 2013 First Data Upload Automation prototype was created.• On December 6th, 2013 Prototype was approved for implementation.• On December 6th, 2013 Meeting with the developers of Active Strategy Enterprise was set for January 6th, 2014 to review ActiveDI interface capabilities.• On December 16th, 2013 Prototype revision 1 released to allow multiple data source input tabs.• On December 20th, 2013 Prototype revision 2 released to allow multiple data source formats.• On January 7-th10th -, 2014 onsite automation training with the developers of Active Strategy started.• On January 13th, 2014 Prototype ActiveDI interface job created and submitted for approval.• On January 20th, 2014 Prototype tested and approved for implementation.• On February 3th, 2014 ETL Project for Data Automation started with Data Sources Definitions.• On February 10th, 2014 the data input and data validation tools were developed, tested, and implemented.• On February 17th, 2014 Data collection, normalization, and data integration were finalized.• On February 24th, 2014 Data upload ActiveDI interface ongoing activities initialized.• On March 3rd, 2014 DOH-Broward agency wide, full scale implementation began.• On April 4th, 2014 Percentage of Metrics Submitted On-Time was 53%.• On May 2nd, 2014 Percentage of Metrics Submitted On-Time was 89%.• On June 6th, 2014 Percentage of Metrics Submitted On-Time was 90%.• On July – September, 2014 Percentage of Metrics Submitted On-Time above 92%• On October 22nd, 2014 29 ETL Data Automation tools, 43 ActiveDI interface Jobs, and more than 8,500 automated measures in Active Strategy Enterprise.  
The automation process was created in response to several pertinent factors which are associated with need of the data substantiation behind the decision making process. All efforts and activities were developed and implemented in order to ensure successful outcomes.  The overall process was implemented in less than 2 months. The improvement of the data collection, normalization, uploading, and analysis improved data accuracy and timely submission of data.  The cross-functional team assembled to assess the current situation, analyze the options, and make recommendations used the systematic approach as follows: define the issue, measure the throughput and output of the processes, analyze the alignment of the activities, implement a series processes to facilitate the operations, and control and monitor the achieved results. Primary data sources used were program specific spreadsheets, logs, state of Florida internal databases such as HMS, PRISM, and FLSHOTS. The data gathered in this data sources are audited by the ETL management analyst staff by using the DOH-Broward ASE Data Automation Model Spreadsheet.  Once the data source is identified, the data is normalized. Subsequent queries are used to identify and select the desired records and points of data extraction. Depending on the program requirements, multiple data sources are accessed and merged.   Process Metrics: Indicators:  Percent of Scorecard Metrics Submitted On-TimePercentage of Measures Meeting or Exceeding TargetsCalculation:  # of Scorecard Metrics Submitted On Time / # of Total Measures in ASE# of Measures Meeting or Exceeding Targets / # of Total Measures in ASEData Source:  MultipleTarget Source: DOH-Broward Strategic ObjectivesTarget:  Program specificFrequency: MonthlyData Entry Delay:  30 days As part of the Florida Health Department in Broward County culture of continuous improvement, the data results are reviewed monthly at the Business Review Meetings. Each scorecard measure is thoroughly analyzed. Underperforming indicators are carefully reviewed and corrective actions issued against them with defined action plans.  Progress status follow-ups meetings are conducted to ensure improvement and compliance.  ETL Data Model has been automated in order to ensure efficiency, effectiveness, and reliability of data. This practice has been tested and validated as it has enabled the successful achievement of the desired results of meeting and maintaining the overall Metrics Submitted on Time and Meeting and exceeding Target at the intended levels.  Initially, manually data entry into the ASE system was not well received by the end users due data entry being time consuming and confusing. As a result of the automation the administration burden has been greatly reduced resulting in a favorable disposition by the end users. 
Upon automation project inception, the ASE system contained 3,000 metrics. Currently, there are 57 programmatic, 8 divisional and 4 director level scorecards encompassing over 9,900 metrics. Through the automation of metrics collection, DOH-Broward learned fewer resources were needed to collect data, less time was required, and more data could be collected. The application includes indicators that are linked to employee’s SMART metrics on their performance expectations. The membership composition of partner collaboration and structure adapted to this automation model ensured that all members of the assembled cross-functional groups are committed to this project by offering their in-kind contributions. The value added by automating metrics collection is that the satisfaction in the collection of data increases as the administrative burden in collecting data decreases making the end users more likely to continue to use the system. Therefore, the automation of performance management metrics will be sustained and maintained. In tying into the DOH-Broward Strategic Plan Cornerstone “Culture of Continuous Improvement”, goal 2.5: Continue to develop & implement performance metrics through the use of Active Strategy for 100% of DOH-Broward programs annually through December 2015. Nevertheless, this practice requires continuous improvement as it is a long-term goal.
I am a previous Model Practices applicant