Public comments are essential components of federal rule making. When any federal agency proposes a new regulation, the agency must then request comments on that proposal from all members of the public. As the public submits comments—many of which are technical—they must be collected, coded, categorized, interpreted, and delivered back to the agency promptly. Comments vary in length and scope, from two-sentence letters to 20-page legal documents. Analysts must understand rules and regulations, know the right keywords to search for, and grasp the context and relevance of comments as they code them. Due to this unique requirement, agencies often rely on contracting partners to help them manage the volume of public comments. When HHS agencies asked for help coding and analyzing public comments, Ripple Effect needed a solution to respond to the challenge rapidly, and build a process to ensure quality and consistency.
Ripple Effect employs a 70-person part time on-call analyst workforce, geographically dispersed and highly skilled in qualitative analysis, to support public comment coding efforts. When a commenting project arises from HHS, we draw upon this team to meet the size of the effort and the agency’s needs. To ensure quality in coding and analysis, we implemented an in-house ‘sign-off’ process in which every coder undergoes a pre-project test to ensure reliability across the team’s understanding of the federal rules, processes, and coding taxonomies. To work smarter and more efficiently, we group analysts with similar schedules and time zones together so that if they run into problems, teammates are available for consultation. These practices equip our analysts to identify proverbial ‘needles in haystacks’—the valuable comments and contextual clues that can help guide revisions to a proposed federal rule.
Since 2017, we’ve supported 11 public comment coding projects for HHS and helped ensure efficient, high-quality analysis of public comments that advise the government on feasibility and implications of proposed regulations. Our internal vetting and sign-off process has enhanced the consistency and quality of our coders’ outputs, and our grouping strategy has helped us boost collaboration and efficiency, keeping many questions and problems from bubbling up to project managers. Our geographic grouping strategy enables us to work around-the-clock, avoid lag time, and collectively work more productively across time zones. Most importantly, we’ve drastically reduced the number of errors found during our quality control process—less re-coding and re-formatting, and more time saved. We’ve built a repeatable process through which we work smarter against challenging time constraints, deliver consistent and valuable work, and help the government finalize regulations informed by the views of those likely to be impacted by them.