<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[BrewClear Engineering]]></title><description><![CDATA[BrewClear Engineering]]></description><link>https://brewclear-engineering.hashnode.dev</link><image><url>https://cdn.hashnode.com/res/hashnode/image/upload/v1593680282896/kNC7E8IR4.png</url><title>BrewClear Engineering</title><link>https://brewclear-engineering.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Tue, 13 Oct 2026 14:14:43 GMT</lastBuildDate><atom:link href="https://brewclear-engineering.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Turning Local Regulations into Data: BrewClear's Rule-Pack Architecture]]></title><description><![CDATA[Part 2 of Building BrewClear: Data Engineering for Local Water Compliance
In the first article, I argued that a lab result cannot be interpreted without its facility and local rules. The next engineer]]></description><link>https://brewclear-engineering.hashnode.dev/turning-local-regulations-into-data-brewclear-s-rule-pack-architecture</link><guid isPermaLink="true">https://brewclear-engineering.hashnode.dev/turning-local-regulations-into-data-brewclear-s-rule-pack-architecture</guid><dc:creator><![CDATA[revathi Selvaraj]]></dc:creator><pubDate>Tue, 06 Oct 2026 19:05:01 GMT</pubDate><content:encoded><![CDATA[<p><em>Part 2 of Building BrewClear: Data Engineering for Local Water Compliance</em></p>
<p>In <a href="https://brewclear-engineering.hashnode.dev/why-wastewater-compliance-is-a-data-engineering-problem">the first article</a>, I argued that a lab result cannot be interpreted without its facility and local rules. The next engineering question is where those rules should live.</p>
<p>The quick implementation would be a chain of conditionals: if the facility is in Michigan, use these thresholds; if it is in Washington, use another set. That might work for a demo with one facility. It breaks down when two facilities in the same state discharge to different municipal treatment systems, or when a treatment authority changes a surcharge schedule without changing state law.</p>
<p>As a data science engineer and BrewClear cofounder, I want the calculation to answer a narrower question: <strong>given this reading and the rule record that applies to this facility, what does the record say?</strong> Finding, verifying, and maintaining that rule record is a separate data problem.</p>
<h2>The correct granularity is local</h2>
<p>The <a href="https://www.epa.gov/npdes/pretreatment-standards-and-requirements-local-limits">U.S. Environmental Protection Agency</a> describes pretreatment local limits as site-specific protections for a publicly owned treatment works (POTW), its workers, sludge, and receiving waters. They may be numeric or narrative. That is why a state code alone is not a sufficient lookup key for an industrial wastewater limit.</p>
<p>For a facility discharging to a municipal sewer, we need to know <em>which</em> treatment authority receives its wastewater. We also need to distinguish that path from a groundwater discharge or a direct surface-water discharge, where a different authority and permit process may apply. BrewClear's permit finder represents those as separate paths; it does not treat them as interchangeable compliance workflows.</p>
<p>This led us to three layers of data:</p>
<p><img src="https://cdn.hashnode.com/uploads/covers/6ab6d832ae46d2c34ea6614f/79c913a7-0165-4474-97c7-3c40f22b33ae.png" alt="Diagram showing how state packs, locality records, and industry packs contribute distinct context to a facility's applicable rule" /></p>
<table>
<thead>
<tr>
<th>Layer</th>
<th>Holds</th>
<th>Why it is separate</th>
</tr>
</thead>
<tbody><tr>
<td><strong>State rule pack</strong></td>
<td>Agency information, discharge paths, questionnaire, resources, and feature capabilities</td>
<td>State guidance changes without requiring a new calculation for every state.</td>
</tr>
<tr>
<td><strong>Locality record</strong></td>
<td>Receiving authority, numeric limits, surcharge thresholds and rates, reporting cadence, sources, and verification notes</td>
<td>The same state can contain many different treatment-authority requirements.</td>
</tr>
<tr>
<td><strong>Industry pack</strong></td>
<td>Parameters and operating traits for a brewery, winery, or distillery</td>
<td>A facility's production context is different from its jurisdiction.</td>
</tr>
</tbody></table>
<p>The facility links to a locality and identifies its state and industry. The calculation then reads the locality record. That relationship is the important part: <strong>a threshold belongs to the applicable authority, not to a generic state-wide default.</strong></p>
<h2>A rule pack is a data contract</h2>
<p>In the current BrewClear proof of concept, each supported state has a JSON file. The file describes the state's paths, agency, resources, questionnaire, capabilities, and locality entries. Industry packs are separate JSON files. A JSON Schema defines the shape a state pack must have: required sections, valid path identifiers, expected data types, and allowed capability fields.</p>
<p>At startup, BrewClear scans the pack directories, parses each file, validates it with the schema, and registers valid packs in memory. Invalid JSON, schema failures, or duplicate IDs are logged and skipped; they do not prevent valid packs from loading. Tests discover every pack file and check structural assumptions, so a newly added pack is included in the conformance suite.</p>
<p>That is a useful contract. It catches a missing <code>localities</code> array or an unexpected capability value before the app tries to use the pack. It also makes the intended extension point visible to another engineer: adding a jurisdiction begins with a data file and a validation run, not a search through every screen for state-specific branches.</p>
<p>But schema validity is a very limited kind of validity. A schema can confirm that a pH limit is numeric. It cannot prove that the number came from the current ordinance, applies to this facility's permit, or uses the right averaging period. For a compliance product, those questions matter more than whether the JSON parses.</p>
<p><img src="https://cdn.hashnode.com/uploads/covers/6ab6d832ae46d2c34ea6614f/914c7d0e-d05f-4267-af87-a6206b3b33ed.png" alt="Diagram contrasting structural rule-pack validation in code with regulatory source verification needed for production" /></p>
<h2>Provenance needs a finer label than “verified state”</h2>
<p>Our Michigan pack illustrates a subtle distinction. Its state-level metadata marks the regulatory framework as verified, while a locality inside that pack can be <code>partially_sourced</code>. The locality also carries source links and notes that distinguish confirmed facts from working estimates. Other locality entries may be marked <code>estimated</code>.</p>
<p>Those labels should never be collapsed into one green “Michigan verified” badge. State guidance and a treatment authority's numeric values have different evidence. We therefore store locality provenance with the limits themselves, including <code>limits_status</code>, source references, and explanatory notes when seeding the database.</p>
<p>This is the beginning of a review workflow, not the end of one. A published webpage can establish that an industrial pretreatment program exists while leaving a particular surcharge rate or permit-specific condition unconfirmed. Before a locality record is used for real reporting, someone must reconcile each applicable value with the controlling ordinance, permit, or authority. BrewClear's demo data is deliberately labeled where that work remains.</p>
<h2>How the data reaches the shared engine</h2>
<p>The pack loader registers state and industry content for the application. On a fresh demo database, the seeder copies each pack's locality entries into PostgreSQL so facilities can reference locality records. A reading is evaluated with the limits on its facility's locality row. The <code>checkReading(reading, locality)</code> function does not need a Michigan-specific or Oregon-specific branch for each numeric threshold; it receives the relevant limits as input.</p>
<p><img src="https://cdn.hashnode.com/uploads/covers/6ab6d832ae46d2c34ea6614f/4eb48e93-9e90-4aa9-ab67-7cd141b2f7fd.png" alt="Diagram showing a reading and its facility's locality rule data entering a shared limit checker" /></p>
<p><strong>The runtime path:</strong> validated packs provide guidance and locality entries; seeded locality rows link to facilities; each reading and its facility's locality record enter the shared checker.</p>
<p>The separation also lets a state pack say which features it is ready to offer. For example, the California demonstration pack sets <code>report_draft</code> to false. That flag is surfaced to the application so an unsupported report workflow can be withheld. In a production design, capability checks also need to be enforced at the API boundary; a UI flag alone is not a security or correctness boundary.</p>
<h2>The difficult part is changing rules safely</h2>
<p>The current architecture makes adding a <em>candidate</em> state pack straightforward. It does not make regulatory content self-updating. In today's proof of concept, locality rows are seeded when the locality table is empty. Editing a JSON pack does not automatically update locality rows in an existing database. That is a deliberate gap between a demonstrable extension mechanism and a production content-management process.</p>
<p>The next stage needs a controlled way to introduce rule changes: source review, effective dates, approval, a database migration or synchronization step, and a record of which rule version produced each historical decision. It also needs an alert if a pack expected in a deployment fails validation and is skipped. Staying online is useful; quietly losing coverage would not be.</p>
<p>The principle behind BrewClear's rule packs is simple: <strong>keep the decision procedure stable, and make the changing regulatory content explicit, attributable data.</strong> The harder work is building the evidence and change process around that data. That is where a flexible architecture becomes a trustworthy one.</p>
<p><strong>Next in the series:</strong> how manual entries, CSV files, lab EDDs, and document extraction become a common reading without losing units, qualifiers, or source information.</p>
]]></content:encoded></item><item><title><![CDATA[Why Wastewater Compliance Is a Data Engineering Problem]]></title><description><![CDATA[The first article in Building BrewClear: Data Engineering for Local Water Compliance
Imagine an environmental consultant opening a lab report for a small brewery. The report has a BOD result and a sam]]></description><link>https://brewclear-engineering.hashnode.dev/why-wastewater-compliance-is-a-data-engineering-problem</link><guid isPermaLink="true">https://brewclear-engineering.hashnode.dev/why-wastewater-compliance-is-a-data-engineering-problem</guid><category><![CDATA[data-engineering]]></category><category><![CDATA[software architecture]]></category><category><![CDATA[sustainability]]></category><dc:creator><![CDATA[revathi Selvaraj]]></dc:creator><pubDate>Fri, 25 Sep 2026 21:06:54 GMT</pubDate><content:encoded><![CDATA[<p><em>The first article in Building BrewClear: Data Engineering for Local Water Compliance</em></p>
<p>Imagine an environmental consultant opening a lab report for a small brewery. The report has a BOD result and a sample date. Somewhere else is the facility's permit. A sewer-use ordinance contains the receiving treatment authority's rules. The reporting deadline is on a calendar, and last month's corrective-action notes are in an email thread.</p>
<p>The number in the lab report matters. But it cannot answer the question the consultant actually has: <strong>What does this result mean for this facility, under the rules that apply to it, and what needs to happen next?</strong></p>
<p>I am a data science engineer and a cofounder of BrewClear. We started building it because this question sits at the intersection of messy source data, local regulatory context, repeatable calculations, and human judgment. That makes it an engineering problem before it becomes a dashboard problem.</p>
<h2>The same measurement needs different context</h2>
<p>In the United States, an industrial facility that discharges to a municipal sewer may be subject to pretreatment requirements administered through a publicly owned treatment works, or POTW. The <a href="https://www.epa.gov/npdes/national-pretreatment-program">U.S. Environmental Protection Agency explains</a> that the national program aims to protect treatment infrastructure and reduce industrial pollutants entering municipal systems. <a href="https://www.epa.gov/npdes/pretreatment-standards-and-requirements-local-limits">Local limits</a> address the needs of a particular POTW, its workers, sludge, and receiving waters. They can be numeric or narrative.</p>
<p>So a BOD result is not a universal green or red number. Interpreting it requires the correct facility, discharge path, treatment authority, applicable limit, measurement unit, sample date, and sometimes a different threshold for a surcharge than for a violation. A result without that context is only a value in a file.</p>
<p>The engineering consequence is clear: we cannot hard-code one set of thresholds and call it a compliance platform. The rule itself must be data, connected to the facility and accompanied by a source and a verification status. We also need to know when a result was sampled and which version of a rule was used to assess it. BrewClear's proof of concept handles some of this context today; full historical rule versioning is still work ahead.</p>
<h2>The inputs do not arrive as one clean dataset</h2>
<p>Consultants receive data in forms created for different purposes. A lab may deliver one row per analyte, with sample identifiers, methods, units, qualifiers, and reporting limits. A simple spreadsheet may have one row per sample. A permit may be a PDF. A deadline may be in a calendar, while evidence of a corrective action lives in an email or document.</p>
<p>Those files cannot safely be merged by matching a date and hoping for the best. The pipeline needs to answer questions such as:</p>
<ul>
<li>Which facility and sampling point does this record belong to?</li>
<li>Are two rows different analytes from one sample, or two separate samples?</li>
<li>Does the lab's name for an analyte match the parameter named in the local rule?</li>
<li>Are the units compatible, and what does a non-detect qualifier mean?</li>
<li>Which values were imported, which were extracted from a document, and which were entered by a person?</li>
</ul>
<p>Our first approach in BrewClear is to normalize several input paths into a common reading record. The current proof of concept accepts manual entries, a simple CSV layout, a canonical lab electronic data deliverable, and document-assisted extraction. The lab parser groups analyte rows into samples and returns warnings for conditions such as unit mismatches or untracked analytes. Extracted document values prefill a form for review; they are not silently treated as confirmed measurements.</p>
<p>That distinction matters. A pipeline is useful only if it preserves the meaning and uncertainty of its inputs. A plausible-looking number with a lost unit or qualifier can be more dangerous than a missing value.</p>
<h2>A decision should be traceable through the system</h2>
<p>Once a reading is normalized, it still needs to be evaluated against the right local rule. In BrewClear, the core check is a deterministic function: it takes a reading and a locality record, then returns parameter-level results and an overall classification. A separate calculation produces a rough surcharge estimate where rates are configured. The same checking logic feeds reports, client packets, and the data tools used by the copilot.</p>
<p>That shared calculation is an architectural choice. If a dashboard, a report, and an AI assistant each implement their own threshold logic, they can disagree about the same sample. Reusing one calculation makes a discrepancy easier to detect and explain. It also gives us something concrete to test with known inputs and expected outputs.</p>
<p>Here is the flow we are building around:</p>
<pre><code class="language-text">Lab file / entry / document
          |
          v
Normalized reading + source + warnings
          |
          +-------- Applicable locality rule + provenance
          |                    |
          v                    v
             Deterministic check
                     |
          Classification + explanation
                     |
          Consultant review and action
                     |
             Draft report / packet
</code></pre>
<p>The diagram also shows a limit of the software. A classification is not the end of the workflow. Someone still needs to assess the source documents, investigate unexpected results, record corrective action when appropriate, and review what will be submitted. BrewClear does not file reports with an agency.</p>
<h2>Why build BrewClear around this workflow?</h2>
<p>A spreadsheet can calculate against a known threshold. It is much harder for a collection of spreadsheets, PDFs, and inboxes to maintain a reliable connection between a facility, its changing local rules, the original measurement, the calculation, and the action taken. That connection is the product we want to build.</p>
<p>Our current proof of concept demonstrates the workflow across Michigan, Washington, Oregon, and California with seeded facilities. It uses state and industry rule packs, locality records, reading ingestion, deterministic checks, reports, and a consultant workspace. This demonstrates the architecture; it does not mean every locality value is ready for unsupervised production use. Regulatory content must be checked against authoritative sources, and the person responsible for a report must review it.</p>
<p>The deeper engineering work is still ahead: stronger validation across varied lab formats, rule effective dates and versioning, end-to-end lineage, better storage of source documents, and production data-quality monitoring. Those are not peripheral improvements. They determine whether a future result can be explained and trusted.</p>
<p>That is why I describe wastewater compliance as a data engineering problem. The goal is not to automate judgment away. It is to give the people making decisions a consistent, traceable picture of the evidence and the rules.</p>
<p><strong>Next in the series:</strong> how BrewClear represents state guidance and locality-specific limits as rule-pack data, so the same checking engine can work across different jurisdictions.</p>
]]></content:encoded></item></channel></rss>