ML Random Forest

[JamesCormierChisholm]

Hi everyone,

I’m currently optimizing a high-performance Random Forest (RF) inference engine running on an Intel Core i9-14900HX-based system, and I’ve reached a performance milestone I’d like to share.

After performing a deep-dive analysis using Intel VTune, we identified significant bottlenecks in our original implementation related to branch misprediction and memory stalls (CPI rate of 3.586). By transitioning our decision tree architecture from pointer-based traversal to a flattened, cache-coherent, array-based layout, we’ve achieved the following:

Performance Results:

• Previous Baseline (CPU): ~8.26 MP/s

• Current Throughput (Optimized, CPU and GPU): ~131.07 MP/s

• Latency: ~8ms for 1,048,576 samples

• Microarchitecture Efficiency: Significant reduction in stall cycles.

We are now effectively compute-bound on the inference engine, with our current overhead being pushed into data orchestration and I/O-heavy preprocessing (spatial fusion/tiling).

The Ask: Given this level of throughput, I am looking for advice on identifying the best industrial or commercial markets for this technology.  I previously was running a less optimized RF across Canada at a national level, on digital elevaton maps versus metals, to identify critical metals opportunities, but the mining industry are not into such innovations yet, it seems.   For other industries, specifically, I am interested in use cases where:

1. Low-latency inference is critical: (e.g., real-time industrial defect detection on assembly lines).

2. High-volume throughput is required: (e.g.,  time-series data processing).

3. Predictive maintenance: Leveraging the RF outputs in a lagged/time-series model to detect degradation trends before failure.

If you have insights into sectors within the Intel ecosystem that could benefit from this specific high-throughput inference capability—or if there are teams working on similar "data-chassis" optimization—I would appreciate your guidance on how to best position this for co-marketing or collaborative development.

Looking forward to hearing your thoughts.

Best regards,

James, Eureka Maps Inc.