SQL: redundant explanations (probably) should be collapsed by pruning rules

[pbailis]

In the mobile_data demo, macrobase-sql returns two singleton explanations when in fact the two correspond to the same tuples.

e.g.,

macrobase-sql> SELECT * FROM DIFF(SPLIT (SELECT *, percentile(battery_drain) AS percentile from mobile) WHERE percentile > .99) ON *;
67156 [main] INFO  QueryEngine  - Using user_id, state, hw_make, firmware_version, app_version as candidate attributes for explanation
67265 [main] INFO  APLSummarizer  - Encoded in: 109 ms
67265 [main] INFO  APLSummarizer  - Encoded Categories: 2
67299 [main] INFO  APLSummarizer  - Number of results: 2
2 rows

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|    app_version     |      user_id       |      hw_make       |       state        |  firmware_version  |      support       |    global_ratio    |   outlier_count    |    total_count     |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|        v50         |         -          |         -          |         -          |         -          |      .873185       |     89.883198      |       902.0        |       1003.0       |
|         -          |         -          |       Emdoor       |         -          |         -          |      .873185       |     89.883198      |       902.0        |       1003.0       |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Can provide CSV on request.

[fabuzaid21]

So, these redundant explanations are not going to be caught by the pruning rules. (I confirmed this by running the same query through the BasicBatchPipeline, and I got the same result.) When v50 and Emdoor are individually considered as order-one candidates, they each meet the minRatio and support thresholds separately. After that, they're never checked again, because they're considered saved explanations, and there's no check afterwards to see if two separate explanations overlap. (In this case, the overlap is 100%.)

This is the sort of thing that @sahaana's explanation collapsing would help—we need to have that additional check to handle this case.