benchmarks.md

layout	title	css
page	ARPfix Benchmark Results	categories.css

Link to ARPfix benchmark: https://github.com/sqlab-sustech/APER-ARPfix-benchmark

Link to Aper executable: APER tool

Link to Lint source code: com/android/tools/lint/checks/PermissionDetector.kt

Link to ARPDroid repository: https://bitbucket.org/malindadoo/arpdroid/src/develop/

Link to RevDroid repository: https://github.com/letitb/revdroid

---

We compare the tools by their:

1. True positive ($$TP$$): numbers of buggy versions that have warnings

2. True negative ($$TN$$): numbers of patched versions that have no warnings

3. False positive ($$FP$$): numbers of patched versions that have warnings

4. False negative ($$FN$$): numbers of buggy versions that have no warnings

The evaluation metrics:

$$Precision=\frac{TP}{TP+FP}$$

$$Recall=\frac{TP}{TP+FN}$$

$$F_1\ score=\frac{2PrecisionRecall}{Precision+Recall}$$

$$FPR=\frac{FP}{FP+TN}$$

---

{% include benchmarkstype-1.html %}

---

{% include benchmarkstype-2.html %}