Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
KKZ20 authored Oct 12, 2023
1 parent 711a960 commit 4cede47
Showing 1 changed file with 11 additions and 2 deletions.
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

# Knowledge-based Version Incompatibility Detection for Deep Learning
# Knowledge-based Version Incompatibility Detection for Deep Learning [![](https://img.shields.io/badge/arXiv-2308.13276-b31b1b.svg)](https://arxiv.org/abs/2308.13276)

This repository contains data and code of DECIDE, a version incompatibility detection tool based on pre-trained language models proposed in *"Knowledge-based Version Incompatibility Detection for Deep Learning"*. Meanwhile, this repository also contains data and code to replicate experiment results in the paper. This repository has been made publicly available on GitHub to support Open Science.

Expand Down Expand Up @@ -90,4 +90,13 @@ $ python compare.py
### RQ5: To what extent can different knowledge consolidation strategies affect the accuracy of the resulting knowledge graph?
In RQ5, we evaluate three different knowledge consolidation strategies (*majority vote*, *weighted majority vote*, and *vote by loss*). We randomly sampled 228 of the 558 relations which were consolidated from multiple predicted relations in the knowledge. We present the manually labeled results in `Experiments/RQ5/RQ5-ALL.xlsx`. Note that to fully evaluate the knowledge consolidation strategies, we excluded 25 incorrect relations that were caused by the mismatch between component names and versions. Thus, we get 203 relations.

Overall, there are 193 correct relations using majority vote (193 / 203 = 95.1%), 190 correct relations using majority vote (193 / 203 = 93.6%), and 186 correct relations using majority vote (186 / 203 = 91.6%).
Overall, there are 193 correct relations using majority vote (193 / 203 = 95.1%), 190 correct relations using majority vote (193 / 203 = 93.6%), and 186 correct relations using majority vote (186 / 203 = 91.6%).

## Citation
```
@article{zhao2023knowledge,
title={Knowledge-Based Version Incompatibility Detection for Deep Learning},
author={Zhao, Zhongkai and Kou, Bonan and Ibrahim, Mohamed Yilmaz and Chen, Muhao and Zhang, Tianyi},
journal={arXiv preprint arXiv:2308.13276},
year={2023}
}

0 comments on commit 4cede47

Please sign in to comment.