Skip to content

Commit

Permalink
Feature: support nn-Meter builder (#43)
Browse files Browse the repository at this point in the history
  • Loading branch information
JiahangXu authored Mar 8, 2022
1 parent dd74162 commit 76d7f88
Show file tree
Hide file tree
Showing 140 changed files with 10,266 additions and 292 deletions.
5 changes: 3 additions & 2 deletions .gitattributes
Git LFS file not shown
8 changes: 4 additions & 4 deletions .github/workflows/integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
path: |
~/.nn_meter
/home/runner/work/nn-Meter/data/testmodels
key: ${{hashFiles('nn_meter/configs/predictors.yaml')}}-${{hashFiles('tests/integration_test.py')}}
key: ${{hashFiles('nn_meter/configs/predictors.yaml')}}-${{hashFiles('tests/integration_test/test_latency_predictor.py')}}

- name: Install dependencies
run: |
Expand All @@ -42,11 +42,11 @@ jobs:
run: pip install -U .

- name: Integration test
run: python tests/integration_test.py
run: python tests/integration_test/test_latency_predictor.py

- name: Diff result with reference
run: diff tests/data/reference_result.txt tests/test_result.txt
run: diff tests/integration_test/data/reference_result.txt tests/integration_test/test_result.txt

- name: clean env
run: rm tests/test_result.txt
run: rm tests/integration_test/test_result.txt

8 changes: 4 additions & 4 deletions .github/workflows/integration-test_nni_based_torch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
path: |
~/.nn_meter
/home/runner/work/nn-Meter/data/testmodels
key: ${{hashFiles('nn_meter/configs/predictors.yaml')}}-${{hashFiles('tests/integration_test.py')}}
key: ${{hashFiles('nn_meter/configs/predictors.yaml')}}-${{hashFiles('tests/integration_test/test_latency_predictor.py')}}

- name: Install dependencies
run: |
Expand All @@ -40,10 +40,10 @@ jobs:
run: pip install -U .

- name: Integration test
run: python tests/integration_test_torch.py --apply-nni
run: python tests/integration_test/test_latency_predictor_torch.py --apply-nni

- name: Diff result with reference
run: diff tests/data/reference_result_nni_based_torch.txt tests/test_result_nni_based_torch.txt
run: diff tests/integration_test/data/reference_result_nni_based_torch.txt tests/integration_test/test_result_nni_based_torch.txt

- name: clean env
run: rm tests/test_result_nni_based_torch.txt
run: rm tests/integration_test/test_result_nni_based_torch.txt
8 changes: 4 additions & 4 deletions .github/workflows/integration-test_onnx_based_torch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
path: |
~/.nn_meter
/home/runner/work/nn-Meter/data/testmodels
key: ${{hashFiles('nn_meter/configs/predictors.yaml')}}-${{hashFiles('tests/integration_test.py')}}
key: ${{hashFiles('nn_meter/configs/predictors.yaml')}}-${{hashFiles('tests/integration_test/test_latency_predictor.py')}}

- name: Install dependencies
run: |
Expand All @@ -41,10 +41,10 @@ jobs:
run: pip install -U .

- name: Integration test
run: python tests/integration_test_torch.py --apply-onnx
run: python tests/integration_test/test_latency_predictor_torch.py --apply-onnx

- name: Diff result with reference
run: diff tests/data/reference_result_onnx_based_torch.txt tests/test_result_onnx_based_torch.txt
run: diff tests/integration_test/data/reference_result_onnx_based_torch.txt tests/integration_test/test_result_onnx_based_torch.txt

- name: clean env
run: rm tests/test_result_onnx_based_torch.txt
run: rm tests/integration_test/test_result_onnx_based_torch.txt
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
Note: This is an beta (preview) version which is still under refining.

**nn-Meter** is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices. The key idea is dividing a whole model inference into kernels, i.e., the execution units of fused operators on a device, and conduct kernel-level prediction. We currently evaluate four popular platforms on a large dataset of 26k models. It achieves 99.0% (mobile CPU), 99.1% (mobile Adreno 640 GPU), 99.0% (mobile Adreno 630 GPU), and 83.4% (Intel VPU) prediction accuracy.

The current supported hardware and inference frameworks:
Expand Down Expand Up @@ -73,7 +71,7 @@ Here is a summary of supported inputs of the two methods.
| Tensorflow | Checkpoint file dumped by `tf.saved_model()` and end with `.pb` | Checkpoint file dumped by `tf.saved_model` and end with `.pb` |
| Torch | Models in `torchvision.models` | Object of `torch.nn.Module` |
| Onnx | Checkpoint file dumped by `torch.onnx.export()` or `onnx.save()` and end with `.onnx` | Checkpoint file dumped by `onnx.save()` or model loaded by `onnx.load()` |
| nn-Meter IR graph | Json file in the format of [nn-Meter IR Graph](./docs/input_models.md#nnmeter-ir-graph) | `dict` object following the format of [nn-Meter IR Graph](./docs/input_models.md#nnmeter-ir-graph) |
| nn-Meter IR graph | Json file in the format of [nn-Meter IR Graph](./docs/predictor/input_models.md#nnmeter-ir-graph) | `dict` object following the format of [nn-Meter IR Graph](./docs/predictor/input_models.md#nnmeter-ir-graph) |
| NNI IR graph | - | NNI IR graph object |

In both methods, users could appoint predictor name and version to target a specific hardware platform (device). Currently, nn-Meter supports prediction on the following four configs:
Expand Down
Loading

0 comments on commit 76d7f88

Please sign in to comment.