gavindiaz/NexQuant
1
0
Fork 0
mirror of https://github.com/NicolasBohn/NexQuant.git synced 2026-08-02 18:07:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix model bug and push (#35)

Browse Source
This commit is contained in:
you-n-g
2024-06-27 18:15:31 +08:00
committed by GitHub
parent f25ef70c98
commit bc8d96e96c
3 changed files with 38 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rdagent/model_implementation/benchmark/eval.py
+10 -1
View File
@@ -19,7 +19,16 @@ class ModelImpValEval:
Assumption:
- If the model structure is similar, the output will change in similar way when we change the input.
- we try to initialize the model param in similar value. So only the model structure is different.
Challenge:
- The key difference between it and implementing factors is that we have parameters in the layers (Factor operators often have no parameters or are given parameters).
- we try to initialize the model param in similar value. So only the model structure is different.
Comparing the correlation of following sequences
- modelA[init1](input1).hidden_out1, modelA[init1](input2).hidden_out1, ...
- modelB[init1](input1).hidden_out1, modelB[init1](input2).hidden_out1, ...
For each hidden output, we can calculate a correlation. The average correlation will be the metrics.
"""
def evaluate(self, gt: ModelTaskImpl, gen: ModelTaskImpl):