Fix log path issues #53
Conversation
There was a problem hiding this comment.
as I have explained before the reason the /* is added is to remove all the modifications in a folder when putting the folder name the folder itself is ignored. So to be safe I added all the git ignores for the folders and added .gitkeep in addition to make sure git doesnt ignore the folder. this was used in the repo in the folder named tables so i just copied the method
measurement_summary_draw_heatmap.py
Outdated
| intermediate_result = json.load(f) | ||
|
|
||
| with open(file_path + '/%s_dataset_%s_%s_%s/intermediate_result_among5.json' % (experiment, 'HumanEval', model, temperature), 'r') as fo: | ||
| with open(file_path + '/%s_dataset_%s_%s_%s/intermediate_result_among5.json' % (experiment, dataset, model, temperature), 'r') as fo: |
There was a problem hiding this comment.
does this appear in the other PR? I don't think this change is correct. (It is supposed to get original_result).
There was a problem hiding this comment.
it seems it is not but I did have to change it to be able to get good results. I had an error showing that it could not find the correct path to input results into and it was because I was using HumanEvalComm dataset and it was searching for HumanEval dataset. I will add it to the other PR if needed.
There was a problem hiding this comment.
You need humaneval dataset to get certain eval metrics. I don't think this change is correct. You should remove this change and re-run this step, as it is introducing errors.
|
can you add PR description to explain why you want to remove the log files first? Your PR description is saying why you are adding gitkeep? |
I have encountered a problem while running the code using ClarifyCoder ; the issue was that it would try to access a log that already existed and for some reason it would write to it instead of making a new log. I added gitkeep to keep the folders, as there is no fallback method to create the log/record/print folders when they don't exist.