tech/hamori
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

refactor: reorganize data/processed/ into mcgill/ and user/ subdirs

Browse Source 
Moved data/processed/{train,val,holdout}/ → data/processed/mcgill/{train,val,holdout}/
so both corpora have their own namespace under data/processed/.
Updated PRETRAIN_DATA paths in make_colab_zip.py accordingly
(path remap workaround no longer needed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Masahiko AMANO
2026-05-21 19:47:32 +03:00
parent 8f657ca916
commit c4dd2fb690
1 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/make_colab_zip.py
+4 -4
View File
@@ -4,8 +4,8 @@ pretrain mode (default):
- src/ (all Python modules) - src/ (all Python modules)
- scripts/pretrain.py - scripts/pretrain.py
- requirements.txt - requirements.txt
- data/processed/mcgill/train/*.pt (remapped from data/processed/train/) - data/processed/mcgill/train/*.pt
- data/processed/mcgill/val/*.pt (remapped from data/processed/val/) - data/processed/mcgill/val/*.pt
finetune mode: finetune mode:
- src/ (all Python modules) - src/ (all Python modules)
@@ -57,8 +57,8 @@ MODE_SCRIPTS: dict[str, list[str]] = {
# Local dir → arc path inside zip # Local dir → arc path inside zip
PRETRAIN_DATA: list[tuple[Path, str]] = [ PRETRAIN_DATA: list[tuple[Path, str]] = [
(ROOT / "data" / "processed" / "train", "data/processed/mcgill/train"), (ROOT / "data" / "processed" / "mcgill" / "train", "data/processed/mcgill/train"),
(ROOT / "data" / "processed" / "val", "data/processed/mcgill/val"), (ROOT / "data" / "processed" / "mcgill" / "val", "data/processed/mcgill/val"),
] ]
FINETUNE_DATA: list[tuple[Path, str]] = [ FINETUNE_DATA: list[tuple[Path, str]] = [