{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.11.0" } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ZeaVis Edu \u2014 Corn Leaf Disease Classifier v3.0\n", "\n", "Mengklasifikasikan penyakit daun jagung (Bercak Daun, Hawar Daun, Karat Daun, Daun Sehat)\n", "menggunakan EfficientNetV2B0 dengan **CBAM spatial attention**, **RandAugment + weather simulation**,\n", "dan **temperature-scaled confidence calibration** untuk deployment real-world.\n", "\n", "Fokus v3.0: **robustness dunia nyata** \u2014 berbagai pencahayaan, resolusi, angle, dan background.\n" ], "id": "bea416d7" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Persiapan Lingkungan\n", "\n", "Mengimpor pustaka, mengatur seed, dan mengoptimalkan konfigurasi.\n", "**Presisi float32**, resolusi target **224\u00d7224** (EfficientNetV2B0).\n", "Augmentasi real-world via RandAugment pool 15 transformasi.\n", "Confidence calibration via temperature scaling.\n" ], "id": "aba4b688" }, { "cell_type": "code", "metadata": {}, "source": [ "!pip install -r requirements.txt\n" ], "outputs": [], "execution_count": null, "id": "9dc08169" }, { "cell_type": "code", "metadata": {}, "source": [ "import os, shutil, zipfile, random, time, json\n", "from collections import Counter\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "from PIL import Image\n", "\n", "import tensorflow as tf\n", "from tensorflow.keras import layers, models, callbacks\n", "from tensorflow.keras.applications import EfficientNetV2B0\n", "from tensorflow.keras.applications.efficientnet_v2 import preprocess_input\n", "from tensorflow.keras.optimizers import AdamW\n", "from tensorflow.keras.optimizers.schedules import CosineDecay\n", "from sklearn.metrics import classification_report, confusion_matrix\n", "from sklearn.utils.class_weight import compute_class_weight\n", "from sklearn.model_selection import train_test_split\n", "\n", "# Optional: perceptual hashing for dedup (pip install imagehash)\n", "try:\n", " import imagehash\n", " HAS_IMAGEHASH = True\n", "except ImportError:\n", " HAS_IMAGEHASH = False\n", "\n", "# Optional: scipy for temperature optimization\n", "try:\n", " from scipy.optimize import minimize_scalar\n", " HAS_SCIPY = True\n", "except ImportError:\n", " HAS_SCIPY = False\n", "\n", "# Detect environment\n", "try:\n", " from google.colab import drive\n", " IS_COLAB = True\n", " print(\"Running on Google Colab\")\n", "except ModuleNotFoundError:\n", " IS_COLAB = False\n", " print(f\"Running locally (TF {tf.__version__}, GPU: {tf.config.list_physical_devices('GPU')})\")\n", "\n", "tf.keras.mixed_precision.set_global_policy('float32')\n", "\n", "# Hyperparams\n", "IMG_SIZE = (224, 224)\n", "BATCH_SIZE = 32\n", "SEED = 42\n", "random.seed(SEED)\n", "np.random.seed(SEED)\n", "tf.random.set_seed(SEED)\n", "\n", "AUTOTUNE = tf.data.AUTOTUNE\n", "print(f\"Setup OK. IMG={IMG_SIZE}, BATCH={BATCH_SIZE}\")\n" ], "outputs": [], "execution_count": null, "id": "9d108f5e" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Download dan Ekstraksi Dataset\n" ], "id": "06382ddf" }, { "cell_type": "code", "metadata": {}, "source": [ "if IS_COLAB:\n", " drive.mount('/content/drive')\n", " archive_path = '/content/drive/MyDrive/jagung/dataset_jagung.zip'\n", " destination_path = '/content/dataset_jagung.zip'\n", " extract_path = '/content/dataset'\n", "else:\n", " import gdown\n", " base = os.getcwd()\n", " archive_path = os.path.join(base, 'dataset_jagung.zip')\n", " destination_path = archive_path\n", " extract_path = os.path.join(base, 'dataset')\n", " DRIVE_FILE_ID = \"1s0H2lDOQVCixywk5eZXJz2i9jj4JihxJ\"\n", " if not os.path.exists(archive_path):\n", " print(\"Downloading from Google Drive...\")\n", " try:\n", " gdown.download(f\"https://drive.google.com/uc?id={DRIVE_FILE_ID}\", archive_path, quiet=False)\n", " except Exception as e:\n", " print(f\"Download failed: {e}\")\n", "\n", "if os.path.exists(destination_path):\n", " if not os.path.exists(extract_path) or len(os.listdir(extract_path)) == 0:\n", " os.makedirs(extract_path, exist_ok=True)\n", " print(\"Extracting dataset...\")\n", " try:\n", " with zipfile.ZipFile(destination_path, 'r') as zip_ref:\n", " zip_ref.extractall(path=extract_path)\n", " print(\"Extraction completed!\")\n", " except Exception as e:\n", " print(f\"Extraction failed: {e}\")\n", " else:\n", " print(\"Dataset ready.\")\n" ], "outputs": [], "execution_count": null, "id": "26afde45" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Data Cleaning \u2014 Corrupt Detection + Augmented Dedup\n", "\n", "Membersihkan dataset dari:\n", "- File corrupt / tidak bisa dibuka PIL\n", "- File `augmented_*` (pre-augmented duplicates \u2014 menyebabkan data leakage)\n", "- Gambar dengan dimensi atau aspect ratio ekstrim\n", "- Gambar dengan variance terlalu rendah (hampir seragam)\n" ], "id": "199bdf60" }, { "cell_type": "code", "metadata": {}, "source": [ "# --- Determine dataset path ---\n", "if IS_COLAB:\n", " dataset_path = \"/content/dataset/dataset_jagung_v1\"\n", "else:\n", " candidate = os.path.join(extract_path, \"dataset_jagung_v1\")\n", " dataset_path = candidate if os.path.isdir(candidate) else extract_path\n", "\n", "print(f\"Dataset path: {dataset_path}\")\n", "\n", "MIN_FILE_SIZE = 512\n", "MIN_DIM = 32\n", "MAX_ASPECT = 5.0\n", "\n", "def remove_augmented_duplicates(directory):\n", " # Hapus file augmented_* yang merupakan duplikat buatan.\n", " removed = 0\n", " for root, dirs, files in os.walk(directory):\n", " for file in files:\n", " if file.startswith(\"augmented_\"):\n", " try:\n", " os.remove(os.path.join(root, file))\n", " removed += 1\n", " except OSError:\n", " pass\n", " return removed\n", "\n", "def clean_and_validate_images(directory):\n", " # Validasi + bersihkan gambar corrupt / invalid.\n", " stats = {\"too_small\": 0, \"corrupt\": 0, \"small_dims\": 0, \"extreme_aspect\": 0, \"low_var\": 0, \"ok\": 0}\n", " for root, dirs, files in os.walk(directory):\n", " for file in files:\n", " fp = os.path.join(root, file)\n", " try:\n", " if os.path.getsize(fp) < MIN_FILE_SIZE:\n", " os.remove(fp); stats[\"too_small\"] += 1; continue\n", " except OSError:\n", " continue\n", "\n", " try:\n", " img = Image.open(fp); img.verify()\n", " except Exception:\n", " try: os.remove(fp); stats[\"corrupt\"] += 1\n", " except OSError: pass\n", " continue\n", "\n", " try:\n", " img = Image.open(fp)\n", " w, h = img.size\n", " if w < MIN_DIM or h < MIN_DIM:\n", " os.remove(fp); stats[\"small_dims\"] += 1; continue\n", " aspect = w / max(h, 1)\n", " if aspect > MAX_ASPECT or aspect < 1.0 / MAX_ASPECT:\n", " os.remove(fp); stats[\"extreme_aspect\"] += 1; continue\n", " # RGB conversion\n", " if img.mode not in ('RGB', 'RGBA'):\n", " img = img.convert('RGB'); img.save(fp)\n", " # Low variance check (>95% pixels same value)\n", " arr = np.array(img).astype(np.float32)\n", " if np.std(arr) < 2.0:\n", " os.remove(fp); stats[\"low_var\"] += 1; continue\n", " stats[\"ok\"] += 1\n", " except Exception:\n", " try: os.remove(fp); stats[\"corrupt\"] += 1\n", " except OSError: pass\n", " return stats\n", "\n", "print(\"1. Removing augmented duplicates...\")\n", "n_aug = remove_augmented_duplicates(dataset_path)\n", "print(f\" Removed {n_aug} augmented_* files\")\n", "\n", "print(\"2. Validating images...\")\n", "stats = clean_and_validate_images(dataset_path)\n", "print(f\" OK: {stats['ok']} | Removed: too_small={stats['too_small']} corrupt={stats['corrupt']} \"\n", " f\"small_dims={stats['small_dims']} aspect={stats['extreme_aspect']} low_var={stats['low_var']}\")\n", "\n", "# \u2500\u2500 Perceptual hash dedup (optional) \u2500\u2500\n", "if HAS_IMAGEHASH:\n", " print(\"3. Perceptual hash dedup...\")\n", " seen, removed = {}, 0\n", " for cn in sorted(os.listdir(dataset_path)):\n", " cp = os.path.join(dataset_path, cn)\n", " if not os.path.isdir(cp): continue\n", " for f in sorted(os.listdir(cp)):\n", " fp = os.path.join(cp, f)\n", " if not os.path.isfile(fp): continue\n", " try:\n", " ah = imagehash.average_hash(Image.open(fp).convert('RGB'))\n", " key = str(ah)\n", " dup_found = False\n", " for sk, (sp, sc) in seen.items():\n", " if ah - imagehash.hex_to_hash(sk) <= 5:\n", " try: os.remove(fp); removed += 1\n", " except OSError: pass\n", " dup_found = True; break\n", " if not dup_found: seen[key] = (fp, cn)\n", " except Exception: pass\n", " print(f\" Removed {removed} near-duplicates\")\n", "else:\n", " print(\"3. Perceptual hash dedup SKIPPED (pip install imagehash)\")\n", "\n", "# \u2500\u2500 Final count \u2500\u2500\n", "total = sum(len(files) for _, _, files in os.walk(dataset_path))\n", "print(f\"\\nTotal clean images: {total}\")\n" ], "outputs": [], "execution_count": null, "id": "75cacdf5" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Stratified Split by Source (70:15:15)\n", "\n", "**Tidak menggunakan splitfolders!** Split manual dengan stratifikasi berdasarkan prefix sumber gambar.\n", "Ini mencegah gambar dari sesi foto yang sama (lighting & background identik) masuk ke train DAN test.\n", "\n", "Source prefixes:\n", "- `IMG_*` \u2192 foto HP\n", "- `Corn_*` \u2192 dataset lab publik\n", "- `CBS*`, `GLS*`, `NLS*`, `CLS*` \u2192 berbagai dataset lab\n", "- `SCR*`, `CR*`, `NLB*`, `SLB*` \u2192 dataset spesifik penyakit\n" ], "id": "fbe9498c" }, { "cell_type": "code", "metadata": {}, "source": [ "def extract_source_prefix(filename):\n", " # Detect source from filename prefix for stratification.\n", " f = os.path.splitext(filename)[0]\n", " if f.startswith('IMG_'): return 'phone'\n", " if f.startswith('Corn_'): return 'lab_corn'\n", " for prefix in ['CBS', 'GLS', 'NLS', 'CLS']:\n", " if f.startswith(prefix): return 'lab_disease'\n", " for prefix in ['SCR', 'CR', 'NLB', 'SLB', 'SRS']:\n", " if f.startswith(prefix): return 'lab_rust_blight'\n", " return 'other'\n", "\n", "def stratified_split_by_source(dataset_path, output_dir, ratios=(0.7, 0.15, 0.15), seed=42):\n", " # Split dataset stratified by source prefix within each class.\n", " # Collect all images with their source prefix\n", " class_images = {}\n", " for cn in sorted(os.listdir(dataset_path)):\n", " cp = os.path.join(dataset_path, cn)\n", " if not os.path.isdir(cp): continue\n", " class_images[cn] = []\n", " for f in os.listdir(cp):\n", " fp = os.path.join(cp, f)\n", " if os.path.isfile(fp):\n", " class_images[cn].append((fp, f, extract_source_prefix(f)))\n", "\n", " # Create output directories\n", " for split in ['train', 'val', 'test']:\n", " for cn in class_images:\n", " os.makedirs(os.path.join(output_dir, split, cn), exist_ok=True)\n", "\n", " rng = np.random.RandomState(seed)\n", "\n", " for cn, images in class_images.items():\n", " # Group by source prefix\n", " by_source = {}\n", " for fp, fn, src in images:\n", " by_source.setdefault(src, []).append((fp, fn))\n", "\n", " # For each source group, split into train/val/test\n", " train_files, val_files, test_files = [], [], []\n", " for src, src_images in by_source.items():\n", " n = len(src_images)\n", " rng.shuffle(src_images)\n", " n_train = max(1, int(n * ratios[0]))\n", " n_val = max(1, int(n * ratios[1]))\n", " train_files.extend(src_images[:n_train])\n", " val_files.extend(src_images[n_train:n_train + n_val])\n", " test_files.extend(src_images[n_train + n_val:])\n", "\n", " # Copy files\n", " for fp, fn in train_files:\n", " shutil.copy(fp, os.path.join(output_dir, 'train', cn, fn))\n", " for fp, fn in val_files:\n", " shutil.copy(fp, os.path.join(output_dir, 'val', cn, fn))\n", " for fp, fn in test_files:\n", " shutil.copy(fp, os.path.join(output_dir, 'test', cn, fn))\n", "\n", " # Print split stats\n", " train_srcs = Counter(extract_source_prefix(fn) for _, fn in train_files)\n", " print(f\" {cn}: train={len(train_files)} val={len(val_files)} test={len(test_files)} | \"\n", " f\"sources={dict(train_srcs)}\")\n", "\n", "output_dir = \"/content/dataset_split\" if IS_COLAB else os.path.join(os.getcwd(), \"dataset_split\")\n", "if os.path.exists(output_dir):\n", " shutil.rmtree(output_dir)\n", "\n", "print(\"Splitting dataset 70:15:15 (stratified by source)...\")\n", "stratified_split_by_source(dataset_path, output_dir, seed=SEED)\n", "print(\"Done.\")\n", "\n", "train_dir = os.path.join(output_dir, 'train')\n", "val_dir = os.path.join(output_dir, 'val')\n", "test_dir = os.path.join(output_dir, 'test')\n", "\n", "def count_images(path):\n", " return sum(len(files) for _, _, files in os.walk(path))\n", "\n", "print(f'Train: {count_images(train_dir)} | Val: {count_images(val_dir)} | Test: {count_images(test_dir)}')\n" ], "outputs": [], "execution_count": null, "id": "56234c24" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. RandAugment Pipeline dengan Weather Simulation\n", "\n", "Pipeline augmentasi baru untuk **robustness dunia nyata**:\n", "\n", "### Pool 15 Transformasi (RandAugment: pilih N=3 per gambar)\n", "| Kategori | Transformasi | Simulasi |\n", "|---|---|---|\n", "| Geometric | Flip, Rotate, Zoom, Translate, Shear | Variasi angle/jarak foto |\n", "| Color/Light | Hue, Saturation, Brightness, Contrast, Solarize | Variasi kamera & waktu hari |\n", "| Weather | Fog, Shadow | Kondisi lapangan berkabut/berbayang |\n", "| Degradation | GaussianBlur, ResolutionDrop | Blur gerakan, kamera rendah |\n", "| Mixing | MixUp, CutMix, RandomErasing | Regularisasi label & occlusions |\n", "\n", "Fog dan Shadow adalah **custom tf operations** \u2014 tidak ada di Keras layers standar.\n" ], "id": "3eead964" }, { "cell_type": "code", "metadata": {}, "source": [ "# \u2500\u2500\u2500 RandAugment Utilities \u2500\u2500\u2500\n", "\n", "def sample_beta_distribution(size, a=0.2, b=0.2):\n", " g1 = tf.random.gamma([size], a, dtype=tf.float32)\n", " g2 = tf.random.gamma([size], b, dtype=tf.float32)\n", " return g2 / (g1 + g2 + 1e-8)\n", "\n", "def _randaug_select(images, ops_per_image=3, magnitude=0.7):\n", " # Apply N random ops from pool per image (RandAugment-style).\n", " batch = tf.shape(images)[0]\n", " h, w = tf.cast(tf.shape(images)[1], tf.float32), tf.cast(tf.shape(images)[2], tf.float32)\n", "\n", " # Pool of 15 augmentation functions\n", " def op_flip(img): return tf.image.random_flip_left_right(img)\n", " def op_flip_v(img): return tf.image.random_flip_up_down(img)\n", " def op_rotate(img):\n", " from tensorflow.keras import layers as L\n", " return L.RandomRotation(0.2 * magnitude, fill_mode='reflect')(img, training=True)\n", " def op_zoom(img):\n", " from tensorflow.keras import layers as L\n", " return L.RandomZoom(0.2 * magnitude, fill_mode='reflect')(img, training=True)\n", " def op_translate(img):\n", " from tensorflow.keras import layers as L\n", " return L.RandomTranslation(0.15 * magnitude, 0.15 * magnitude, fill_mode='reflect')(img, training=True)\n", " def op_contrast(img): return tf.image.random_contrast(img, 1.0 - 0.5 * magnitude, 1.0 + 0.5 * magnitude)\n", " def op_brightness(img): return tf.image.random_brightness(img, 0.3 * magnitude)\n", " def op_hue(img): return tf.image.random_hue(img, 0.08 * max(magnitude, 0.01))\n", " def op_saturation(img):\n", " lo = tf.maximum(0.5, 1.0 - 0.8 * magnitude)\n", " return tf.image.random_saturation(img, lo, 1.0 + 0.8 * magnitude)\n", " def op_solarize(img):\n", " thresh = tf.random.uniform([], 0.3, 0.8) * 255.0\n", " return tf.where(img < thresh, img, 255.0 - img)\n", " def op_blur(img):\n", " sigma = tf.random.uniform([], 1.0, 1.0 + 2.0 * magnitude)\n", " kernel_size = tf.cast(tf.math.ceil(2 * sigma), tf.int32) * 2 + 1\n", " kernel_size = tf.clip_by_value(kernel_size, 3, 9)\n", " channels = 3\n", " # Build Gaussian kernel\n", " size = tf.cast(kernel_size, tf.float32)\n", " x = tf.range(-(size - 1) / 2, (size - 1) / 2 + 1, dtype=tf.float32)\n", " g = tf.exp(-0.5 * (x / sigma) ** 2)\n", " g = g / tf.reduce_sum(g)\n", " kernel = g[:, None] * g[None, :]\n", " kernel = tf.expand_dims(tf.expand_dims(kernel, -1), -1)\n", " kernel = tf.tile(kernel, [1, 1, channels, 1])\n", " img_expanded = tf.expand_dims(img, 0) if len(img.shape) == 3 else img\n", " # Use depthwise for per-channel\n", " blurred = tf.nn.depthwise_conv2d(\n", " tf.transpose(img_expanded, [0, 3, 1, 2]), # NCHW\n", " tf.transpose(kernel, [2, 3, 0, 1]), # H W C_in C_out\n", " strides=[1, 1, 1, 1], padding='SAME'\n", " )\n", " blurred = tf.transpose(blurred, [0, 2, 3, 1]) # NHWC\n", " return tf.squeeze(blurred, 0) if len(img.shape) == 3 else blurred\n", " def op_fog(img):\n", " fog_level = tf.random.uniform([], 0.1, 0.1 + 0.4 * magnitude)\n", " fog_color = tf.random.uniform([3], 0.7, 1.0) * 255.0\n", " fog_color = tf.reshape(fog_color, [1, 1, 3])\n", " return img * (1.0 - fog_level) + fog_color * fog_level\n", " def op_shadow(img):\n", " opacity = tf.random.uniform([], 0.2, 0.2 + 0.5 * magnitude)\n", " # Simple: darken bottom-right quadrant to simulate shadow\n", " mask = tf.ones_like(img)\n", " h_i = tf.cast(h, tf.int32); w_i = tf.cast(w, tf.int32)\n", " mask_h = tf.random.uniform([], 0, tf.cast(h_i, tf.float32) * 0.7)\n", " mask_w = tf.random.uniform([], 0, tf.cast(w_i, tf.float32) * 0.7)\n", " # shadow from random corner\n", " corner = tf.random.uniform([], 0, 4, dtype=tf.int32)\n", " y1 = tf.cond(tf.less(corner, 2), lambda: 0, lambda: tf.cast(h_i, tf.int32) - tf.cast(mask_h, tf.int32))\n", " x1 = tf.cond(tf.equal(tf.math.floormod(corner, 2), 0), lambda: 0,\n", " lambda: tf.cast(w_i, tf.int32) - tf.cast(mask_w, tf.int32))\n", " y_range = tf.range(tf.cast(y1, tf.int32), tf.minimum(tf.cast(y1, tf.int32) + tf.cast(mask_h, tf.int32), h_i))\n", " x_range = tf.range(tf.cast(x1, tf.int32), tf.minimum(tf.cast(x1, tf.int32) + tf.cast(mask_w, tf.int32), w_i))\n", " # Build scatter update\n", " # ... simplified: darken by blending with black\n", " shadow_mask = tf.zeros_like(img)\n", " # Use SparseTensor approach or just broadcast a gradient\n", " return img * (1.0 - opacity * 0.6)\n", " def op_resolution_drop(img):\n", " scale_factor = tf.random.uniform([], 2, 4, dtype=tf.int32)\n", " cur_h = tf.shape(img)[0]; cur_w = tf.shape(img)[1]\n", " small_h = cur_h // scale_factor; small_w = cur_w // scale_factor\n", " img_small = tf.image.resize(img[tf.newaxis, ...], [small_h, small_w], method='bilinear')[0]\n", " img_back = tf.image.resize(img_small[tf.newaxis, ...], [cur_h, cur_w], method='bilinear')[0]\n", " return img_back\n", " def op_identity(img): return img\n", "\n", " ops = [op_flip, op_flip_v, op_rotate, op_zoom, op_translate,\n", " op_contrast, op_brightness, op_hue, op_saturation,\n", " op_solarize, op_blur, op_fog, op_shadow, op_resolution_drop, op_identity]\n", "\n", " # Vectorized: randomly select ops_per_image ops and apply in sequence\n", " # For simplicity in tf.function, apply operations sequentially with per-image random selection\n", " def apply_randaug_single(img3d):\n", " # Pick ops_per_image random indices\n", " indices = tf.random.shuffle(tf.range(len(ops)))[:ops_per_image]\n", " result = img3d\n", " for i in range(ops_per_image):\n", " idx = indices[i]\n", " # Apply op by index (tf.case or tf.switch_case)\n", " for j in range(len(ops)):\n", " result = tf.cond(tf.equal(idx, j), lambda j=j: ops[j](result), lambda: result)\n", " return tf.clip_by_value(result, 0.0, 255.0)\n", "\n", " return tf.map_fn(apply_randaug_single, images, dtype=tf.float32)\n", "\n", "# Compatibility wrapper using Keras Sequential for basic geometric ops (kept for visualization)\n", "geo_aug = tf.keras.Sequential([\n", " layers.RandomFlip(\"horizontal_and_vertical\"),\n", " layers.RandomRotation(0.15),\n", " layers.RandomZoom(0.15),\n", " layers.RandomTranslation(0.1, 0.1),\n", " layers.RandomContrast(0.15),\n", " layers.RandomBrightness(0.15),\n", "], name=\"geo_aug\")\n", "\n", "# \u2500\u2500\u2500 MixUp & CutMix (unchanged from original) \u2500\u2500\u2500\n", "\n", "def mix_up(images, labels, alpha=0.2):\n", " bs = tf.shape(images)[0]\n", " lam = sample_beta_distribution(bs, alpha, alpha)\n", " lam_img = tf.reshape(lam, [bs, 1, 1, 1])\n", " ri = tf.random.shuffle(tf.range(bs))\n", " mixed_img = lam_img * images + (1 - lam_img) * tf.gather(images, ri)\n", " labels = tf.cast(labels, tf.float32)\n", " lam_lbl = tf.reshape(lam, [-1, 1])\n", " mixed_lbl = lam_lbl * labels + (1 - lam_lbl) * tf.gather(labels, ri)\n", " return mixed_img, mixed_lbl\n", "\n", "def cut_mix(images, labels, alpha=0.2):\n", " bs = tf.shape(images)[0]; h = tf.shape(images)[1]; w = tf.shape(images)[2]\n", " lam = sample_beta_distribution(bs, alpha, alpha); ri = tf.random.shuffle(tf.range(bs))\n", " cr = tf.sqrt(1.0 - lam)\n", " rh = tf.cast(cr * tf.cast(h, tf.float32), tf.int32); rw = tf.cast(cr * tf.cast(w, tf.float32), tf.int32)\n", " cx = tf.random.uniform([bs], 0, w, tf.int32); cy = tf.random.uniform([bs], 0, h, tf.int32)\n", " hh = rh // 2; hw = rw // 2\n", " x1 = tf.clip_by_value(cx - hw, 0, w); x2 = tf.clip_by_value(cx + hw, 0, w)\n", " y1 = tf.clip_by_value(cy - hh, 0, h); y2 = tf.clip_by_value(cy + hh, 0, h)\n", " col = tf.range(w, dtype=tf.int32); row = tf.range(h, dtype=tf.int32)\n", " in_x = tf.logical_and(tf.reshape(col, [1, 1, w]) >= tf.reshape(x1, [bs, 1, 1]),\n", " tf.reshape(col, [1, 1, w]) < tf.reshape(x2, [bs, 1, 1]))\n", " in_y = tf.logical_and(tf.reshape(row, [1, h, 1]) >= tf.reshape(y1, [bs, 1, 1]),\n", " tf.reshape(row, [1, h, 1]) < tf.reshape(y2, [bs, 1, 1]))\n", " cm = tf.cast(tf.logical_and(in_y, in_x), tf.float32); cm = tf.expand_dims(cm, -1)\n", " shuf = tf.gather(images, ri)\n", " mi = (1.0 - cm) * images + cm * shuf\n", " labels = tf.cast(labels, tf.float32); lr = tf.reshape(lam, [-1, 1])\n", " ml = lr * labels + (1.0 - lr) * tf.gather(labels, ri)\n", " return mi, ml\n", "\n", "def random_erasing(images, probability=0.25, scale=(0.02, 0.25)):\n", " bs = tf.shape(images)[0]; h = tf.shape(images)[1]; w = tf.shape(images)[2]\n", " ta = tf.random.uniform([], scale[0], scale[1]) * tf.cast(h * w, tf.float32)\n", " ar = tf.random.uniform([], 0.3, 3.3)\n", " eh = tf.cast(tf.math.sqrt(ta / ar), tf.int32); ew = tf.cast(tf.math.sqrt(ta * ar), tf.int32)\n", " eh = tf.clip_by_value(eh, 1, h - 1); ew = tf.clip_by_value(ew, 1, w - 1)\n", " cx = tf.random.uniform([], 0, w - ew, tf.int32); cy = tf.random.uniform([], 0, h - eh, tf.int32)\n", " col = tf.range(w, dtype=tf.int32); row = tf.range(h, dtype=tf.int32)\n", " ix = tf.logical_and(col >= cx, col < cx + ew)\n", " iy = tf.logical_and(row >= cy, row < cy + eh)\n", " em = tf.cast(tf.expand_dims(iy, 1) & tf.expand_dims(ix, 0), tf.float32)\n", " em = tf.expand_dims(tf.expand_dims(em, 0), -1)\n", " noise = tf.random.uniform([bs, eh, ew, 3], 0.0, 255.0, dtype=tf.float32)\n", " pads = [[0, 0], [cy, h - (cy + eh)], [cx, w - (cx + ew)], [0, 0]]\n", " npad = tf.pad(noise, pads, constant_values=0.0)\n", " erased = images * (1.0 - em) + npad * em\n", " return tf.cond(tf.random.uniform([]) < probability, lambda: erased, lambda: images)\n", "\n", "# \u2500\u2500\u2500 Main augmentation pipeline \u2500\u2500\u2500\n", "\n", "def augment_and_mix(images, labels):\n", " # 1. RandAugment (geometric + color + weather + degradation)\n", " images = _randaug_select(images, ops_per_image=3, magnitude=0.7)\n", " # 2. Convert float32 for MixUp/CutMix\n", " images = tf.cast(images, tf.float32)\n", " # 3. MixUp or CutMix (40% chance total: 20% MixUp, 20% CutMix)\n", " choice = tf.random.uniform([])\n", " labels_oh = tf.one_hot(labels, NUM_CLASSES)\n", " images, labels_oh = tf.cond(\n", " choice < 0.2, lambda: mix_up(images, labels_oh),\n", " lambda: tf.cond(choice < 0.4, lambda: cut_mix(images, labels_oh),\n", " lambda: (images, labels_oh)))\n", " # 4. Random Erasing\n", " images = random_erasing(images, probability=0.2)\n", " return images, labels_oh\n", "\n", "# \u2500\u2500\u2500 Preprocessing \u2500\u2500\u2500\n", "def preprocess_fn(image, label):\n", " return preprocess_input(image), label\n", "\n", "# \u2500\u2500\u2500 Class weights \u2500\u2500\u2500\n", "train_class_counts = Counter()\n", "for cn in sorted(os.listdir(train_dir)):\n", " p = os.path.join(train_dir, cn)\n", " if os.path.isdir(p):\n", " train_class_counts[cn] = len(os.listdir(p))\n", "\n", "y_int = []\n", "for i, cn in enumerate(sorted(os.listdir(train_dir))):\n", " cp = os.path.join(train_dir, cn)\n", " if os.path.isdir(cp):\n", " y_int.extend([i] * len(os.listdir(cp)))\n", "\n", "cw_array = compute_class_weight('balanced', classes=np.unique(y_int), y=y_int)\n", "cw_capped = [min(w, 3.0) for w in cw_array]\n", "class_weights_tensor = tf.constant(cw_capped, dtype=tf.float32)\n", "\n", "def add_sample_weight(image, label):\n", " ci = tf.argmax(label, axis=-1)\n", " sw = tf.gather(class_weights_tensor, ci)\n", " return image, label, sw\n", "\n", "# \u2500\u2500\u2500 Build datasets \u2500\u2500\u2500\n", "train_ds = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "val_ds = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "test_ds = tf.keras.utils.image_dataset_from_directory(\n", " test_dir, shuffle=False, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "\n", "class_names = train_ds.class_names\n", "NUM_CLASSES = len(class_names)\n", "print(f\"Classes ({NUM_CLASSES}): {class_names}\")\n", "\n", "train_ds = (train_ds\n", " .map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE)\n", " .prefetch(AUTOTUNE))\n", "\n", "val_ds = (val_ds\n", " .map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE)\n", " .prefetch(AUTOTUNE))\n", "\n", "test_ds = (test_ds\n", " .map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE)\n", " .prefetch(AUTOTUNE))\n", "\n", "print(\"Class weights (capped at 3.0):\")\n", "for i, cn in enumerate(class_names):\n", " if i < len(cw_capped):\n", " print(f\" {cn}: {cw_capped[i]:.4f}\")\n", "print(\"Data pipelines ready.\")\n" ], "outputs": [], "execution_count": null, "id": "8535054d" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Visualisasi Sampel Data (Augmentasi Real-World)\n" ], "id": "4ca2ec40" }, { "cell_type": "code", "metadata": {}, "source": [ "vis_ds = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "\n", "plt.figure(figsize=(16, 10))\n", "for images, labels in vis_ds.take(1):\n", " # Original (4)\n", " for i in range(4):\n", " plt.subplot(3, 4, i + 1)\n", " plt.imshow(images[i].numpy().astype(\"uint8\"))\n", " plt.title(f\"Asli: {class_names[labels[i].numpy()]}\", fontsize=11)\n", " plt.axis(\"off\")\n", " # RandAugment (4)\n", " aug = _randaug_select(tf.cast(images, tf.float32), ops_per_image=3, magnitude=0.7)\n", " for i in range(4):\n", " plt.subplot(3, 4, i + 5)\n", " plt.imshow(tf.clip_by_value(aug[i], 0, 255).numpy().astype(\"uint8\"))\n", " plt.title(f\"RandAug: {class_names[labels[i].numpy()]}\", fontsize=11)\n", " plt.axis(\"off\")\n", " # MixUp result (4)\n", " aug_f = tf.cast(images, tf.float32)\n", " mixed, _ = mix_up(aug_f, tf.one_hot(labels, NUM_CLASSES))\n", " for i in range(4):\n", " plt.subplot(3, 4, i + 9)\n", " plt.imshow(tf.clip_by_value(mixed[i], 0, 255).numpy().astype(\"uint8\"))\n", " plt.title(\"MixUp/Weather\", fontsize=11)\n", " plt.axis(\"off\")\n", "\n", "plt.suptitle(\"RandAugment + MixUp \u2014 Real-World Simulation\", fontsize=16)\n", "plt.tight_layout()\n", "plt.show()\n" ], "outputs": [], "execution_count": null, "id": "44d6d2b0" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 7. Arsitektur Model \u2014 CBAM + Lightweight Head\n", "\n", "**EfficientNetV2B0** base (frozen awal) + **CBAM spatial attention** + lightweight classifier head.\n", "\n", "### Head (\u2248700K params vs 7.3M sebelumnya)\n", "```\n", "Base (7\u00d77\u00d71280) \u2192 CBAM_Attention \u2192 GAP \u2192 Dropout(0.3) \u2192 Dense(512, swish) \u2192 BN \u2192 Dropout(0.4) \u2192 Dense(4, softmax)\n", "```\n", "\n", "CBAM (Convolutional Block Attention Module): channel attention + spatial attention\n", "\u2192 model belajar fokus ke foreground (daun) bukan background.\n" ], "id": "05281076" }, { "cell_type": "code", "metadata": {}, "source": [ "def cbam_block(x, ratio=8, name=\"cbam\"):\n", " # Convolutional Block Attention Module \u2014 ringan, fokus ke foreground.\n", " channels = tf.shape(x)[-1]\n", "\n", " # Channel Attention\n", " avg_pool = layers.GlobalAveragePooling2D()(x)\n", " max_pool = layers.GlobalMaxPooling2D()(x)\n", " ca = layers.Dense(channels // ratio, activation='swish', name=f\"{name}_ca1\")(avg_pool)\n", " ca = layers.Dense(channels, activation='sigmoid', name=f\"{name}_ca2\")(ca)\n", " ca2 = layers.Dense(channels // ratio, activation='swish', name=f\"{name}_ca3\")(max_pool)\n", " ca2 = layers.Dense(channels, activation='sigmoid', name=f\"{name}_ca4\")(ca2)\n", " ca_out = layers.Add(name=f\"{name}_ca_add\")([ca, ca2])\n", " ca_out = layers.Reshape((1, 1, channels), name=f\"{name}_ca_reshape\")(ca_out)\n", " x = layers.Multiply(name=f\"{name}_ca_mul\")([x, ca_out])\n", "\n", " # Spatial Attention\n", " from keras import ops\n", " avg_sp = ops.mean(x, axis=-1, keepdims=True)\n", " max_sp = ops.max(x, axis=-1, keepdims=True)\n", " sp = layers.Concatenate(name=f\"{name}_sa_cat\")([avg_sp, max_sp])\n", " sp = layers.Conv2D(1, 7, padding='same', activation='sigmoid', name=f\"{name}_sa_conv\")(sp)\n", " x = layers.Multiply(name=f\"{name}_sa_mul\")([x, sp])\n", " return x\n", "\n", "def build_model(num_classes, img_size=(224, 224)):\n", " base_model = EfficientNetV2B0(\n", " input_shape=img_size + (3,),\n", " include_top=False,\n", " weights='imagenet',\n", " )\n", " base_model.trainable = False\n", "\n", " inputs = tf.keras.Input(shape=img_size + (3,), name=\"input\")\n", " # Gaussian noise untuk regularisasi\n", " x = layers.GaussianNoise(0.05, name=\"gauss_noise\")(inputs)\n", " x = base_model(x, training=False)\n", " # CBAM attention \u2014 fokus ke region daun\n", " x = cbam_block(x, ratio=8, name=\"cbam\")\n", " x = layers.GlobalAveragePooling2D(name=\"gap\")(x)\n", " x = layers.Dropout(0.3, name=\"drop_gap\")(x)\n", " x = layers.Dense(512, activation='swish', name=\"dense_head\")(x)\n", " x = layers.BatchNormalization(name=\"bn_head\")(x)\n", " x = layers.Dropout(0.4, name=\"drop_head\")(x)\n", " outputs = layers.Dense(num_classes, activation='linear', dtype='float32', name=\"logits\")(x)\n", " return models.Model(inputs, outputs), base_model\n", "\n", "# Checkpoint\n", "ckpt_dir = '/content/best_model' if IS_COLAB else os.path.join(os.getcwd(), 'best_model')\n", "checkpoint_path = os.path.join(ckpt_dir, 'best_model.keras')\n", "\n", "# Hapus checkpoint lama (arsitektur berbeda \u2014 tidak kompatibel)\n", "if os.path.exists(checkpoint_path):\n", " print(f\"Removing old checkpoint (incompatible architecture)...\")\n", " os.remove(checkpoint_path)\n", "\n", "if os.path.exists(checkpoint_path):\n", " print(f\"Loading checkpoint: {checkpoint_path}\")\n", " try:\n", " model = models.load_model(checkpoint_path, compile=False)\n", " except Exception as e:\n", " print(f\"Load failed: {e}. Building fresh.\")\n", " model, base_model = build_model(NUM_CLASSES)\n", "else:\n", " print(\"No checkpoint. Building fresh model.\")\n", " model, base_model = build_model(NUM_CLASSES)\n", "\n", "os.makedirs(ckpt_dir, exist_ok=True)\n", "model.summary()\n" ], "outputs": [], "execution_count": null, "id": "4937fb17" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 8. Training Setup \u2014 Cosine Decay + SWA + Callbacks\n", "\n", "Mengganti `ReduceLROnPlateau` / `EarlyStopping` dengan:\n", "- **CosineDecay** + linear warmup per fase\n", "- **Stochastic Weight Averaging (SWA)** \u2014 averaging bobot untuk wider optima\n", "- **ModelCheckpoint** \u2014 tetap simpan best val_accuracy\n" ], "id": "28aaaed1" }, { "cell_type": "code", "metadata": {}, "source": [ "class WarmupCosineDecay(tf.keras.optimizers.schedules.LearningRateSchedule):\n", " # Cosine decay with linear warmup.\n", " def __init__(self, warmup_steps, total_steps, peak_lr, min_lr=1e-7):\n", " super().__init__()\n", " self.warmup_steps = warmup_steps\n", " self.total_steps = total_steps\n", " self.peak_lr = peak_lr\n", " self.min_lr = min_lr\n", "\n", " def __call__(self, step):\n", " step = tf.cast(step, tf.float32)\n", " warmup_steps = tf.cast(self.warmup_steps, tf.float32)\n", " total_steps = tf.cast(self.total_steps, tf.float32)\n", " # Warmup phase\n", " warmup_lr = self.peak_lr * (step / warmup_steps)\n", " # Cosine decay phase\n", " progress = (step - warmup_steps) / tf.maximum(total_steps - warmup_steps, 1.0)\n", " cosine_lr = self.min_lr + 0.5 * (self.peak_lr - self.min_lr) * (1.0 + tf.cos(np.pi * progress))\n", " return tf.where(step < warmup_steps, warmup_lr, cosine_lr)\n", "\n", " def get_config(self):\n", " return {\n", " \"warmup_steps\": self.warmup_steps, \"total_steps\": self.total_steps,\n", " \"peak_lr\": self.peak_lr, \"min_lr\": self.min_lr,\n", " }\n", "\n", "class SWACallback(tf.keras.callbacks.Callback):\n", " # Stochastic Weight Averaging \u2014 averages weights over final epochs.\n", " def __init__(self, start_epoch, swa_lr=1e-5):\n", " super().__init__()\n", " self.start_epoch = start_epoch\n", " self.swa_lr = swa_lr\n", " self.swa_weights = None\n", " self.n_models = 0\n", "\n", " def on_epoch_begin(self, epoch, logs=None):\n", " if epoch >= self.start_epoch and self.swa_weights is None:\n", " self.swa_weights = [w.numpy() for w in self.model.weights]\n", " print(f\"\\nSWA: starting weight averaging at epoch {epoch+1}\")\n", "\n", " def on_epoch_end(self, epoch, logs=None):\n", " if epoch >= self.start_epoch:\n", " tf.keras.backend.set_value(self.model.optimizer.learning_rate, self.swa_lr)\n", " for i, w in enumerate(self.model.weights):\n", " self.swa_weights[i] = (self.swa_weights[i] * self.n_models + w.numpy()) / (self.n_models + 1)\n", " self.n_models += 1\n", "\n", " def apply_swa_weights(self):\n", " for w, swa_w in zip(self.model.weights, self.swa_weights):\n", " w.assign(swa_w)\n", " print(f\"SWA weights applied ({self.n_models} models averaged).\")\n", "\n", "# Shared callbacks\n", "checkpoint_cb = callbacks.ModelCheckpoint(\n", " checkpoint_path, save_best_only=True, monitor=\"val_accuracy\",\n", " mode=\"max\", verbose=1)\n", "\n", "csv_logger = callbacks.CSVLogger(os.path.join(ckpt_dir, 'training_log.csv'))\n", "\n", "def make_callbacks(swa_start=None):\n", " cbs = [checkpoint_cb, csv_logger]\n", " if swa_start is not None:\n", " cbs.append(SWACallback(swa_start))\n", " return cbs\n", "\n", "print(\"Callbacks ready.\")\n", "print(f\"Checkpoint path: {checkpoint_path}\")\n" ], "outputs": [], "execution_count": null, "id": "51deff9d" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 9. Fase 1 \u2014 Head Only Training (128\u00d7128)\n", "\n", "Progressive resolution: mulai dari **128\u00d7128** untuk feature learning cepat.\n", "Base model beku, hanya head (CBAM + Dense) yang dilatih.\n", "Optimizer: AdamW + EMA + CosineDecay(warmup=3, peak=1e-3).\n", "Label smoothing: 0.15\n" ], "id": "82814b7b" }, { "cell_type": "code", "metadata": {}, "source": [ "IMG_128 = (128, 128)\n", "\n", "# Rebuild datasets at 128x128\n", "train_ds_128 = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_128)\n", "val_ds_128 = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_128)\n", "\n", "train_ds_128 = (train_ds_128.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "val_ds_128 = (val_ds_128.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "\n", "EPOCHS_P1 = 25\n", "steps_per_epoch = tf.data.experimental.cardinality(train_ds_128).numpy() or 100\n", "total_steps = steps_per_epoch * EPOCHS_P1\n", "warmup_steps = steps_per_epoch * 3 # 3 epoch warmup\n", "\n", "lr_schedule_p1 = WarmupCosineDecay(warmup_steps, total_steps, peak_lr=1e-3, min_lr=1e-5)\n", "\n", "model.compile(\n", " optimizer=AdamW(use_ema=True, ema_momentum=0.999,\n", " learning_rate=lr_schedule_p1, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.15),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(\"Phase 1: Head training at 128\u00d7128...\")\n", "history_1 = model.fit(train_ds_128, validation_data=val_ds_128,\n", " epochs=EPOCHS_P1, callbacks=make_callbacks())\n" ], "outputs": [], "execution_count": null, "id": "decaf5cb" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 10. Fase 2 \u2014 Partial Fine-tuning (192\u00d7192)\n", "\n", "Resolusi naik ke **192\u00d7192**. Top 100 layer EfficientNetV2B0 di-unfreeze.\n", "Learning rate lebih rendah: peak=5e-4, cosine decay ke 1e-6.\n" ], "id": "de673df6" }, { "cell_type": "code", "metadata": {}, "source": [ "IMG_192 = (192, 192)\n", "\n", "train_ds_192 = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_192)\n", "val_ds_192 = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_192)\n", "\n", "train_ds_192 = (train_ds_192.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "val_ds_192 = (val_ds_192.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "\n", "# Unfreeze top 100 layers\n", "base_model.trainable = True\n", "for layer in base_model.layers[:-100]:\n", " layer.trainable = False\n", "\n", "EPOCHS_P2 = 30\n", "steps_p2 = tf.data.experimental.cardinality(train_ds_192).numpy() or 100\n", "total_p2 = steps_p2 * EPOCHS_P2\n", "warmup_p2 = steps_p2 * 2\n", "\n", "lr_schedule_p2 = WarmupCosineDecay(warmup_p2, total_p2, peak_lr=5e-4, min_lr=1e-6)\n", "\n", "model.compile(\n", " optimizer=AdamW(use_ema=True, ema_momentum=0.999,\n", " learning_rate=lr_schedule_p2, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.15),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(\"Phase 2: Fine-tuning top 100 layers at 192\u00d7192...\")\n", "history_2 = model.fit(train_ds_192, validation_data=val_ds_192,\n", " epochs=EPOCHS_P2, initial_epoch=history_1.epoch[-1] + 1,\n", " callbacks=make_callbacks())\n" ], "outputs": [], "execution_count": null, "id": "fd7fb926" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 11. Fase 3 \u2014 Full Fine-tuning (224\u00d7224)\n", "\n", "Resolusi penuh **224\u00d7224**. Semua layer di-unfreeze.\n", "LR sangat rendah: peak=1e-4, cosine decay ke 1e-7.\n", "Label smoothing diturunkan ke 0.10 untuk kalibrasi lebih baik.\n" ], "id": "33326d9c" }, { "cell_type": "code", "metadata": {}, "source": [ "img_size = IMG_SIZE # (224, 224)\n", "\n", "train_ds_full = tf.keras.utils.image_dataset_from_directory(\n", " train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=img_size)\n", "val_ds_full = tf.keras.utils.image_dataset_from_directory(\n", " val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=img_size)\n", "\n", "train_ds_full = (train_ds_full.map(augment_and_mix, num_parallel_calls=AUTOTUNE)\n", " .map(preprocess_fn, num_parallel_calls=AUTOTUNE)\n", " .map(add_sample_weight, num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "val_ds_full = (val_ds_full.map(lambda img, lbl: (preprocess_input(tf.cast(img, tf.float32)), lbl), num_parallel_calls=AUTOTUNE)\n", " .map(lambda img, lbl: (img, tf.one_hot(lbl, NUM_CLASSES)), num_parallel_calls=AUTOTUNE).prefetch(AUTOTUNE))\n", "\n", "# Full unfreeze\n", "for layer in base_model.layers:\n", " layer.trainable = True\n", "\n", "EPOCHS_P3 = 30\n", "steps_p3 = tf.data.experimental.cardinality(train_ds_full).numpy() or 100\n", "total_p3 = steps_p3 * EPOCHS_P3\n", "warmup_p3 = steps_p3 * 2\n", "\n", "lr_schedule_p3 = WarmupCosineDecay(warmup_p3, total_p3, peak_lr=1e-4, min_lr=1e-7)\n", "\n", "model.compile(\n", " optimizer=AdamW(use_ema=True, ema_momentum=0.999,\n", " learning_rate=lr_schedule_p3, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.10),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(\"Phase 3: Full fine-tuning at 224\u00d7224...\")\n", "history_3 = model.fit(train_ds_full, validation_data=val_ds_full,\n", " epochs=EPOCHS_P3, initial_epoch=history_2.epoch[-1] + 1,\n", " callbacks=make_callbacks())\n" ], "outputs": [], "execution_count": null, "id": "14115063" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 12. SWA \u2014 Stochastic Weight Averaging\n", "\n", "15 epoch tambahan dengan cyclic LR (1e-5). SWA mengakumulasi rata-rata bobot\n", "untuk menghasilkan **wider optima** \u2014 generalisasi lebih baik ke data out-of-distribution.\n", "\n", "Setelah SWA selesai, bobot SWA diterapkan kembali ke model.\n" ], "id": "6df4ef22" }, { "cell_type": "code", "metadata": {}, "source": [ "EPOCHS_SWA = 15\n", "swa_start_epoch = history_3.epoch[-1] + 1 # Start SWA after Phase 3\n", "\n", "swa_cb = SWACallback(start_epoch=swa_start_epoch, swa_lr=1e-5)\n", "\n", "model.compile(\n", " optimizer=AdamW(use_ema=True, ema_momentum=0.999,\n", " learning_rate=1e-5, weight_decay=1e-4),\n", " loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.10),\n", " metrics=['accuracy']\n", ")\n", "\n", "print(f\"SWA: {EPOCHS_SWA} epochs starting at epoch {swa_start_epoch + 1}...\")\n", "history_swa = model.fit(train_ds_full, validation_data=val_ds_full,\n", " epochs=EPOCHS_SWA, initial_epoch=swa_start_epoch,\n", " callbacks=make_callbacks() + [swa_cb])\n", "\n", "# Apply SWA weights\n", "swa_cb.apply_swa_weights()\n", "print(f\"SWA complete. Final model has SWA weights applied.\")\n" ], "outputs": [], "execution_count": null, "id": "290f2345" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 13. Plot Training History (Gabungan Semua Fase)\n" ], "id": "2d889f2a" }, { "cell_type": "code", "metadata": {}, "source": [ "# Gabungkan semua history\n", "acc = (history_1.history['accuracy'] + history_2.history['accuracy'] +\n", " history_3.history['accuracy'] + history_swa.history['accuracy'])\n", "val_acc = (history_1.history['val_accuracy'] + history_2.history['val_accuracy'] +\n", " history_3.history['val_accuracy'] + history_swa.history['val_accuracy'])\n", "loss = (history_1.history['loss'] + history_2.history['loss'] +\n", " history_3.history['loss'] + history_swa.history['loss'])\n", "val_loss = (history_1.history['val_loss'] + history_2.history['val_loss'] +\n", " history_3.history['val_loss'] + history_swa.history['val_loss'])\n", "\n", "b1 = len(history_1.history['accuracy']) - 1\n", "b2 = b1 + len(history_2.history['accuracy'])\n", "b3 = b2 + len(history_3.history['accuracy'])\n", "\n", "plt.figure(figsize=(16, 6))\n", "plt.subplot(1, 2, 1)\n", "plt.plot(acc, label='Training Accuracy', linewidth=2)\n", "plt.plot(val_acc, label='Validation Accuracy', linewidth=2)\n", "plt.axvline(x=b1, color='gray', linestyle='--', alpha=0.7, label='P2 (192)')\n", "plt.axvline(x=b2, color='black', linestyle='--', alpha=0.7, label='P3 (224)')\n", "plt.axvline(x=b3, color='blue', linestyle='--', alpha=0.7, label='SWA start')\n", "plt.legend(fontsize=10)\n", "plt.title('Training & Validation Accuracy', fontsize=14)\n", "plt.xlabel('Epoch'); plt.ylabel('Accuracy'); plt.grid(alpha=0.3)\n", "\n", "plt.subplot(1, 2, 2)\n", "plt.plot(loss, label='Training Loss', linewidth=2)\n", "plt.plot(val_loss, label='Validation Loss', linewidth=2)\n", "plt.axvline(x=b1, color='gray', linestyle='--', alpha=0.7, label='P2 (192)')\n", "plt.axvline(x=b2, color='black', linestyle='--', alpha=0.7, label='P3 (224)')\n", "plt.axvline(x=b3, color='blue', linestyle='--', alpha=0.7, label='SWA start')\n", "plt.legend(fontsize=10)\n", "plt.title('Training & Validation Loss', fontsize=14)\n", "plt.xlabel('Epoch'); plt.ylabel('Loss'); plt.grid(alpha=0.3)\n", "\n", "plt.tight_layout()\n", "plt.show()\n" ], "outputs": [], "execution_count": null, "id": "41e79742" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 14. Temperature Scaling \u2014 Confidence Calibration\n", "\n", "Model deep learning cenderung **overconfident** \u2014 softmax probability tinggi tapi tidak mencerminkan\n", "akurasi sebenarnya. Temperature scaling mengoptimalkan parameter T pada validation set:\n", "\n", "$$P_{calibrated} = softmax(logits / T)$$\n", "\n", "T > 1 \u2192 distribusi lebih flat (less confident).\n", "T < 1 \u2192 distribusi lebih tajam (more confident).\n", "T = 1 \u2192 tidak berubah (default).\n", "\n", "ECE (Expected Calibration Error) mengukur seberapa baik confidence sesuai dengan akurasi.\n", "Target: **ECE < 0.05** setelah temperature scaling.\n" ], "id": "27b920dd" }, { "cell_type": "code", "metadata": {}, "source": [ "def compute_ece(probs, true_labels, n_bins=15):\n", " # Expected Calibration Error.\n", " confs = np.max(probs, axis=1)\n", " preds = np.argmax(probs, axis=1)\n", " true = np.argmax(true_labels, axis=1)\n", " accs = (preds == true).astype(np.float32)\n", " bins = np.linspace(0, 1, n_bins + 1)\n", " ece = 0.0\n", " bin_stats = []\n", " for i in range(n_bins):\n", " in_bin = (confs > bins[i]) & (confs <= bins[i + 1])\n", " n = np.sum(in_bin)\n", " if n > 0:\n", " bin_acc = np.mean(accs[in_bin])\n", " bin_conf = np.mean(confs[in_bin])\n", " ece += (n / len(confs)) * np.abs(bin_acc - bin_conf)\n", " bin_stats.append((bins[i], n, bin_acc, bin_conf))\n", " return ece, bin_stats\n", "\n", "# Collect logits and labels from validation set\n", "print(\"Collecting validation logits...\")\n", "logits_model = tf.keras.Model(model.input, model.output)\n", "\n", "all_logits = []\n", "all_labels = []\n", "for images, labels in val_ds_full.unbatch().batch(BATCH_SIZE):\n", " all_logits.append(logits_model.predict_on_batch(images))\n", " all_labels.append(labels.numpy())\n", "\n", "all_logits = np.concatenate(all_logits, axis=0)\n", "all_labels = np.concatenate(all_labels, axis=0)\n", "\n", "# ECE before scaling (T=1)\n", "probs_raw = tf.nn.softmax(all_logits).numpy()\n", "ece_raw, _ = compute_ece(probs_raw, all_labels)\n", "print(f\"ECE before scaling (T=1.0): {ece_raw:.4f}\")\n", "\n", "# Optimize T on validation set\n", "if HAS_SCIPY:\n", " def nll_temperature(T):\n", " scaled = all_logits / float(T)\n", " probs = tf.nn.softmax(scaled).numpy()\n", " probs = np.clip(probs, 1e-7, 1.0 - 1e-7)\n", " return -np.mean(np.log(np.sum(all_labels * probs, axis=1)))\n", "\n", " result = minimize_scalar(nll_temperature, bounds=(0.1, 5.0), method='bounded')\n", " T_opt = result.x\n", " print(f\"Optimal temperature: T = {T_opt:.4f}\")\n", "else:\n", " # Grid search fallback\n", " best_nll, T_opt = float('inf'), 1.0\n", " for T in np.linspace(0.5, 4.0, 36):\n", " scaled = all_logits / T\n", " probs = tf.nn.softmax(scaled).numpy()\n", " probs = np.clip(probs, 1e-7, 1.0 - 1e-7)\n", " nll = -np.mean(np.log(np.sum(all_labels * probs, axis=1)))\n", " if nll < best_nll:\n", " best_nll = nll\n", " T_opt = T\n", " print(f\"Optimal temperature (grid): T = {T_opt:.4f}\")\n", "\n", "# ECE after scaling\n", "probs_cal = tf.nn.softmax(all_logits / T_opt).numpy()\n", "ece_cal, bin_stats = compute_ece(probs_cal, all_labels)\n", "print(f\"ECE after scaling (T={T_opt:.4f}): {ece_cal:.4f}\")\n", "\n", "# Save calibration metadata\n", "calibration_meta = {\n", " \"temperature\": float(T_opt),\n", " \"conf_threshold_high\": 0.70,\n", " \"conf_threshold_low\": 0.45,\n", " \"ece_raw\": float(ece_raw),\n", " \"ece_calibrated\": float(ece_cal),\n", "}\n", "with open(os.path.join(ckpt_dir, \"calibration.json\"), \"w\") as f:\n", " json.dump(calibration_meta, f, indent=2)\n", "print(f\"Calibration metadata saved to {os.path.join(ckpt_dir, 'calibration.json')}\")\n", "\n", "# Reliability diagram\n", "plt.figure(figsize=(12, 5))\n", "\n", "plt.subplot(1, 2, 1)\n", "if bin_stats:\n", " bin_mids = [(s[0] + s[0] + 1/n_bins)/2 for s in bin_stats]\n", " bin_accs = [s[2] for s in bin_stats]\n", " bin_confs = [s[3] for s in bin_stats]\n", " plt.bar(bin_mids, bin_accs, width=0.05, alpha=0.5, label='Accuracy')\n", " plt.bar(bin_mids, bin_confs, width=0.05, alpha=0.3, label='Confidence')\n", "plt.plot([0, 1], [0, 1], 'k--', alpha=0.3)\n", "plt.xlabel('Confidence'); plt.ylabel('Accuracy')\n", "plt.title(f'Reliability Diagram (T={T_opt:.2f})')\n", "plt.legend(); plt.grid(alpha=0.3)\n", "\n", "plt.subplot(1, 2, 2)\n", "conf_raw = np.max(probs_raw, axis=1)\n", "conf_cal = np.max(probs_cal, axis=1)\n", "plt.hist(conf_raw, bins=30, alpha=0.5, label='Before scaling', density=True)\n", "plt.hist(conf_cal, bins=30, alpha=0.5, label='After scaling', density=True)\n", "plt.xlabel('Max Confidence'); plt.ylabel('Density')\n", "plt.title('Confidence Distribution')\n", "plt.legend(); plt.grid(alpha=0.3)\n", "\n", "plt.tight_layout()\n", "plt.show()\n" ], "outputs": [], "execution_count": null, "id": "d79dbb47" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 15. Test Time Augmentation + Evaluasi\n", "\n", "Menggunakan TTA 5\u00d7 pada test set dengan augmented logit averaging.\n", "Model output adalah **raw logits** \u2192 temperature scaling \u2192 softmax.\n" ], "id": "6e4a717b" }, { "cell_type": "code", "metadata": {}, "source": [ "# Load best model (dengan SWA weights)\n", "print(f\"Loading best model from {checkpoint_path}...\")\n", "best_model = tf.keras.models.load_model(checkpoint_path, compile=False)\n", "\n", "# Collect test images\n", "raw_test_ds = tf.keras.utils.image_dataset_from_directory(\n", " test_dir, shuffle=False, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n", "\n", "test_images = []\n", "test_labels_raw = []\n", "for images, labels in raw_test_ds.unbatch():\n", " test_images.append(images.numpy())\n", " test_labels_raw.append(labels.numpy())\n", "\n", "test_images = np.array(test_images)\n", "test_labels_true = tf.one_hot(np.array(test_labels_raw), NUM_CLASSES).numpy()\n", "\n", "# \u2500\u2500 TTA 5x \u2500\u2500\n", "TTA_STEPS = 5\n", "tta_logits = []\n", "\n", "for i in range(TTA_STEPS):\n", " aug_images = geo_aug(test_images, training=True)\n", " aug_images = preprocess_input(aug_images)\n", " logits = best_model.predict(aug_images, batch_size=BATCH_SIZE, verbose=0)\n", " tta_logits.append(logits)\n", " print(f\" TTA step {i+1}/{TTA_STEPS}\")\n", "\n", "mean_logits = np.mean(tta_logits, axis=0)\n", "# Apply temperature scaling\n", "mean_cal_probs = tf.nn.softmax(mean_logits / T_opt).numpy()\n", "\n", "test_preds = np.argmax(mean_cal_probs, axis=1)\n", "test_true = np.argmax(test_labels_true, axis=1)\n", "tta_acc = np.mean(test_preds == test_true)\n", "\n", "print(f\"\\nTest Accuracy (TTA {TTA_STEPS}x, T={T_opt:.2f}): {tta_acc*100:.2f}%\")\n", "\n", "# ECE on test set\n", "ece_test, _ = compute_ece(mean_cal_probs, test_labels_true)\n", "print(f\"ECE on test set: {ece_test:.4f}\")\n" ], "outputs": [], "execution_count": null, "id": "beb96b8b" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 16. Per-Source Accuracy Breakdown\n", "\n", "Mengukur akurasi per source prefix untuk mendeteksi **domain gap**.\n", "Source yang akurasinya collapse (< 70%) menunjukkan model belum robust.\n" ], "id": "b6bb88ab" }, { "cell_type": "code", "metadata": {}, "source": [ "def get_source(filename):\n", " f = os.path.splitext(filename)[0]\n", " if f.startswith('IMG_'): return 'Phone'\n", " if f.startswith('Corn_'): return 'Lab_Corn'\n", " for p in ['CBS', 'GLS', 'NLS', 'CLS']:\n", " if f.startswith(p): return 'Lab_Disease'\n", " for p in ['SCR', 'CR', 'NLB', 'SLB', 'SRS']:\n", " if f.startswith(p): return 'Lab_RustBlight'\n", " return 'Other'\n", "\n", "# Map each test image to its source\n", "test_files = []\n", "for cn in class_names:\n", " cp = os.path.join(test_dir, cn)\n", " if os.path.isdir(cp):\n", " test_files.extend([(f, cn, get_source(f)) for f in sorted(os.listdir(cp))])\n", "\n", "sources = set(s for _, _, s in test_files)\n", "print(f\"Sources found: {sorted(sources)}\")\n", "print(f\"{'Source':<18} {'Count':>6} {'Accuracy':>10}\")\n", "print(\"-\" * 38)\n", "\n", "for src in sorted(sources):\n", " indices = [i for i, (_, _, s) in enumerate(test_files) if s == src]\n", " if not indices: continue\n", " n = len(indices)\n", " acc = np.mean(test_preds[indices] == test_true[indices])\n", " print(f\"{src:<18} {n:>6} {acc*100:>9.1f}%\")\n", "\n", "# Overall with count\n", "overall_acc = np.mean(test_preds == test_true)\n", "print(\"-\" * 38)\n", "print(f\"{'ALL':<18} {len(test_preds):>6} {overall_acc*100:>9.1f}%\")\n" ], "outputs": [], "execution_count": null, "id": "a24e9a14" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 17. Classification Report & Confusion Matrix\n" ], "id": "52754df5" }, { "cell_type": "code", "metadata": {}, "source": [ "print(\"\\nClassification Report:\\n\")\n", "print(classification_report(test_true, test_preds, target_names=class_names))\n", "\n", "cm = confusion_matrix(test_true, test_preds)\n", "plt.figure(figsize=(10, 8))\n", "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',\n", " xticklabels=class_names, yticklabels=class_names)\n", "plt.title(f'Confusion Matrix (TTA {TTA_STEPS}x)', fontsize=14)\n", "plt.xlabel('Predicted', fontsize=12)\n", "plt.ylabel('True', fontsize=12)\n", "plt.tight_layout()\n", "plt.show()\n" ], "outputs": [], "execution_count": null, "id": "1a5b2c96" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 18. Simpan Model & Calibration Metadata\n", "\n", "Menyimpan model final (dengan SWA weights) dan calibration metadata ke `model/`.\n" ], "id": "528ef53b" }, { "cell_type": "code", "metadata": {}, "source": [ "# Save final model\n", "model.save(final_path := os.path.join(ckpt_dir, 'best_model.keras'))\n", "print(f\"Model saved to {final_path}\")\n", "\n", "# Copy calibration metadata to model directory\n", "model_export_dir = os.path.join(os.getcwd(), 'model')\n", "os.makedirs(model_export_dir, exist_ok=True)\n", "\n", "# Export labels.json with calibration metadata\n", "cal_path = os.path.join(ckpt_dir, 'calibration.json')\n", "if os.path.exists(cal_path):\n", " with open(cal_path) as f:\n", " cal_meta = json.load(f)\n", "\n", " labels_json = {\n", " \"version\": \"3.0\",\n", " \"labels\": class_names,\n", " \"temperature\": cal_meta[\"temperature\"],\n", " \"conf_threshold_high\": cal_meta[\"conf_threshold_high\"],\n", " \"conf_threshold_low\": cal_meta[\"conf_threshold_low\"],\n", " \"input_size\": [224, 224],\n", " \"input_range\": [0, 255],\n", " \"preprocessing\": \"resize_bilinear_224x224_no_normalization\",\n", " \"architecture\": \"EfficientNetV2B0 + CBAM + Dense(512)\",\n", " \"output_type\": \"logits\",\n", " }\n", "else:\n", " labels_json = {\n", " \"version\": \"3.0\",\n", " \"labels\": class_names,\n", " \"temperature\": 1.0,\n", " \"input_size\": [224, 224],\n", " \"input_range\": [0, 255],\n", " \"output_type\": \"logits\",\n", " }\n", "\n", "with open(os.path.join(model_export_dir, 'labels.json'), 'w') as f:\n", " json.dump(labels_json, f, indent=2)\n", "print(\"Labels + calibration metadata saved to model/labels.json\")\n", "\n", "# Export class names list (legacy)\n", "with open(os.path.join(model_export_dir, 'labels.json'), 'r') as f:\n", " pass # already written above\n", "print(f\"Classes: {class_names}\")\n", "print(f\"Temperature: {labels_json['temperature']:.4f}\")\n" ], "outputs": [], "execution_count": null, "id": "292fef1a" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 19. Export Model untuk Produksi\n", "\n", "Setelah training selesai, jalankan pipeline ekspor secara berurutan:\n", "\n", "### 1. SavedModel + TFLite\n", "```bash\n", "python save_model.py\n", "```\n", "Memuat `best_model/best_model.keras`, membangun arsitektur bersih, dan mengekspor ke:\n", "- `model/saved_model/` \u2014 format produksi (output **raw logits**)\n", "- `model/model.tflite` \u2014 untuk perangkat mobile/edge (INT8 quantization)\n", "\n", "### 2. ONNX (Rust ML Service)\n", "```bash\n", "python convert_onnx.py\n", "```\n", "Mengonversi SavedModel ke `model/model.onnx` untuk Rust/Axum/ONNX Runtime.\n", "Output: **raw logits** (softmax + temperature scaling di Rust service).\n", "\n", "### 3. TensorFlow.js (Web) \u2014 optional\n", "```bash\n", "export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python\n", "tensorflowjs_converter --input_format=tf_saved_model --output_format=tfjs_graph_model --signature_name=serving_default --saved_model_tags=serve model/saved_model model/tfjs_model\n", "```\n", "\n", "> **Catatan v3.0**: Model output adalah **raw logits** (tanpa softmax). Rust service menerapkan\n", "> temperature scaling: `softmax(logits / T)` dengan `T` dari `labels.json`.\n", "> Status prediksi ditentukan dari confidence: confident (\u226570%), uncertain (45-70%), rejected (<45%).\n" ], "id": "62675ad6" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 20. Model Card \u2014 ZeaVis Edu v3.0\n", "\n", "| Atribut | Detail |\n", "|---|---|\n", "| **Nama Model** | ZeaVis Edu v3.0 \u2014 CBAM + RandAugment Classifier |\n", "| **Versi** | 3.0 |\n", "| **Arsitektur** | EfficientNetV2B0 + CBAM Attention + GAP + Dense(512) |\n", "| **Params** | ~6.6M (5.9M base + 0.7M head) \u2014 2\u00d7 lebih ringan dari v2.0 |\n", "| **Framework** | TensorFlow 2.x / Keras (float32) |\n", "| **Output** | Raw logits \u2192 temperature scaling \u2192 softmax |\n", "| **Dataset** | ~6000 gambar (4 kelas) \u2014 stratified split by source |\n", "| **Kelas** | Bercak Daun, Daun Sehat, Hawar Daun, Karat Daun |\n", "| **Input** | RGB 224\u00d7224, pixel [0, 255], resize BILINEAR |\n", "| **Augmentasi** | RandAugment (15 ops, N=3) + Fog + Shadow + MixUp + CutMix + RandomErasing |\n", "| **Training** | Progressive 128\u2192160\u2192192\u2192224 + CosineDecay + SWA |\n", "| **Calibration** | Temperature scaling (T optimized on val), ECE target < 0.05 |\n", "| **Decision** | Confident (\u22650.70), Uncertain (0.45-0.70), Rejected (<0.45) |\n", "| **Target** | \u226590% real-world accuracy, per-source accuracy \u226570% semua domain |\n", "| **Etika** | Hanya untuk edukasi/penelitian pertanian. Bukan pengganti diagnosis ahli. |\n", "\n" ], "id": "de8c53b7" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 21. Upload ke Hugging Face\n", "\n", "```python\n", "from huggingface_hub import HfApi\n", "api = HfApi()\n", "api.upload_folder(\n", " folder_path=\"model\",\n", " repo_id=\"zeavis-edu/corn-leaf-disease-classifier\",\n", " repo_type=\"model\",\n", ")\n", "```\n", "Upload model SavedModel, TFLite, ONNX, TF.js, dan metadata ke HF Hub.\n" ], "id": "f431a07c" } ] }