bebas
This commit is contained in:
elytra7 committed
commit
b2659dcd4e
26 files changed
+1631
No files matched your search
@@ -0,0 +1,6 @@
|
|||||||
|
best_model/best_model.keras filter=lfs diff=lfs merge=lfs -text
|
||||||
|
model/saved_model/variables/variables.data-00000-of-00001 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
model/tfjs_model/*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/best_model.keras filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/variables.data* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/tfjs_model/*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
@@ -0,0 +1,364 @@
|
|||||||
|
# Corn Leaf Disease Classification
|
||||||
|
|
||||||
|
Pipeline lengkap untuk klasifikasi penyakit daun jagung menggunakan **EfficientNetV2B0**, mulai dari persiapan dataset, pelatihan di Google Colab, hingga ekspor model ke format **TFLite** dan **TensorFlow.js** untuk kebutuhan produksi.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Daftar Isi
|
||||||
|
|
||||||
|
1. [Kelas yang Diklasifikasikan](#1-kelas-yang-diklasifikasikan)
|
||||||
|
2. [Struktur Proyek](#2-struktur-proyek)
|
||||||
|
3. [Prasyarat & Instalasi](#3-prasyarat--instalasi)
|
||||||
|
4. [Sumber Dataset](#4-sumber-dataset)
|
||||||
|
5. [Tahap 1 — Preprocessing Lokal](#5-tahap-1--preprocessing-lokal)
|
||||||
|
6. [Tahap 2 — Upload ke Google Drive & Training di Colab](#6-tahap-2--upload-ke-google-drive--training-di-colab)
|
||||||
|
7. [Tahap 3 — Download Model dari Colab](#7-tahap-3--download-model-dari-colab)
|
||||||
|
8. [Tahap 4 — Ekspor Model untuk Produksi](#8-tahap-4--ekspor-model-untuk-produksi)
|
||||||
|
9. [Output Akhir](#9-output-akhir)
|
||||||
|
10. [Troubleshooting](#10-troubleshooting)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Kelas yang Diklasifikasikan
|
||||||
|
|
||||||
|
Model dilatih untuk mengenali **4 kelas** kondisi daun jagung:
|
||||||
|
|
||||||
|
| Label (Bahasa Indonesia) | Deskripsi |
|
||||||
|
|---|---|
|
||||||
|
| **Bercak Daun** | *Gray Leaf Spot* — bercak abu-abu memanjang pada daun |
|
||||||
|
| **Hawar Daun** | *Northern/Southern Leaf Blight* — hawar coklat berbentuk panjang |
|
||||||
|
| **Karat Daun** | *Common Rust* — bintik coklat kemerahan berbentuk pustula |
|
||||||
|
| **Daun Sehat** | Daun jagung tanpa gejala penyakit |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Struktur Proyek
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── dataset_1.zip # File ZIP dataset 1 (dari Kaggle)
|
||||||
|
├── dataset_2.zip # File ZIP dataset 2 (dari Kaggle)
|
||||||
|
├── dataset_3.zip # File ZIP dataset 3 (dari SciDB)
|
||||||
|
│
|
||||||
|
├── preprocessing.py # Skrip persiapan & penggabungan dataset
|
||||||
|
├── notebook.ipynb # Notebook pelatihan (dijalankan di Google Colab)
|
||||||
|
├── save_model.py # Skrip ekspor model ke TFLite & TensorFlow.js
|
||||||
|
│
|
||||||
|
├── best_model/
|
||||||
|
│ └── best_model.keras # Model hasil training terbaik (diunduh dari Colab)
|
||||||
|
│
|
||||||
|
├── model/ # Output ekspor (dibuat otomatis)
|
||||||
|
│ ├── saved_model/ # Format SavedModel TensorFlow
|
||||||
|
│ ├── tfjs_model/ # Format TensorFlow.js
|
||||||
|
│ └── model.tflite # Format TFLite untuk Mobile
|
||||||
|
│
|
||||||
|
├── requirements.txt # Daftar dependensi Python
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Prasyarat & Instalasi
|
||||||
|
|
||||||
|
### Kebutuhan Sistem
|
||||||
|
- Python **3.9 – 3.11**
|
||||||
|
- Akun **Google** (untuk Google Drive & Google Colab)
|
||||||
|
- Akun **Kaggle** (untuk mengunduh dataset)
|
||||||
|
- Koneksi internet
|
||||||
|
|
||||||
|
### Instalasi Dependensi
|
||||||
|
|
||||||
|
Disarankan menggunakan virtual environment:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Buat environment baru (opsional tapi dianjurkan)
|
||||||
|
python -m venv venv
|
||||||
|
source venv/bin/activate # Linux / macOS
|
||||||
|
# venv\Scripts\activate # Windows
|
||||||
|
|
||||||
|
# Instal semua dependensi
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Sumber Dataset
|
||||||
|
|
||||||
|
Proyek ini menggabungkan **3 dataset** dari sumber berbeda untuk menghasilkan data yang lebih beragam dan representatif.
|
||||||
|
|
||||||
|
### Dataset 1 — Kaggle (Corn Leaf Disease - Indonesia)
|
||||||
|
> 🔗 https://www.kaggle.com/datasets/ndisan/corn-leaf-disease
|
||||||
|
|
||||||
|
Berisi gambar penyakit daun jagung dengan label dalam Bahasa Indonesia. Dataset ini memiliki **4 folder**, namun label **"Karat Daun" tidak digunakan** karena gambar di dalamnya tidak merepresentasikan penyakit karat yang sebenarnya.
|
||||||
|
|
||||||
|
| Folder di Dataset 1 | Tindakan |
|
||||||
|
|---|---|
|
||||||
|
| `Bercak Daun/` | Digunakan |
|
||||||
|
| `Hawar Daun/` | Digunakan |
|
||||||
|
| `Daun Sehat/` | Digunakan (sebagai tambahan) |
|
||||||
|
| `Karat Daun/` | **Dibuang** — data tidak representatif |
|
||||||
|
|
||||||
|
### Dataset 2 — Kaggle (Corn or Maize Leaf Disease)
|
||||||
|
> 🔗 https://www.kaggle.com/datasets/smaranjitghose/corn-or-maize-leaf-disease-dataset
|
||||||
|
|
||||||
|
Digunakan untuk **menggantikan** data Karat Daun dari Dataset 1 dan menambah variasi gambar Daun Sehat.
|
||||||
|
|
||||||
|
| Folder di Dataset 2 | Dipetakan ke Label |
|
||||||
|
|---|---|
|
||||||
|
| `Common_Rust/` | **Karat Daun** |
|
||||||
|
| `Healthy/` | **Daun Sehat** (tambahan) |
|
||||||
|
| `Blight/` | Tidak diproses |
|
||||||
|
| `Gray_Leaf_Spot/` | Tidak diproses |
|
||||||
|
|
||||||
|
### Dataset 3 — SciDB (China Agricultural Dataset)
|
||||||
|
> 🔗 https://www.scidb.cn/en/detail?dataSetId=19536c73f6d74946a212719a94f53ab3
|
||||||
|
|
||||||
|
Dataset dengan label berbahasa Mandarin. Digunakan untuk **menambah variasi data** pada tiga kelas utama. Pemetaan label dilakukan menggunakan file `desc.json` yang disertakan dalam dataset.
|
||||||
|
|
||||||
|
| Label Mandarin | Dipetakan ke Label |
|
||||||
|
|---|---|
|
||||||
|
| `大斑病` (Hawar Besar) | **Hawar Daun** |
|
||||||
|
| `小斑病` (Hawar Kecil) | **Hawar Daun** |
|
||||||
|
| `褐斑病` (Bercak Coklat) | **Bercak Daun** |
|
||||||
|
| `弯孢霉叶斑病` (Bercak Curvularia) | **Bercak Daun** |
|
||||||
|
| `圆斑病` (Bercak Bulat) | **Bercak Daun** |
|
||||||
|
| `灰斑病` (Bercak Abu-abu) | **Bercak Daun** |
|
||||||
|
| `南方锈病` (Karat Selatan) | **Karat Daun** |
|
||||||
|
| `普通锈病` (Karat Biasa) | **Karat Daun** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Tahap 1 — Preprocessing Lokal
|
||||||
|
|
||||||
|
Skrip `preprocessing.py` menjalankan **5 tahap** secara berurutan untuk menyiapkan dataset akhir yang siap diunggah ke Google Drive.
|
||||||
|
|
||||||
|
### Persiapan
|
||||||
|
|
||||||
|
Unduh ketiga dataset dari sumber di atas, lalu letakkan file ZIP-nya di direktori yang sama dengan `preprocessing.py`:
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── preprocessing.py
|
||||||
|
├── dataset_1.zip
|
||||||
|
├── dataset_2.zip
|
||||||
|
└── dataset_3.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
### Menjalankan Preprocessing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python preprocessing.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### Alur Kerja Otomatis
|
||||||
|
|
||||||
|
#### Tahap 1 — Ekstraksi ZIP
|
||||||
|
Mengekstrak ketiga file `.zip` menjadi folder masing-masing (`dataset_1/`, `dataset_2/`, `dataset_3/`).
|
||||||
|
|
||||||
|
#### Tahap 2 — Menggabungkan Dataset 1 & 2
|
||||||
|
- Menyalin folder `Bercak Daun`, `Hawar Daun`, dan `Daun Sehat` dari Dataset 1 ke folder `dataset/`.
|
||||||
|
- **Folder `Karat Daun` dari Dataset 1 dilewati** karena gambarnya tidak representatif.
|
||||||
|
- Mengambil gambar `Common_Rust` dari Dataset 2 → disimpan ke `dataset/Karat Daun/`.
|
||||||
|
- Mengambil gambar `Healthy` dari Dataset 2 → digabung ke `dataset/Daun Sehat/`.
|
||||||
|
|
||||||
|
#### Tahap 3 — Menggabungkan Dataset 3
|
||||||
|
- Membaca file `desc.json` dari Dataset 3 yang berisi pasangan nama gambar dan labelnya.
|
||||||
|
- Memetakan label Mandarin ke label target (lihat tabel di atas).
|
||||||
|
- Menyalin gambar yang relevan ke folder `dataset/` yang sesuai.
|
||||||
|
|
||||||
|
#### Tahap 4 — Pembersihan Data (Cleaning)
|
||||||
|
Menghapus file gambar spesifik yang bermasalah (rusak, duplikat, atau tidak relevan):
|
||||||
|
- `CBS28.jpg`
|
||||||
|
- `Corn_Common_Rust (1275).jpg`
|
||||||
|
- `Corn_Common_Rust (1289).jpg`
|
||||||
|
- `Corn_Common_Rust (1295).jpg`
|
||||||
|
- `Corn_Gray_Spot (1).jpg`
|
||||||
|
|
||||||
|
#### Tahap 5 — Kompresi ke ZIP
|
||||||
|
Mengemas seluruh folder `dataset/` menjadi `dataset.zip` yang siap diunggah ke Google Drive.
|
||||||
|
|
||||||
|
### Hasil Akhir Preprocessing
|
||||||
|
|
||||||
|
```
|
||||||
|
dataset/
|
||||||
|
├── Bercak Daun/ ← DS1 + DS3
|
||||||
|
├── Hawar Daun/ ← DS1 + DS3
|
||||||
|
├── Karat Daun/ ← DS2 (Common Rust) + DS3
|
||||||
|
└── Daun Sehat/ ← DS1 + DS2 (Healthy)
|
||||||
|
|
||||||
|
dataset.zip ← File siap upload ke Google Drive
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Tahap 2 — Upload ke Google Drive & Training di Colab
|
||||||
|
|
||||||
|
### Langkah 1: Upload `dataset.zip` ke Google Drive
|
||||||
|
|
||||||
|
1. Buka [Google Drive](https://drive.google.com) di browser.
|
||||||
|
2. Upload file `dataset.zip` ke lokasi yang mudah diingat, misalnya:
|
||||||
|
```
|
||||||
|
My Drive/corn_disease/dataset.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
### Langkah 2: Buka Notebook di Google Colab
|
||||||
|
|
||||||
|
1. Upload `notebook.ipynb` ke Google Drive, atau buka langsung dari Colab:
|
||||||
|
- Klik kanan file `notebook.ipynb` di Google Drive → **"Open with"** → **Google Colaboratory**.
|
||||||
|
2. Aktifkan GPU: **Runtime** → **Change runtime type** → pilih **T4 GPU**.
|
||||||
|
|
||||||
|
### Langkah 3: Mount Google Drive di Colab
|
||||||
|
|
||||||
|
Di sel pertama notebook, jalankan:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from google.colab import drive
|
||||||
|
drive.mount('/content/drive')
|
||||||
|
```
|
||||||
|
|
||||||
|
Ikuti instruksi autentikasi yang muncul.
|
||||||
|
|
||||||
|
### Langkah 4: Ekstrak Dataset di Colab
|
||||||
|
|
||||||
|
```python
|
||||||
|
import zipfile
|
||||||
|
|
||||||
|
zip_path = '/content/drive/MyDrive/corn_disease/dataset.zip'
|
||||||
|
extract_path = '/content/dataset'
|
||||||
|
|
||||||
|
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
|
||||||
|
zip_ref.extractall(extract_path)
|
||||||
|
|
||||||
|
print("Dataset berhasil diekstrak!")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Langkah 5: Jalankan Training
|
||||||
|
|
||||||
|
Jalankan seluruh sel di `notebook.ipynb` secara berurutan. Notebook akan melatih model `EfficientNetV2B0` dan menyimpan model terbaik secara otomatis ke:
|
||||||
|
|
||||||
|
```
|
||||||
|
/content/drive/MyDrive/corn_disease/best_model.keras
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Tips:** Aktifkan **"Run all"** dan pantau grafik loss/accuracy. Proses training bisa memakan waktu 30–90 menit tergantung ukuran dataset dan kuota GPU Colab.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Tahap 3 — Download Model dari Colab
|
||||||
|
|
||||||
|
Setelah training selesai, unduh `best_model.keras` dari Google Drive ke komputer lokal.
|
||||||
|
|
||||||
|
### Opsi A: Unduh via Google Drive (Direkomendasikan)
|
||||||
|
|
||||||
|
1. Buka Google Drive → navigasi ke folder `corn_disease/`.
|
||||||
|
2. Klik kanan `best_model.keras` → **Download**.
|
||||||
|
3. Letakkan file yang diunduh ke folder `best_model/` di proyek lokal:
|
||||||
|
```
|
||||||
|
best_model/best_model.keras
|
||||||
|
```
|
||||||
|
|
||||||
|
### Opsi B: Unduh Langsung dari Colab
|
||||||
|
|
||||||
|
Tambahkan sel berikut di akhir notebook, lalu jalankan:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from google.colab import files
|
||||||
|
files.download('/content/drive/MyDrive/corn_disease/best_model.keras')
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Tahap 4 — Ekspor Model untuk Produksi
|
||||||
|
|
||||||
|
Setelah `best_model/best_model.keras` tersedia di lokal, jalankan skrip ekspor.
|
||||||
|
|
||||||
|
### Persiapan Struktur Folder
|
||||||
|
|
||||||
|
Pastikan struktur direktori sudah benar:
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── best_model/
|
||||||
|
│ └── best_model.keras ← File hasil unduhan dari Colab
|
||||||
|
└── save_model.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### Langkah 1: Ekspor ke SavedModel & TFLite
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python save_model.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Skrip ini akan:
|
||||||
|
1. Memuat model dari `best_model/best_model.keras`.
|
||||||
|
2. Membangun ulang arsitektur **tanpa layer augmentasi** (wajib untuk kompatibilitas TFJS).
|
||||||
|
3. Menyalin semua bobot terlatih ke arsitektur baru.
|
||||||
|
4. Mengekspor ke `model/saved_model/` (format SavedModel).
|
||||||
|
5. Mengonversi ke `model/model.tflite` (format TFLite untuk Android/iOS).
|
||||||
|
|
||||||
|
### Langkah 2: Konversi ke TensorFlow.js (untuk Web/Browser)
|
||||||
|
|
||||||
|
Konversi TFJS **harus dilakukan via CLI**, bukan dari dalam Python, untuk menghindari konflik library Protobuf.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Langkah 2a: Atur variabel lingkungan
|
||||||
|
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
|
||||||
|
|
||||||
|
# Langkah 2b: Konversi
|
||||||
|
tensorflowjs_converter \
|
||||||
|
--input_format=tf_saved_model \
|
||||||
|
--output_format=tfjs_graph_model \
|
||||||
|
--signature_name=serving_default \
|
||||||
|
--saved_model_tags=serve \
|
||||||
|
model/saved_model \
|
||||||
|
model/tfjs_model
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Output Akhir
|
||||||
|
|
||||||
|
Setelah seluruh pipeline selesai dijalankan, berikut file output yang tersedia:
|
||||||
|
|
||||||
|
| File / Direktori | Format | Kegunaan |
|
||||||
|
|---|---|---|
|
||||||
|
| `dataset/` | Folder gambar terstruktur | Dataset akhir hasil preprocessing |
|
||||||
|
| `dataset.zip` | ZIP | Dataset untuk diupload ke Google Drive / Colab |
|
||||||
|
| `best_model/best_model.keras` | Keras | Model terlatih lengkap (dengan optimizer) |
|
||||||
|
| `model/saved_model/` | SavedModel (PB) | Inferensi server-side & jembatan konversi TFJS |
|
||||||
|
| `model/model.tflite` | TFLite | Inferensi di perangkat **Android / iOS** |
|
||||||
|
| `model/tfjs_model/` | TensorFlow.js | Inferensi di **browser / Node.js** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Troubleshooting
|
||||||
|
|
||||||
|
### `FileNotFoundError: dataset_1.zip tidak ditemukan`
|
||||||
|
**Solusi:** Pastikan ketiga file ZIP sudah diunduh dan diletakkan di direktori yang sama dengan `preprocessing.py`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `FileNotFoundError: Model file not found at best_model/best_model.keras`
|
||||||
|
**Solusi:** Unduh `best_model.keras` dari Google Drive/Colab dan letakkan di folder `best_model/`. Pastikan nama filenya tepat.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `ValueError: incompatible with expected resource` (saat konversi TFJS)
|
||||||
|
**Penyebab:** Layer augmentasi data Keras (`random_flip`, dll.) bersifat *stateful* dinamis dan tidak bisa dibekukan oleh konverter TFJS.
|
||||||
|
|
||||||
|
**Solusi:** Sudah ditangani otomatis oleh fungsi `build_clean_model()` dalam `save_model.py`, yang membangun ulang arsitektur tanpa layer augmentasi sebelum proses ekspor.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `VersionError: Detected incompatible Protobuf Gencode/Runtime versions`
|
||||||
|
**Penyebab:** Konflik antara library C++ Protobuf yang dibawa oleh `tensorflow_decision_forests` dengan *runtime* Python di lingkungan virtual lokal.
|
||||||
|
|
||||||
|
**Solusi:** Pastikan konversi TFJS dijalankan via **CLI langsung** (bukan `subprocess` dari Python) dengan variabel lingkungan berikut:
|
||||||
|
```bash
|
||||||
|
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Sesi Colab terputus saat training
|
||||||
|
**Solusi:** Gunakan callback `ModelCheckpoint` di notebook untuk menyimpan checkpoint secara berkala ke Google Drive, sehingga training bisa dilanjutkan dari checkpoint terakhir tanpa mengulang dari awal.
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4e8e3a4f86ae1594aeb221c40db3bc80e87c05b05f386bbb628ff1da16397424
|
||||||
|
size 112903200
|
||||||
Binary file not shown.
@@ -0,0 +1 @@
|
|||||||
|
©©ءّ¸ث‡ق¬§¢¥ؤج™إأà’–سèز÷ئخ ’ڑھ¸¥ؤي—·(،وظ—ئƒ„÷ٌ2:10851638082828504866
|
||||||
Binary file not shown.
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4e3831052cde64562f0a4e51d5f224c2c819dd625a1d89b35ef58d74b82f1858
|
||||||
|
size 53214594
|
||||||
Binary file not shown.
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:613179718d653f027cf5e70a02f223b423a98abaffe7120ef626793aec23177f
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:20325958a36f2afdf149083a4a2079aee4725bb3d0c01f96694f617bb7c29a27
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:853bb38c854ff011b06ed98d7037b25501e02a0b718e65159acbbc62e97874ae
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:bf9a4b64fb22a0e83a2862f119a61842a72ec8ba15981ece3291b049dd4db11a
|
||||||
|
size 2432372
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:84e7ddbfcf1fd5d55bf1198cc65fb024ff9110e544195779b42281245246b16d
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8d98aa415626d122e2a324d5ffd3a536bcd03f42957114bc23b3d4fc706c7eb8
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:30cb3863522eb0e1f1688249f58629ed1e1f03e82d723fde7a13763ade2d60da
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2f8fffbfc14379d113100374729ce0236306b6f1aa8b32808e07020adfb3f0a0
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:516a4af1cae907747b7dc91169579a29bd477a99ad4e638fbb205a9b0acaae9d
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:76757c0eb38bc7c38c12d8a8ab8ab68b83d17f377cceca985905e5c48dece798
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f5f1a2bd449370e6ffd13143d78e6d1b2803df4cbae968fee5ab4244349a64b5
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8c79a0792c83c7bcbf8cb2768d3e6d12a928f95506c64c6818698cea0e3f2fec
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6c5a36d2a203ece90b2978f5295f42c7a0f9ff72a2edcdbcaa306e67c29b3d72
|
||||||
|
size 4194304
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"format": "graph-model", "generatedBy": "2.19.0", "convertedBy": "TensorFlow.js Converter v4.22.0", "signature": {"inputs": {"inputs": {"name": "inputs:0", "dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "-1"}, {"size": "224"}, {"size": "224"}, {"size": "3"}]}}}, "outputs": {"output_0": {"name": "Identity:0", "dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "-1"}, {"size": "4"}]}}}}, "modelTopology": {"node": [{"name": "StatefulPartitionedCall/functional_6_1/batch_normalization_2_1/Line truncated
|
||||||
@@ -0,0 +1,943 @@
|
|||||||
|
{
|
||||||
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "fqNIt28_pKnS"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"1. Persiapan Lingkungan"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 1,
|
||||||
|
"metadata": {
|
||||||
|
"id": "n6mBf6uO1VS6"
|
||||||
|
},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"!pip install -q split-folders"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 2,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/"
|
||||||
|
},
|
||||||
|
"id": "dNC9QMXh1mWq",
|
||||||
|
"outputId": "132f1b20-f2b3-4acc-a0f5-7464e74ca613"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"TensorFlow Version: 2.20.0\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"import os\n",
|
||||||
|
"import shutil\n",
|
||||||
|
"import numpy as np\n",
|
||||||
|
"import matplotlib.pyplot as plt\n",
|
||||||
|
"import seaborn as sns\n",
|
||||||
|
"import splitfolders\n",
|
||||||
|
"import tarfile\n",
|
||||||
|
"import zipfile\n",
|
||||||
|
"from PIL import Image\n",
|
||||||
|
"\n",
|
||||||
|
"from pathlib import Path\n",
|
||||||
|
"from PIL import Image\n",
|
||||||
|
"from google.colab import drive\n",
|
||||||
|
"\n",
|
||||||
|
"\n",
|
||||||
|
"import tensorflow as tf\n",
|
||||||
|
"from tensorflow.keras import layers, models, callbacks\n",
|
||||||
|
"from tensorflow.keras.applications import EfficientNetV2B0\n",
|
||||||
|
"from sklearn.metrics import classification_report, confusion_matrix\n",
|
||||||
|
"\n",
|
||||||
|
"print(f\"TensorFlow Version: {tf.__version__}\")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "A6j2Vv8ypnrd"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"2. Download dan Ekstraksi Dataset"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 3,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/"
|
||||||
|
},
|
||||||
|
"id": "bsGJEEFs7Flg",
|
||||||
|
"outputId": "30228ce8-1b64-47f5-93f7-30632b2af1c5"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Mounted at /content/drive\n",
|
||||||
|
"Moving ZIP file to /content...\n",
|
||||||
|
"File successfully moved.\n",
|
||||||
|
"Extracting dataset...\n",
|
||||||
|
"Extraction completed!\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"drive.mount('/content/drive')\n",
|
||||||
|
"\n",
|
||||||
|
"archive_path = '/content/drive/MyDrive/jagung/dataset_jagung.zip'\n",
|
||||||
|
"destination_path = '/content/dataset_jagung.zip'\n",
|
||||||
|
"extract_path = '/content/dataset'\n",
|
||||||
|
"\n",
|
||||||
|
"if os.path.exists(archive_path):\n",
|
||||||
|
" print(\"Moving ZIP file to /content...\")\n",
|
||||||
|
" shutil.copy(archive_path, destination_path)\n",
|
||||||
|
" print(\"File successfully moved.\")\n",
|
||||||
|
"else:\n",
|
||||||
|
" print(f\"Error: File not found at {archive_path}\")\n",
|
||||||
|
"\n",
|
||||||
|
"if os.path.exists(destination_path):\n",
|
||||||
|
" if not os.path.exists(extract_path) or len(os.listdir(extract_path)) == 0:\n",
|
||||||
|
" os.makedirs(extract_path, exist_ok=True)\n",
|
||||||
|
" print(\"Extracting dataset...\")\n",
|
||||||
|
" try:\n",
|
||||||
|
" with zipfile.ZipFile(destination_path, 'r') as zip_ref:\n",
|
||||||
|
" zip_ref.extractall(path=extract_path)\n",
|
||||||
|
" print(\"Extraction completed!\")\n",
|
||||||
|
" except Exception as e:\n",
|
||||||
|
" print(f\"Extraction failed: {e}\")\n",
|
||||||
|
" else:\n",
|
||||||
|
" print(\"Dataset is already extracted and ready for use.\")\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "148LNQ7Ips01"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"4. Data Cleaning dan Validasi Gambat (RGB)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 4,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/"
|
||||||
|
},
|
||||||
|
"id": "bIPb9DIX1spd",
|
||||||
|
"outputId": "422957b6-c1e7-4f65-8834-14673b50a70e"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Starting data cleanup process...\n",
|
||||||
|
"Cleanup completed. Total of 0 problematic files removed.\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"dataset_path = \"/content/dataset/dataset_jagung_v1\"\n",
|
||||||
|
"\n",
|
||||||
|
"def clean_image_data(directory):\n",
|
||||||
|
" removed_count = 0\n",
|
||||||
|
" for root, dirs, files in os.walk(directory):\n",
|
||||||
|
" for file in files:\n",
|
||||||
|
" file_path = os.path.join(root, file)\n",
|
||||||
|
" try:\n",
|
||||||
|
" img = Image.open(file_path)\n",
|
||||||
|
" img.verify()\n",
|
||||||
|
"\n",
|
||||||
|
" img = Image.open(file_path)\n",
|
||||||
|
" if img.mode != 'RGB':\n",
|
||||||
|
" img = img.convert('RGB')\n",
|
||||||
|
" img.save(file_path)\n",
|
||||||
|
" except Exception as e:\n",
|
||||||
|
" print(f\"Removing corrupt/invalid file: {file_path}\")\n",
|
||||||
|
" os.remove(file_path)\n",
|
||||||
|
" removed_count += 1\n",
|
||||||
|
" return removed_count\n",
|
||||||
|
"\n",
|
||||||
|
"print(\"Starting data cleanup process...\")\n",
|
||||||
|
"removed = clean_image_data(dataset_path)\n",
|
||||||
|
"print(f\"Cleanup completed. Total of {removed} problematic files removed.\")\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "YSQX48hTp3vW"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"5. Data Spliting (70/15/15/)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 5,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/"
|
||||||
|
},
|
||||||
|
"id": "8Zg2hJw4a3RJ",
|
||||||
|
"outputId": "3daeef44-6e99-48d4-da8e-53f7f431a550"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Splitting dataset into Train, Validation, and Test...\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stderr",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Copying files: 7106 files [01:11, 99.29 files/s] "
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Dataset splitting completed.\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stderr",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"output_dir = \"/content/dataset_split\"\n",
|
||||||
|
"\n",
|
||||||
|
"if os.path.exists(output_dir):\n",
|
||||||
|
" shutil.rmtree(output_dir)\n",
|
||||||
|
"\n",
|
||||||
|
"print(\"Splitting dataset into Train, Validation, and Test...\")\n",
|
||||||
|
"splitfolders.ratio(dataset_path, output=output_dir,\n",
|
||||||
|
" seed=42, ratio=(0.7, 0.15, 0.15),\n",
|
||||||
|
" group_prefix=None, move=False)\n",
|
||||||
|
"print(\"Dataset splitting completed.\")\n",
|
||||||
|
"\n",
|
||||||
|
"train_dir = os.path.join(output_dir, 'train')\n",
|
||||||
|
"val_dir = os.path.join(output_dir, 'val')\n",
|
||||||
|
"test_dir = os.path.join(output_dir, 'test')"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "F0q8ELD5qPAY"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"6. Data Loader dan Augmentasi"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 6,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/"
|
||||||
|
},
|
||||||
|
"id": "rNU4YGv61wFX",
|
||||||
|
"outputId": "beda9f63-d52a-46dd-a28c-6d1ac17e7169"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Found 4973 files belonging to 4 classes.\n",
|
||||||
|
"Found 1063 files belonging to 4 classes.\n",
|
||||||
|
"Found 1070 files belonging to 4 classes.\n",
|
||||||
|
"Number of classes: 4\n",
|
||||||
|
"Class names: ['Bercak Daun', 'Daun Sehat', 'Karat Daun', 'hawar daun']\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"BATCH_SIZE = 32\n",
|
||||||
|
"IMG_SIZE = (224, 224)\n",
|
||||||
|
"\n",
|
||||||
|
"train_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||||||
|
" train_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||||||
|
"\n",
|
||||||
|
"val_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||||||
|
" val_dir, shuffle=True, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||||||
|
"\n",
|
||||||
|
"test_ds = tf.keras.utils.image_dataset_from_directory(\n",
|
||||||
|
" test_dir, shuffle=False, batch_size=BATCH_SIZE, image_size=IMG_SIZE)\n",
|
||||||
|
"\n",
|
||||||
|
"class_names = train_ds.class_names\n",
|
||||||
|
"NUM_CLASSES = len(class_names)\n",
|
||||||
|
"print(f\"Number of classes: {NUM_CLASSES}\")\n",
|
||||||
|
"print(f\"Class names: {class_names}\")\n",
|
||||||
|
"\n",
|
||||||
|
"def to_one_hot(image, label):\n",
|
||||||
|
" return image, tf.one_hot(label, NUM_CLASSES)\n",
|
||||||
|
"\n",
|
||||||
|
"train_ds = train_ds.map(to_one_hot, num_parallel_calls=tf.data.AUTOTUNE)\n",
|
||||||
|
"val_ds = val_ds.map(to_one_hot, num_parallel_calls=tf.data.AUTOTUNE)\n",
|
||||||
|
"test_ds = test_ds.map(to_one_hot, num_parallel_calls=tf.data.AUTOTUNE)\n",
|
||||||
|
"\n",
|
||||||
|
"AUTOTUNE = tf.data.AUTOTUNE\n",
|
||||||
|
"train_ds = train_ds.prefetch(buffer_size=AUTOTUNE)\n",
|
||||||
|
"val_ds = val_ds.prefetch(buffer_size=AUTOTUNE)\n",
|
||||||
|
"test_ds = test_ds.prefetch(buffer_size=AUTOTUNE)\n",
|
||||||
|
"\n",
|
||||||
|
"data_augmentation = tf.keras.Sequential([\n",
|
||||||
|
" layers.RandomFlip(\"horizontal_and_vertical\"),\n",
|
||||||
|
" layers.RandomRotation(0.2),\n",
|
||||||
|
" layers.RandomZoom(0.2),\n",
|
||||||
|
" layers.RandomTranslation(0.1, 0.1),\n",
|
||||||
|
" layers.RandomContrast(0.2),\n",
|
||||||
|
" layers.RandomBrightness(0.2)\n",
|
||||||
|
"], name=\"data_augmentation\")\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "BwSHXm-WqXeE"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"7. Visualisasi Sample dan Hasil Augmentasi"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 7,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/",
|
||||||
|
"height": 610
|
||||||
|
},
|
||||||
|
"id": "tc8SYmZH3lGS",
|
||||||
|
"outputId": "44f07316-12d9-41a7-8069-5b108a335a18"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABJEAAAJRCAYAAAD1diY8AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs/XncJVdV74+/91DTGZ+55/SQhIQMBJAwJ+RKICiIoheviMogiIAoeuGlX1/K5O/q66oXFbwXwQE1CFdBRECZrkRmMAgkkBBCku4kPfczn7GGvffvj111nqfTTToJGFqtD68m3efUqdq1q2rVWp/9WWsJ55yjRo0aNWrUqFGjRo0aNWrUqFGjRo17gfxuD6BGjRo1atSoUaNGjRo1atSoUaPG2Y+aRKpRo0aNGjVq1KhRo0aNGjVq1KhxRtQkUo0aNWrUqFGjRo0aNWrUqFGjRo0zoiaRatSoUaNGjLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 1200x600 with 8 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"plt.figure(figsize=(12, 6))\n",
|
||||||
|
"for images, labels in train_ds.take(1):\n",
|
||||||
|
" for i in range(4):\n",
|
||||||
|
" ax = plt.subplot(2, 4, i + 1)\n",
|
||||||
|
" plt.imshow(images[i].numpy().astype(\"uint8\"))\n",
|
||||||
|
" orig_class_idx = np.argmax(labels[i].numpy())\n",
|
||||||
|
"\n",
|
||||||
|
" plt.title(f\"Ori: {class_names[orig_class_idx]}\")\n",
|
||||||
|
" plt.axis(\"off\")\n",
|
||||||
|
"\n",
|
||||||
|
" augmented_images = data_augmentation(images, training=True)\n",
|
||||||
|
" for i in range(4):\n",
|
||||||
|
" ax = plt.subplot(2, 4, i + 5)\n",
|
||||||
|
" plt.imshow(augmented_images[i].numpy().astype(\"uint8\"))\n",
|
||||||
|
" orig_class_idx = np.argmax(labels[i].numpy())\n",
|
||||||
|
"\n",
|
||||||
|
" plt.title(f\"Aug: {class_names[orig_class_idx]}\")\n",
|
||||||
|
" plt.axis(\"off\")\n",
|
||||||
|
"plt.tight_layout()\n",
|
||||||
|
"plt.show()"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "vQefwIFlqhU0"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"8. Arsitektur Model CNN (EfficientNetv2B0)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 8,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/",
|
||||||
|
"height": 737
|
||||||
|
},
|
||||||
|
"id": "iswHWuZb3pZj",
|
||||||
|
"outputId": "a83c5bc3-e512-4e01-c359-0766daaa608c"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"No checkpoint found.\n",
|
||||||
|
"Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/efficientnet_v2/efficientnetv2-b0_notop.h5\n",
|
||||||
|
"\u001b[1m24274472/24274472\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 0us/step\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"functional_1\"</span>\n",
|
||||||
|
"</pre>\n"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
"\u001b[1mModel: \"functional_1\"\u001b[0m\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
|
||||||
|
"┃<span style=\"font-weight: bold\"> Layer (type) </span>┃<span style=\"font-weight: bold\"> Output Shape </span>┃<span style=\"font-weight: bold\"> Param # </span>┃\n",
|
||||||
|
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
|
||||||
|
"│ input_layer_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">224</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">224</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>) │ <span style=\"color: #00af00; text-decoration-color: #0Line truncated
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ data_augmentation (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Sequential</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">224</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">224</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>) │ <span style=\"color: #00af00; text-decoration-color: #0Line truncated
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ efficientnetv2-b0 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Functional</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">7</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">7</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1280</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">Line truncated
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ conv2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">7</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">7</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">Line truncated
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ batch_normalization │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">7</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">7</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">2,048</span> │\n",
|
||||||
|
"│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">BatchNormalization</span>) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ max_pooling2d (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">MaxPooling2D</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>) │ <span style=\"color: #00af00; text-decoration-color: #0Line truncated
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dropout (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dropout</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">512</span>) │ <span style=\"color: #00af00; text-decoration-color: #0Line truncated
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ conv2d_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Conv2D</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">256</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">Line truncated
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ batch_normalization_1 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">3</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">256</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">1,024</span> │\n",
|
||||||
|
"│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">BatchNormalization</span>) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ global_average_pooling2d │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">256</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
|
||||||
|
"│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">GlobalAveragePooling2D</span>) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dropout_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dropout</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">256</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dense (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1024</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">263,168</span> │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ batch_normalization_2 │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1024</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">4,096</span> │\n",
|
||||||
|
"│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">BatchNormalization</span>) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dropout_2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dropout</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1024</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dense_1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>) │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">4</span>) │ <span style=\"color: #00af00; text-decoration-color: #00af00\">4,100</span> │\n",
|
||||||
|
"└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
|
||||||
|
"</pre>\n"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
"┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
|
||||||
|
"┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
|
||||||
|
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
|
||||||
|
"│ input_layer_2 (\u001b[38;5;33mInputLayer\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m224\u001b[0m, \u001b[38;5;34m224\u001b[0m, \u001b[38;5;34m3\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ data_augmentation (\u001b[38;5;33mSequential\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m224\u001b[0m, \u001b[38;5;34m224\u001b[0m, \u001b[38;5;34m3\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ efficientnetv2-b0 (\u001b[38;5;33mFunctional\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m7\u001b[0m, \u001b[38;5;34m7\u001b[0m, \u001b[38;5;34m1280\u001b[0m) │ \u001b[38;5;34m5,919,312\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ conv2d (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m7\u001b[0m, \u001b[38;5;34m7\u001b[0m, \u001b[38;5;34m512\u001b[0m) │ \u001b[38;5;34m5,898,752\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ batch_normalization │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m7\u001b[0m, \u001b[38;5;34m7\u001b[0m, \u001b[38;5;34m512\u001b[0m) │ \u001b[38;5;34m2,048\u001b[0m │\n",
|
||||||
|
"│ (\u001b[38;5;33mBatchNormalization\u001b[0m) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ max_pooling2d (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m512\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dropout (\u001b[38;5;33mDropout\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m512\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ conv2d_1 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m256\u001b[0m) │ \u001b[38;5;34m1,179,904\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ batch_normalization_1 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m3\u001b[0m, \u001b[38;5;34m256\u001b[0m) │ \u001b[38;5;34m1,024\u001b[0m │\n",
|
||||||
|
"│ (\u001b[38;5;33mBatchNormalization\u001b[0m) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ global_average_pooling2d │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m256\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
|
||||||
|
"│ (\u001b[38;5;33mGlobalAveragePooling2D\u001b[0m) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dropout_1 (\u001b[38;5;33mDropout\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m256\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dense (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1024\u001b[0m) │ \u001b[38;5;34m263,168\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ batch_normalization_2 │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1024\u001b[0m) │ \u001b[38;5;34m4,096\u001b[0m │\n",
|
||||||
|
"│ (\u001b[38;5;33mBatchNormalization\u001b[0m) │ │ │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dropout_2 (\u001b[38;5;33mDropout\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1024\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n",
|
||||||
|
"├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
|
||||||
|
"│ dense_1 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m4\u001b[0m) │ \u001b[38;5;34m4,100\u001b[0m │\n",
|
||||||
|
"└─────────────────────────────────┴────────────────────────┴───────────────┘\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">13,272,404</span> (50.63 MB)\n",
|
||||||
|
"</pre>\n"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
"\u001b[1m Total params: \u001b[0m\u001b[38;5;34m13,272,404\u001b[0m (50.63 MB)\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">7,349,508</span> (28.04 MB)\n",
|
||||||
|
"</pre>\n"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
"\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m7,349,508\u001b[0m (28.04 MB)\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/html": [
|
||||||
|
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">5,922,896</span> (22.59 MB)\n",
|
||||||
|
"</pre>\n"
|
||||||
|
],
|
||||||
|
"text/plain": [
|
||||||
|
"\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m5,922,896\u001b[0m (22.59 MB)\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"def build_model(num_classes):\n",
|
||||||
|
" base_model = EfficientNetV2B0(\n",
|
||||||
|
" input_shape=IMG_SIZE + (3,),\n",
|
||||||
|
" include_top=False,\n",
|
||||||
|
" weights='imagenet',\n",
|
||||||
|
" )\n",
|
||||||
|
" base_model.trainable = False\n",
|
||||||
|
"\n",
|
||||||
|
" inputs = tf.keras.Input(shape=IMG_SIZE + (3,))\n",
|
||||||
|
"\n",
|
||||||
|
" x = data_augmentation(inputs)\n",
|
||||||
|
" x = base_model(x, training=False)\n",
|
||||||
|
"\n",
|
||||||
|
" x = layers.Conv2D(512, (3, 3), padding='same', activation='swish')(x)\n",
|
||||||
|
" x = layers.BatchNormalization()(x)\n",
|
||||||
|
" x = layers.MaxPooling2D((2, 2))(x)\n",
|
||||||
|
" x = layers.Dropout(0.2)(x)\n",
|
||||||
|
"\n",
|
||||||
|
" x = layers.Conv2D(256, (3, 3), padding='same', activation='swish')(x)\n",
|
||||||
|
" x = layers.BatchNormalization()(x)\n",
|
||||||
|
"\n",
|
||||||
|
" x = layers.GlobalAveragePooling2D()(x)\n",
|
||||||
|
" x = layers.Dropout(0.3)(x)\n",
|
||||||
|
"\n",
|
||||||
|
" x = layers.Dense(1024, activation='swish')(x)\n",
|
||||||
|
" x = layers.BatchNormalization()(x)\n",
|
||||||
|
" x = layers.Dropout(0.4)(x)\n",
|
||||||
|
"\n",
|
||||||
|
" outputs = layers.Dense(num_classes, activation='sigmoid', dtype='float32')(x)\n",
|
||||||
|
"\n",
|
||||||
|
" model = models.Model(inputs, outputs)\n",
|
||||||
|
" return model, base_model\n",
|
||||||
|
"\n",
|
||||||
|
"checkpoint_path = '/content/best_model/best_model.keras'\n",
|
||||||
|
"\n",
|
||||||
|
"if os.path.exists(checkpoint_path):\n",
|
||||||
|
" print(f\"Loading weights from model checkpoint: {checkpoint_path}...\")\n",
|
||||||
|
" try:\n",
|
||||||
|
" model = models.load_model(checkpoint_path)\n",
|
||||||
|
" except Exception as e:\n",
|
||||||
|
" print(f\"Failed to load checkpoint model: {e}.\")\n",
|
||||||
|
" model, base_model = build_model(NUM_CLASSES)\n",
|
||||||
|
"else:\n",
|
||||||
|
" print(\"No checkpoint found.\")\n",
|
||||||
|
" model, base_model = build_model(NUM_CLASSES)\n",
|
||||||
|
"\n",
|
||||||
|
"model.summary()\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "yik1l5ngqt_1"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"9. Implementasi Callbacks"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 9,
|
||||||
|
"metadata": {
|
||||||
|
"id": "s4Trlvno3wny"
|
||||||
|
},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"checkpoint_cb = callbacks.ModelCheckpoint(\n",
|
||||||
|
" \"best_model/best_model.keras\",\n",
|
||||||
|
" save_best_only=True,\n",
|
||||||
|
" monitor=\"val_loss\",\n",
|
||||||
|
" mode=\"min\"\n",
|
||||||
|
")\n",
|
||||||
|
"\n",
|
||||||
|
"early_stopping_cb = callbacks.EarlyStopping(\n",
|
||||||
|
" monitor=\"val_loss\",\n",
|
||||||
|
" patience=7,\n",
|
||||||
|
" restore_best_weights=True,\n",
|
||||||
|
" mode=\"min\"\n",
|
||||||
|
")\n",
|
||||||
|
"\n",
|
||||||
|
"reduce_lr_cb = callbacks.ReduceLROnPlateau(\n",
|
||||||
|
" monitor='val_loss',\n",
|
||||||
|
" factor=0.2,\n",
|
||||||
|
" patience=3,\n",
|
||||||
|
" min_lr=1e-6,\n",
|
||||||
|
" verbose=1,\n",
|
||||||
|
" mode=\"min\"\n",
|
||||||
|
")\n",
|
||||||
|
"\n",
|
||||||
|
"callbacks_list = [checkpoint_cb, early_stopping_cb, reduce_lr_cb]\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "UKyFNmVvqyHx"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"10. Training Model"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 10,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/"
|
||||||
|
},
|
||||||
|
"id": "cq5rllO63zXA",
|
||||||
|
"outputId": "0d3c03c8-d354-4b8b-befb-ecbb68b18735"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Training...\n",
|
||||||
|
"Epoch 1/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m207s\u001b[0m 1s/step - accuracy: 0.9153 - loss: 0.2220 - val_accuracy: 0.9459 - val_loss: 0.1425 - learning_rate: 0.0010\n",
|
||||||
|
"Epoch 2/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m186s\u001b[0m 1s/step - accuracy: 0.9512 - loss: 0.1323 - val_accuracy: 0.9781 - val_loss: 0.0640 - learning_rate: 0.0010\n",
|
||||||
|
"Epoch 3/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m191s\u001b[0m 1s/step - accuracy: 0.9649 - loss: 0.0948 - val_accuracy: 0.9730 - val_loss: 0.0645 - learning_rate: 0.0010\n",
|
||||||
|
"Epoch 4/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m180s\u001b[0m 1s/step - accuracy: 0.9707 - loss: 0.0796 - val_accuracy: 0.9748 - val_loss: 0.0763 - learning_rate: 0.0010\n",
|
||||||
|
"Epoch 5/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 958ms/step - accuracy: 0.9725 - loss: 0.0770\n",
|
||||||
|
"Epoch 5: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m191s\u001b[0m 1s/step - accuracy: 0.9733 - loss: 0.0757 - val_accuracy: 0.9661 - val_loss: 0.0838 - learning_rate: 0.0010\n",
|
||||||
|
"Epoch 6/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m185s\u001b[0m 1s/step - accuracy: 0.9778 - loss: 0.0564 - val_accuracy: 0.9824 - val_loss: 0.0445 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 7/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m179s\u001b[0m 1s/step - accuracy: 0.9786 - loss: 0.0557 - val_accuracy: 0.9857 - val_loss: 0.0419 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 8/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m167s\u001b[0m 1s/step - accuracy: 0.9804 - loss: 0.0534 - val_accuracy: 0.9880 - val_loss: 0.0354 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 9/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m171s\u001b[0m 1s/step - accuracy: 0.9807 - loss: 0.0495 - val_accuracy: 0.9882 - val_loss: 0.0317 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 10/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m179s\u001b[0m 1s/step - accuracy: 0.9842 - loss: 0.0427 - val_accuracy: 0.9894 - val_loss: 0.0319 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 11/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m173s\u001b[0m 1s/step - accuracy: 0.9833 - loss: 0.0452 - val_accuracy: 0.9892 - val_loss: 0.0293 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 12/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m214s\u001b[0m 1s/step - accuracy: 0.9839 - loss: 0.0413 - val_accuracy: 0.9889 - val_loss: 0.0320 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 13/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m181s\u001b[0m 1s/step - accuracy: 0.9864 - loss: 0.0393 - val_accuracy: 0.9880 - val_loss: 0.0329 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 14/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 1s/step - accuracy: 0.9878 - loss: 0.0355\n",
|
||||||
|
"Epoch 14: ReduceLROnPlateau reducing learning rate to 4.0000001899898055e-05.\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m246s\u001b[0m 1s/step - accuracy: 0.9877 - loss: 0.0358 - val_accuracy: 0.9885 - val_loss: 0.0340 - learning_rate: 2.0000e-04\n",
|
||||||
|
"Epoch 15/15\n",
|
||||||
|
"\u001b[1m156/156\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m176s\u001b[0m 1s/step - accuracy: 0.9866 - loss: 0.0358 - val_accuracy: 0.9897 - val_loss: 0.0326 - learning_rate: 4.0000e-05\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"model.compile(\n",
|
||||||
|
" optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),\n",
|
||||||
|
" loss=tf.keras.losses.BinaryCrossentropy(),\n",
|
||||||
|
" metrics=[tf.keras.metrics.BinaryAccuracy(name='accuracy')]\n",
|
||||||
|
")\n",
|
||||||
|
"\n",
|
||||||
|
"print(\"Training...\")\n",
|
||||||
|
"EPOCHS_PHASE_1 = 15\n",
|
||||||
|
"\n",
|
||||||
|
"history_1 = model.fit(\n",
|
||||||
|
" train_ds,\n",
|
||||||
|
" validation_data=val_ds,\n",
|
||||||
|
" epochs=EPOCHS_PHASE_1,\n",
|
||||||
|
" callbacks=callbacks_list\n",
|
||||||
|
")"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "Fq0qppJ_rGDE"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"11. Plot History"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 11,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/",
|
||||||
|
"height": 545
|
||||||
|
},
|
||||||
|
"id": "wOW7f13UD4wy",
|
||||||
|
"outputId": "eff4a3d7-9ddc-4992-da43-0ab09631e47d"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABRQAAAIQCAYAAADw0OJwAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs3Xd4VGXax/HvpPeEFFIgEAglhBakKUpxF6Uoq4iAyEpRcEXBVdaGq4jsq1iQRdEF1wIuiNgQOwgIKohUKUqRHtJp6X3mvH9MMjAkQAJJJgm/z3XNNTNnnjnnPpMoJ/c8z32bDMMwEBEREREREREREakAJ0cHICIiIiIiIiIiInWHEooiIiIiIiIiIiJSYUooioiIiIiIiIiISIUpoSgiIiIiIiIiIiIVpoSiiIiIiIiIiIiIVJgSiiIiIiIiIiIiIlJhSiiKiIiIiIiIiIhIhSmhKCIiIiIiIiIiILine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 1600x600 with 2 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"def plot_history(history_1, initial_epochs):\n",
|
||||||
|
" acc = history_1.history['accuracy']\n",
|
||||||
|
" val_acc = history_1.history['val_accuracy']\n",
|
||||||
|
" loss = history_1.history['loss']\n",
|
||||||
|
" val_loss = history_1.history['val_loss']\n",
|
||||||
|
"\n",
|
||||||
|
" plt.figure(figsize=(16, 6))\n",
|
||||||
|
"\n",
|
||||||
|
" plt.subplot(1, 2, 1)\n",
|
||||||
|
" plt.plot(acc, label='Training Accuracy')\n",
|
||||||
|
" plt.plot(val_acc, label='Validation Accuracy')\n",
|
||||||
|
" plt.legend(loc='lower right')\n",
|
||||||
|
" plt.title('Training and Validation Accuracy')\n",
|
||||||
|
"\n",
|
||||||
|
" plt.subplot(1, 2, 2)\n",
|
||||||
|
" plt.plot(loss, label='Training Loss')\n",
|
||||||
|
" plt.plot(val_loss, label='Validation Loss')\n",
|
||||||
|
" plt.legend(loc='upper right')\n",
|
||||||
|
" plt.title('Training and Validation Loss')\n",
|
||||||
|
" plt.show()\n",
|
||||||
|
"\n",
|
||||||
|
"plot_history(history_1, EPOCHS_PHASE_1)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "TxXzQ8ZrrmiS"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"12. Test Time Augmentation"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 12,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/"
|
||||||
|
},
|
||||||
|
"id": "5vnDCrqLD5hT",
|
||||||
|
"outputId": "b4e04c95-c570-4817-cf1f-37321560c89c"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Evaluating model using TTA (Test-Time Augmentation) for Multi-Label...\n",
|
||||||
|
"TTA Step 1/5\n",
|
||||||
|
"TTA Step 2/5\n",
|
||||||
|
"TTA Step 3/5\n",
|
||||||
|
"TTA Step 4/5\n",
|
||||||
|
"TTA Step 5/5\n",
|
||||||
|
"\n",
|
||||||
|
"Subset Accuracy (Exact Match) on Test Set (with TTA): 97.20%\n",
|
||||||
|
"Binary Accuracy (Per-Class Average) on Test Set (with TTA): 98.86%\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"print(\"Evaluating model using TTA (Test-Time Augmentation) for Multi-Label...\")\n",
|
||||||
|
"\n",
|
||||||
|
"test_images = []\n",
|
||||||
|
"test_labels_true = []\n",
|
||||||
|
"\n",
|
||||||
|
"for images, labels in test_ds.unbatch():\n",
|
||||||
|
" test_images.append(images.numpy())\n",
|
||||||
|
" test_labels_true.append(labels.numpy())\n",
|
||||||
|
"\n",
|
||||||
|
"test_images = np.array(test_images)\n",
|
||||||
|
"test_labels_true = np.array(test_labels_true)\n",
|
||||||
|
"\n",
|
||||||
|
"TTA_STEPS = 5\n",
|
||||||
|
"tta_predictions = []\n",
|
||||||
|
"\n",
|
||||||
|
"for i in range(TTA_STEPS):\n",
|
||||||
|
" print(f\"TTA Step {i+1}/{TTA_STEPS}\")\n",
|
||||||
|
" aug_images = data_augmentation(test_images, training=True)\n",
|
||||||
|
" preds = model.predict(aug_images, batch_size=BATCH_SIZE, verbose=0)\n",
|
||||||
|
" tta_predictions.append(preds)\n",
|
||||||
|
"\n",
|
||||||
|
"mean_tta_preds = np.mean(tta_predictions, axis=0)\n",
|
||||||
|
"\n",
|
||||||
|
"THRESHOLD = 0.5\n",
|
||||||
|
"test_preds_binary = (mean_tta_preds > THRESHOLD).astype(int)\n",
|
||||||
|
"\n",
|
||||||
|
"correct_predictions = np.equal(test_preds_binary, test_labels_true.astype(int))\n",
|
||||||
|
"tta_accuracy = np.mean(correct_predictions)\n",
|
||||||
|
"\n",
|
||||||
|
"print(f\"\\nSubset Accuracy (Exact Match) on Test Set (with TTA): {np.mean(np.all(correct_predictions, axis=1)) * 100:.2f}%\")\n",
|
||||||
|
"print(f\"Binary Accuracy (Per-Class Average) on Test Set (with TTA): {tta_accuracy * 100:.2f}%\")\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"id": "y-ADQvPIr1Wo"
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"13. Metrik Evaluasi"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 13,
|
||||||
|
"metadata": {
|
||||||
|
"colab": {
|
||||||
|
"base_uri": "https://localhost:8080/",
|
||||||
|
"height": 1000
|
||||||
|
},
|
||||||
|
"id": "4ea9-S50EtQ6",
|
||||||
|
"outputId": "f1632cab-a233-4072-d0ec-70f141f83757"
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"\n",
|
||||||
|
"Classification Report (Multi-Label):\n",
|
||||||
|
"\n",
|
||||||
|
" precision recall f1-score support\n",
|
||||||
|
"\n",
|
||||||
|
" Bercak Daun 0.96 0.94 0.95 224\n",
|
||||||
|
" Daun Sehat 0.99 1.00 1.00 345\n",
|
||||||
|
" Karat Daun 0.97 0.98 0.98 259\n",
|
||||||
|
" hawar daun 0.98 0.97 0.98 242\n",
|
||||||
|
"\n",
|
||||||
|
" micro avg 0.98 0.98 0.98 1070\n",
|
||||||
|
" macro avg 0.98 0.97 0.97 1070\n",
|
||||||
|
"weighted avg 0.98 0.98 0.98 1070\n",
|
||||||
|
" samples avg 0.97 0.98 0.98 1070\n",
|
||||||
|
"\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "stderr",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"/usr/local/lib/python3.12/dist-packages/sklearn/metrics/_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in samples with no predicted labels. Use `zero_division` parameter to control this behavior.\n",
|
||||||
|
" _warn_prf(average, modifier, f\"{metric.capitalize()} is\", len(result))\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABKUAAAPeCAYAAADd/6nHAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAqppJREFUeJzs3Xd0VOX69vFrEsikkQIk0kM10hTFQi+CRgFFEBERCUiXohQF9UiJCIqKFKXooRMUkSLNQxGQIgdRikjv/JQoHQwBAsnz/sGbOQyTwASSPRP8ftZiLeaZZ/a+92SY3Fyz59k2Y4wRAAAAAAAAYCEfTxcAAAAAAACAfx5CKQAAAAAAAFiOUAoAAAAAAACWI5QCAAAAAACA5QilAAAAAAAAYDlCKQAAAAAAAFiOUAoAAAAAAACWI5QCAAAAAACA5QilAAAAAAAAYDlCKcAL7N27V48//rhCQ0Nls9k0bLine truncated
|
||||||
|
"text/plain": [
|
||||||
|
"<Figure size 1200x1000 with 4 Axes>"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "display_data"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"from sklearn.metrics import multilabel_confusion_matrix\n",
|
||||||
|
"\n",
|
||||||
|
"print(\"\\nClassification Report (Multi-Label):\\n\")\n",
|
||||||
|
"print(classification_report(test_labels_true, test_preds_binary, target_names=class_names))\n",
|
||||||
|
"\n",
|
||||||
|
"mcm = multilabel_confusion_matrix(test_labels_true, test_preds_binary)\n",
|
||||||
|
"\n",
|
||||||
|
"fig, axes = plt.subplots(2, 2, figsize=(12, 10))\n",
|
||||||
|
"axes = axes.ravel()\n",
|
||||||
|
"\n",
|
||||||
|
"for i, class_name in enumerate(class_names):\n",
|
||||||
|
" sns.heatmap(mcm[i], annot=True, fmt='d', cmap='Blues', ax=axes[i], cbar=False,\n",
|
||||||
|
" xticklabels=['Negative', 'Positive'], yticklabels=['Negative', 'Positive'])\n",
|
||||||
|
" axes[i].set_title(f'Confusion Matrix: {class_name}')\n",
|
||||||
|
" axes[i].set_xlabel('Predicted')\n",
|
||||||
|
" axes[i].set_ylabel('True')\n",
|
||||||
|
"\n",
|
||||||
|
"plt.tight_layout()\n",
|
||||||
|
"plt.show()"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": {
|
||||||
|
"accelerator": "GPU",
|
||||||
|
"colab": {
|
||||||
|
"gpuType": "T4",
|
||||||
|
"provenance": []
|
||||||
|
},
|
||||||
|
"kernelspec": {
|
||||||
|
"display_name": "Python 3",
|
||||||
|
"name": "python3"
|
||||||
|
},
|
||||||
|
"language_info": {
|
||||||
|
"name": "python"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 0
|
||||||
|
}
|
||||||
@@ -0,0 +1,197 @@
|
|||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import zipfile
|
||||||
|
import json
|
||||||
|
|
||||||
|
# ==========================================
|
||||||
|
# KONFIGURASI DAN MAPPING
|
||||||
|
# ==========================================
|
||||||
|
DAFTAR_ZIP = ['dataset_1.zip', 'dataset_2.zip', 'dataset_3.zip']
|
||||||
|
TARGET_DIR = "dataset"
|
||||||
|
|
||||||
|
PEMETAAN_KATEGORI = {
|
||||||
|
"大斑病": "Hawar Daun",
|
||||||
|
"小斑病": "Hawar Daun",
|
||||||
|
"褐斑病": "Bercak Daun",
|
||||||
|
"弯孢霉叶斑病": "Bercak Daun",
|
||||||
|
"圆斑病": "Bercak Daun",
|
||||||
|
"灰斑病": "Bercak Daun",
|
||||||
|
"南方锈病": "Karat Daun",
|
||||||
|
"普通锈病": "Karat Daun",
|
||||||
|
}
|
||||||
|
|
||||||
|
DAFTAR_FILE_HAPUS = [
|
||||||
|
"CBS28.jpg",
|
||||||
|
"Corn_Common_Rust (1275).jpg",
|
||||||
|
"Corn_Common_Rust (1289).jpg",
|
||||||
|
"Corn_Common_Rust (1295).jpg",
|
||||||
|
"Corn_Gray_Spot (1).jpg"
|
||||||
|
]
|
||||||
|
|
||||||
|
# ==========================================
|
||||||
|
# TAHAP 1: EKSTRAKSI DATASET
|
||||||
|
# ==========================================
|
||||||
|
def ekstrak_semua_zip():
|
||||||
|
print("--- TAHAP 1: Mengekstrak File ZIP ---")
|
||||||
|
for zip_file in DAFTAR_ZIP:
|
||||||
|
if os.path.exists(zip_file):
|
||||||
|
folder_name = os.path.splitext(zip_file)[0]
|
||||||
|
os.makedirs(folder_name, exist_ok=True)
|
||||||
|
try:
|
||||||
|
with zipfile.ZipFile(zip_file, 'r') as zip_ref:
|
||||||
|
zip_ref.extractall(folder_name)
|
||||||
|
print(f" [OK] {zip_file} -> {folder_name}/")
|
||||||
|
except zipfile.BadZipFile:
|
||||||
|
print(f" [ERROR] {zip_file} rusak.")
|
||||||
|
else:
|
||||||
|
print(f" [SKIP] File {zip_file} tidak ditemukan.")
|
||||||
|
print("\n")
|
||||||
|
|
||||||
|
# ==========================================
|
||||||
|
# TAHAP 2: GABUNGKAN DATASET 1 & 2
|
||||||
|
# ==========================================
|
||||||
|
def cari_folder_ds2(base_path, keywords):
|
||||||
|
if not os.path.exists(base_path): return None
|
||||||
|
for f in os.listdir(base_path):
|
||||||
|
f_lower = f.lower()
|
||||||
|
if any(k in f_lower for k in keywords):
|
||||||
|
return os.path.join(base_path, f)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def gabungkan_dataset_1_dan_2():
|
||||||
|
print("--- TAHAP 2: Menggabungkan Dataset 1 & 2 ---")
|
||||||
|
os.makedirs(TARGET_DIR, exist_ok=True)
|
||||||
|
|
||||||
|
# 1. Salin dari dataset_1
|
||||||
|
folder_dari_ds1 = ["Bercak Daun", "Daun Sehat", "Hawar Daun"]
|
||||||
|
for folder in folder_dari_ds1:
|
||||||
|
src = os.path.join("dataset_1", folder)
|
||||||
|
dst = os.path.join(TARGET_DIR, folder)
|
||||||
|
if os.path.exists(src):
|
||||||
|
shutil.copytree(src, dst, dirs_exist_ok=True)
|
||||||
|
print(f" [OK] Menyalin folder {src} ke {dst}")
|
||||||
|
|
||||||
|
# 2. Salin gambar dari dataset_2
|
||||||
|
base_ds2 = os.path.join("dataset_2", "data")
|
||||||
|
mapping_ds2 = {
|
||||||
|
("common_rust", "commont_rust"): "Karat Daun",
|
||||||
|
("healthy",): "Daun Sehat"
|
||||||
|
}
|
||||||
|
|
||||||
|
if os.path.exists(base_ds2):
|
||||||
|
for keywords, target_subfolder in mapping_ds2.items():
|
||||||
|
src_folder = cari_folder_ds2(base_ds2, keywords)
|
||||||
|
dst_folder = os.path.join(TARGET_DIR, target_subfolder)
|
||||||
|
os.makedirs(dst_folder, exist_ok=True)
|
||||||
|
|
||||||
|
if src_folder and os.path.exists(src_folder):
|
||||||
|
file_count = 0
|
||||||
|
for file_name in os.listdir(src_folder):
|
||||||
|
full_file_name = os.path.join(src_folder, file_name)
|
||||||
|
if os.path.isfile(full_file_name):
|
||||||
|
shutil.copy(full_file_name, dst_folder)
|
||||||
|
file_count += 1
|
||||||
|
print(f" [OK] Menyalin {file_count} gambar dari {src_folder} ke {dst_folder}")
|
||||||
|
else:
|
||||||
|
print(f" [SKIP] Folder untuk '{target_subfolder}' tidak ditemukan di {base_ds2}")
|
||||||
|
print("\n")
|
||||||
|
|
||||||
|
# ==========================================
|
||||||
|
# TAHAP 3: GABUNGKAN DATASET 3 (JSON MAPPING)
|
||||||
|
# ==========================================
|
||||||
|
def cari_gambar_fleksibel(folder_sumber, nama_file_target):
|
||||||
|
nama_file_target = nama_file_target.strip()
|
||||||
|
path_langsung = os.path.join(folder_sumber, nama_file_target)
|
||||||
|
if os.path.exists(path_langsung):
|
||||||
|
return path_langsung
|
||||||
|
|
||||||
|
target_lower = nama_file_target.lower()
|
||||||
|
for f in os.listdir(folder_sumber):
|
||||||
|
if f.lower() == target_lower or os.path.splitext(f)[0].lower() == os.path.splitext(target_lower)[0]:
|
||||||
|
return os.path.join(folder_sumber, f)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def gabungkan_dataset_3():
|
||||||
|
print("--- TAHAP 3: Menggabungkan Dataset 3 berdasarkan JSON ---")
|
||||||
|
folder_data = os.path.join("dataset_3", "data")
|
||||||
|
file_json = os.path.join("dataset_3", "desc.json")
|
||||||
|
|
||||||
|
if not os.path.exists(file_json):
|
||||||
|
print(f" [SKIP] {file_json} tidak ditemukan.\n")
|
||||||
|
return
|
||||||
|
|
||||||
|
with open(file_json, 'r', encoding='utf-8') as f:
|
||||||
|
data_label = json.load(f)
|
||||||
|
|
||||||
|
berhasil = 0
|
||||||
|
for item in data_label:
|
||||||
|
image_name = item.get("image_name")
|
||||||
|
label = item.get("label", "").strip()
|
||||||
|
|
||||||
|
if image_name and label in PEMETAAN_KATEGORI:
|
||||||
|
nama_folder_target = PEMETAAN_KATEGORI[label]
|
||||||
|
folder_tujuan = os.path.join(TARGET_DIR, nama_folder_target)
|
||||||
|
os.makedirs(folder_tujuan, exist_ok=True)
|
||||||
|
|
||||||
|
path_sumber = cari_gambar_fleksibel(folder_data, image_name)
|
||||||
|
if path_sumber:
|
||||||
|
nama_asli = os.path.basename(path_sumber)
|
||||||
|
shutil.copy(path_sumber, os.path.join(folder_tujuan, nama_asli))
|
||||||
|
berhasil += 1
|
||||||
|
|
||||||
|
print(f" [OK] Berhasil merutekan {berhasil} gambar dari dataset_3 ke '{TARGET_DIR}'\n")
|
||||||
|
|
||||||
|
# ==========================================
|
||||||
|
# TAHAP 4: PEMBERSIHAN DATA (CLEANING)
|
||||||
|
# ==========================================
|
||||||
|
def bersihkan_dataset():
|
||||||
|
print("--- TAHAP 4: Menghapus File Spesifik ---")
|
||||||
|
set_hapus = set(DAFTAR_FILE_HAPUS)
|
||||||
|
terhapus = 0
|
||||||
|
|
||||||
|
if os.path.exists(TARGET_DIR):
|
||||||
|
for root, _, files in os.walk(TARGET_DIR):
|
||||||
|
for nama_file in files:
|
||||||
|
if nama_file in set_hapus:
|
||||||
|
path_lengkap = os.path.join(root, nama_file)
|
||||||
|
try:
|
||||||
|
os.remove(path_lengkap)
|
||||||
|
print(f" [TERHAPUS] {path_lengkap}")
|
||||||
|
set_hapus.remove(nama_file)
|
||||||
|
terhapus += 1
|
||||||
|
except Exception as e:
|
||||||
|
print(f" [GAGAL] {path_lengkap} ({e})")
|
||||||
|
|
||||||
|
print(f" [OK] Total file dihapus: {terhapus}")
|
||||||
|
if set_hapus:
|
||||||
|
print(f" [INFO] {len(set_hapus)} file tidak ditemukan (mungkin sudah terhapus sebelumnya):")
|
||||||
|
for sisa in set_hapus:
|
||||||
|
print(f" - {sisa}")
|
||||||
|
print("\n")
|
||||||
|
|
||||||
|
# ==========================================
|
||||||
|
# TAHAP 5: BUNGKUS KE ZIP
|
||||||
|
# ==========================================
|
||||||
|
def zip_dataset():
|
||||||
|
print("--- TAHAP 5: Mengompresi Folder Dataset ---")
|
||||||
|
if os.path.exists(TARGET_DIR):
|
||||||
|
print(f" Membuat file {TARGET_DIR}.zip, mohon tunggu sebentar...")
|
||||||
|
# shutil.make_archive(nama_output_tanpa_ext, format, folder_yang_dizip)
|
||||||
|
shutil.make_archive(TARGET_DIR, 'zip', TARGET_DIR)
|
||||||
|
print(f" [OK] Berhasil! File '{TARGET_DIR}.zip' sudah siap.\n")
|
||||||
|
else:
|
||||||
|
print(f" [ERROR] Folder '{TARGET_DIR}' tidak ditemukan, proses zip dibatalkan.\n")
|
||||||
|
|
||||||
|
|
||||||
|
# ==========================================
|
||||||
|
# MAIN EXECUTION
|
||||||
|
# ==========================================
|
||||||
|
if __name__ == "__main__":
|
||||||
|
print("=== MEMULAI PREPROCESSING DATASET ===\n")
|
||||||
|
ekstrak_semua_zip()
|
||||||
|
gabungkan_dataset_1_dan_2()
|
||||||
|
gabungkan_dataset_3()
|
||||||
|
bersihkan_dataset()
|
||||||
|
zip_dataset()
|
||||||
|
print("=== PREPROCESSING SELESAI ===")
|
||||||
|
print(f"Dataset akhir Anda kini siap digunakan di dalam folder '{TARGET_DIR}' dan '{TARGET_DIR}.zip'.")
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
tensorflow>=2.13.0
|
||||||
|
tensorflowjs>=4.10.0
|
||||||
|
jupyter>=1.0.0
|
||||||
|
ipykernel>=6.25.0
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
import os
|
||||||
|
import logging
|
||||||
|
import traceback
|
||||||
|
import tensorflow as tf
|
||||||
|
from tensorflow.keras import layers, models
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
|
||||||
|
|
||||||
|
def build_clean_model(num_classes, img_size=(224, 224)):
|
||||||
|
base_model = tf.keras.applications.EfficientNetV2B0(
|
||||||
|
input_shape=img_size + (3,),
|
||||||
|
include_top=False,
|
||||||
|
weights=None,
|
||||||
|
)
|
||||||
|
inputs = tf.keras.Input(shape=img_size + (3,))
|
||||||
|
x = base_model(inputs, training=False)
|
||||||
|
x = layers.Conv2D(512, (3, 3), padding='same', activation='swish')(x)
|
||||||
|
x = layers.BatchNormalization()(x)
|
||||||
|
x = layers.MaxPooling2D((2, 2))(x)
|
||||||
|
x = layers.Dropout(0.2)(x)
|
||||||
|
x = layers.Conv2D(256, (3, 3), padding='same', activation='swish')(x)
|
||||||
|
x = layers.BatchNormalization()(x)
|
||||||
|
x = layers.GlobalAveragePooling2D()(x)
|
||||||
|
x = layers.Dropout(0.3)(x)
|
||||||
|
x = layers.Dense(1024, activation='swish')(x)
|
||||||
|
x = layers.BatchNormalization()(x)
|
||||||
|
x = layers.Dropout(0.4)(x)
|
||||||
|
outputs = layers.Dense(num_classes, activation='sigmoid', dtype='float32')(x)
|
||||||
|
return models.Model(inputs, outputs)
|
||||||
|
|
||||||
|
logging.info("=== EXPORT STARTED ===")
|
||||||
|
try:
|
||||||
|
MODEL_KERAS_PATH = "best_model/best_model.keras"
|
||||||
|
OUTPUT_DIR = "model"
|
||||||
|
saved_model_dir = os.path.join(OUTPUT_DIR, "saved_model")
|
||||||
|
tflite_path = os.path.join(OUTPUT_DIR, "model.tflite")
|
||||||
|
|
||||||
|
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||||
|
|
||||||
|
if not os.path.exists(MODEL_KERAS_PATH):
|
||||||
|
raise FileNotFoundError(f"Model file not found at {MODEL_KERAS_PATH}")
|
||||||
|
|
||||||
|
logging.info(f"Loading trained weights from {MODEL_KERAS_PATH}...")
|
||||||
|
original_model = tf.keras.models.load_model(MODEL_KERAS_PATH, compile=False)
|
||||||
|
|
||||||
|
logging.info("Building clean architecture...")
|
||||||
|
clean_model = build_clean_model(num_classes=original_model.output_shape[-1])
|
||||||
|
clean_model.set_weights(original_model.get_weights())
|
||||||
|
logging.info("Weights cloned successfully.")
|
||||||
|
|
||||||
|
logging.info(f"Exporting to SavedModel format at: {saved_model_dir}...")
|
||||||
|
tf.saved_model.save(clean_model, saved_model_dir)
|
||||||
|
logging.info("SavedModel export completed successfully.")
|
||||||
|
|
||||||
|
logging.info(f"Converting to TFLite format at: {tflite_path}...")
|
||||||
|
converter = tf.lite.TFLiteConverter.from_keras_model(clean_model)
|
||||||
|
converter.target_spec.supported_ops = [
|
||||||
|
tf.lite.OpsSet.TFLITE_BUILTINS,
|
||||||
|
tf.lite.OpsSet.SELECT_TF_OPS,
|
||||||
|
]
|
||||||
|
converter.optimizations = [tf.lite.Optimize.DEFAULT]
|
||||||
|
tflite_model = converter.convert()
|
||||||
|
with open(tflite_path, "wb") as f:
|
||||||
|
f.write(tflite_model)
|
||||||
|
logging.info("TFLite conversion completed successfully.")
|
||||||
|
logging.info("=== EXPORT COMPLETED ===")
|
||||||
|
|
||||||
|
except Exception:
|
||||||
|
logging.error("EXPORT FAILED")
|
||||||
|
logging.error(traceback.format_exc())
|
||||||
Reference in new issue
Block a user