Gradient Descent from Scratch

This repository documents my journey of understanding and implementing Gradient Descent from scratch using Python.

The repository starts with the core algorithm on a simple dataset and then applies the same implementation to a real-world machine learning dataset.

---

Repository Structure

gradient-descent/
│
├── README.md
├── requirements.txt
├── .gitignore
│
├── basics/
│   ├── gradient_descent.ipynb
│   └── README.md
│   └── images/
│   └── .gitignore
│   └── requirements.txt
│
└── insurance/
    ├── gradient_descent_insurance.ipynb
    ├── data/
    ├── images/
    └── README.md
    └── .gitignore
    └── requirements.txt

---

Projects

Basics

Implementation of Gradient Descent on a simple linear dataset to understand:

• Cost Function
• Gradient Computation
• Learning Rate
• Parameter Updates
• Convergence

---

Insurance Dataset

Application of the same Gradient Descent algorithm on the Medical Cost Personal Dataset.

Features include:

• Feature Standardization
• Early Stopping
• Cost vs Iterations Visualization
• Regression Line
• Comparison with Scikit-learn
• Regression Evaluation Metrics

---

Technologies Used

• Python
• NumPy
• Pandas
• Matplotlib
• Scikit-learn
• Jupyter Notebook

---

Learning Outcomes

Through these projects I gained a practical understanding of:

• Linear Regression
• Gradient Descent
• Mean Squared Error
• Feature Scaling
• Learning Rate
• Model Convergence
• Regression Metrics

---

Future Work

• Mini-batch Gradient Descent
• Stochastic Gradient Descent (SGD)
• Logistic Regression