3.3.8 · AI-ML › Deep Learning Frameworks

Overview

TensorFlow Google ka open-source numerical computation library hai jo machine learning ke liye optimize kiya gaya hai, jabki Keras ek high-level neural network API hai jo TensorFlow (aur doosre backends) ke upar run karta hai. TensorFlow 2.x se, tf.keras official high-level API hai jo directly TensorFlow mein integrated hai.

YE kyun matter karta hai: ~95% production deep learning frameworks use karti hai. Backprop haath se likhna educational hai lekin impractical. TensorFlow computational graph, device placement, aur optimization handle karta hai taaki tum architecture par focus kar sako.

Core Concepts

1. Tensors: The Fundamental Data Structure

Shape hierarchy:

• Rank 0: scalar (single number)
• Rank 1: vector (1D array)
• Rank 2: matrix (2D array)
• Rank 3+: higher-order tensors (images, videos, batches)

Immutable kyun? TensorFlow ko computation graph optimize karne deta hai—ek baar define ho jaaye, toh operations compile aur parallelize ho sakte hain bina side effects ki chinta kiye.

import tensorflow as tf
 
# Creating tensors
scalar = tf.constant(42)                # Shape: ()
vector = tf.constant([1, 2, 3])            # Shape: (3,)
matrix = tf.constant([[1, 2], [3, 4]])     # Shape: (2, 2)
batch_images = tf.zeros((32, 28, 28, 1))   # Shape: (batch, height, width, channels)

Element-wise operations

c = a + b # 6.0, 8.0, [10.0, 12.0]] d = a * b # Element-wise multiplication

Matrix operations

e = tf.matmul(a, b) # Matrix multiplication f = tf.transpose(a) # Transpose

**Ye step kyun?** Har operation computation graph mein ek naya tensor node create karta hai. TensorFlow automatic differentiation ke liye dependencies track karta hai.

### 2. Models Banana: Teen Approaches

#### Sequential API (Sabse Saral)

> [!definition] Sequential Model
> ==Sequential model== layers ka ek linear stack hota hai jahan har layer ka exactly ek input tensor aur ek output tensor hota hai.

**First principles se DERIVATION:**
Neural network functions ka ek composition hai: $y = f_n(f_{n-1}(...f_2(f_1(x))))$

Code mein, ye ban jaata hai:
```python
from tensorflow import keras

model = keras.Sequential([
    keras.layers.Dense(128, activation='relu', input_shape=(784,)),  # f₁
    keras.layers.Dropout(0.2),                                # regularization
    keras.layers.Dense(64, activation='relu'),                       # f₂
    keras.layers.Dense(10, activation='softmax')                     # f₃ (output)
])

Ye structure kyun?

• input_shape=(784,): Pehli layer ko input dimensions jaanni chahiye (flattened 28×28 image)
• Dense(128): 128 neurons wala fully connected layer → 784×128 + 128 = 100,480 parameters seekhta hai
• activation='relu': Non-linearity (iske bina, layers stack karna = single linear transformation)
• Dropout(0.2): Training ke dauran 20% activations randomly zero kar deta hai → overfitting rokta hai
• softmax: Logits ko 10 classes par probability distribution mein convert karta hai

$\mathbf{z}=\mathbf{xW}+\mathbf{b}$ $\mathbf{a}=\sigma (\mathbf{z})$

jahan $\mathbf{x}$ input hai (batch_size × n), $\sigma$ activation function hai.

Derivation: Har neuron $j$ compute karta hai $z_j={\sum }_{i=1}^n{w}_{ij}{x}_i+b_j$, efficiency ke liye matrix multiplication ke roop mein vectorized.

Functional API (Flexible)

inputs = keras.Input(shape=(784,))
x = keras.layers.Dense(128, activation='relu')(inputs)
x = keras.layers.Dropout(0.2)(x)
x = keras.layers.Dense(64, activation='relu')(x)
outputs = keras.layers.Dense(10, activation='softmax')(x)
 
model = keras.Model(inputs=inputs, outputs=outputs)

Ye kyun use karein? Complex architectures enable karta hai:

# Multi-input example
input1 = keras.Input(shape=(128,))
input2 = keras.Input(shape=(64,))
x1 = keras.layers.Dense(64)(input1)
x2 = keras.layers.Dense(64)(input2)
merged = keras.layers.concatenate([x1, x2])
output = keras.layers.Dense(1)(merged)
 
model = keras.Model(inputs=[input1, input2], outputs=output)

Model Subclassing (Sabse Zyada Control)

class CustomModel(keras.Model):
    def __init__(self):
        super().__init__()
        self.dense1 = keras.layers.Dense(128, activation='relu')
        self.dropout = keras.layers.Dropout(0.2)
        self.dense2 = keras.layers.Dense(10, activation='softmax')
    def call(self, inputs, training=False):
        x = self.dense1(inputs)
        if training:  # Only apply dropout during training
            x = self.dropout(x, training=training)
        return self.dense2(x)

Subclass kyun karein? Dynamic computation graphs, custom training loops, research experimentation.

3. Training Pipeline: Compile → Fit → Evaluate

1. Loss function $\mathcal{L}(\theta )$: Prediction error measure karta hai
2. Optimizer: Parameters update karta hai $\theta \leftarrow \theta -\eta {\nabla }_{\theta }\mathcal{L}$
3. Metrics: Performance monitor karta hai (accuracy, F1, etc.)

Gradient descent ki DERIVATION: Taylor expansion se shuru karte hue: $\mathcal{L}(\theta -\eta \nabla \mathcal{L})\approx \mathcal{L}(\theta )-\eta |\nabla \mathcal{L}|{|}^2$

Chhote $\eta >0$ ke liye, gradient ke opposite direction mein jaane se loss decrease hota hai.

# Compile: Define learning algorithm
model.compile(
    optimizer=keras.optimizers.Adam(learning_rate=0.001),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=['accuracy']
)

Ye choices kyun?

• Adam optimizer: Har parameter ke liye adaptive learning rates (momentum + RMSprop combine karta hai)
• SparseCategoricalCrossentropy: Integer labels (0-9) ke liye one-hot vectors ki jagah
• Accuracy metric: Correct predictions ka % (loss se zyada interpretable)

Load data

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

Preprocessing

x_train = x_train.reshape(-1, 784).astype('float32') / 255.0 # Flatten & normalize x_test = x_test.reshape(-1, 784).astype('float32') / 255.0

Build model

model = keras.Sequential([ keras.layers.Dense(128, activation='relu', input_shape=(784,)), keras.layers.Dropout(0.2), keras.layers.Dense(10, activation='softmax') ])

Compile

model.compile( optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] )

Train

history = model.fit( x_train, y_train, batch_size=32, epochs=5, validation_split=0.2, # Use 20% of training data for validation verbose=1 )

Evaluate

test_loss, test_acc = model.evaluate(x_test, y_test) print(f"Test accuracy: {test_acc:.4f}")

**Step-by-step WHY:**
1. **Reshape to (6000, 784):** Dense layers flat vectors expect karti hain, 2D images nahi
2. **255 se divide karo:** Pixels ko [0,1] range mein normalize karta hai → faster convergence (gradients better scaled hote hain)
3. **batch_size=32:** Ek baar 32 examples process karo → memory aur gradient noise ke beech balance
4. **epochs=5:** Ek epoch = pure training set se ek baar guzarna
5. **validation_split=0.2:** Unseen data par loss track karke overfitting monitor karo

### 4. Key Operations

> [!formula] Common Layer Types

**Dense (Fully Connected):**
$$\mathbf{y} = \sigma(\mathbf{W}\mathbf{x} + \mathbf{b})$$
Parameters: $n_{in} \times n_{out} + n_{out}$

**Dropout:**
Training ke dauran: $\mathbf{y}_i = \begin{cases} 0 & \text{with probability } p \\ \frac{\mathbf{x}_i}{1-p} & \text{otherwise} \end{cases}$

**$(1-p)$ se divide kyun karein?** Expected value maintain karta hai: $\mathbb{E}[\mathbf{y}_i] = \mathbb{E}[\mathbf{x}_i]$ (inverted dropout)

**Softmax (classification ke liye output layer):**
$$\text{softmax}(\mathbf{z})_i = \frac{e^{z_i}}{\sum_{j=1}^{K} e^{z_j}}$$

**Derivation:** Hum aisi outputs chahte hain jo:
1. 1 tak sum karein (probability distribution)
2. Positive hon
3. Ordering preserve karein (bada logit → badi probability)

Exponential (2) aur (3) satisfy karta hai, normalization (1) satisfy karta hai.

## Common Patterns

### Data Pipeline
```python
# Create tf.data.Dataset for efficient loading
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_dataset = train_dataset.shuffle(1000).batch(32).prefetch(tf.data.AUTOTUNE)

# Use in training
model.fit(train_dataset, epochs=5)

Ye approach kyun?

• shuffle: Order randomize karta hai → model ko sequence patterns sikhne se rokta hai
• batch: Examples group karta hai → efficient GPU computation
• prefetch: Current batch train hote waqt next batch load karta hai → I/O latency hide karta hai

Callbacks

callbacks = [
    keras.callbacks.EarlyStopping(patience=3, restore_best_weights=True),
    keras.callbacks.ModelCheckpoint('best_model.h5', save_best_only=True),
    keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=2)
]
 
model.fit(x_train, y_train, epochs=100, callbacks=callbacks)

Callbacks kyun use karein?

• EarlyStopping: Jab validation loss improve karna band kar de toh training rok deta hai (overfitting rokta hai)
• ModelCheckpoint: Training ke dauran best model save karta hai (sirf last epoch nahi)
• ReduceLROnPlateau: Jab stuck ho jaaye toh learning rate decrease karta hai → plateaus se baahir nikalne mein madad karta hai

Ye sahi kyun lagta hai: Network ko training ke through appropriate scale seekhni chahiye.

Problem: Badi input values → bade gradients → unstable training ya exploding gradients. Optimizer learning rates normalized inputs ke liye tuned hoti hain.

Fix:

x = x / 255.0  # Scale to [0, 1]
# OR
x = (x - mean) / std  # Standardize to zero mean, unit variance

Steel-man: Agar tum specific input scale ke liye learning rate aur weight initialization carefully tune karo, toh unnormalized data kaam kar sakta hai—lekin ye fragile hai aur convergence slow karta hai.

But labels are integers: [3, 7, 2 ...]

**Ye sahi kyun lagta hai:** "Categorical" categories ke liye sahi lagta hai.

**Problem:** `categorical_crossentropy` shape (batch_size, num_classes) expect karta hai one-hot encoding ke saath. Integer labels ka shape (batch_size,) hota hai.

**Fix:**
```python
# For integer labels: use sparse version
model.compile(loss='sparse_categorical_crossentropy', ...)

# OR one-hot encode labels first
y_train_onehot = keras.utils.to_categorical(y_train, num_classes=10)
model.compile(loss='categorical_crossentropy', ...)

Problem: Dropout randomly neurons zero karta hai → stochastic predictions. Test time par hum deterministic, averaged predictions chahte hain.

Keras automatically kya karta hai:

• Training: Dropout active (scaling ke saath)
• Inference (model.predict(), model.evaluate()): Dropout disabled

Manual control:

# Force training mode
predictions = model(x, training=True)  # Dropout active
 
# Force inference mode
predictions = model(x, training=False)  # Dropout disabled

Yahin Keras aata hai—ye ek simple instruction manual ki tarah hai. "Is 28×28 grid of pixels ko lo, 784 numbers mein flatten karo, is matrix se multiply karo, ye numbers add karo, ye squishing function apply karo..." kehne ki jagah tum bas Dense(128, activation='relu') kehte ho aur Keras saari details handle kar leta hai.

Ek model train karna sikhane jaisa hai:

1. Robot ko bahut saare examples dikhao ("ye 3 hai", "ye 7 hai")
2. Ye andaza lagata hai aur tum batate ho jab galat ho
3. Ye apne "brain connections" (weights) adjust karta hai agli baar better karne ke liye
4. Hazaron examples dekhne ke baad, ye un naye numbers ko pehchanने mein bahut accha ho jaata hai jो usne kabhi nahi dekhe

Batch_size groups mein padhne jaisa hai—ek baar mein ek example se sikhne ki jagah (slow!), tum ise 32 examples saath dikhate ho aur average galti ke basis par update karte ho. Bahut fast!

"Ek baby CALF chalna seekhta hai" → Ek model predict karna seekhta hai

Connections

• 3.1.01-Gradient-Descent - TensorFlow gradients automatically compute karta hai
• 3.2.03-Backpropagation - model.fit() ke andar kya hota hai
• 3.3.01-Neural-Network-Basics - Dense layers ke peeche ka math
• 3.07-Activation-Functions - Hum ReLU, softmax kyun use karte hain
• 3.3.09-PyTorch-basics - Alternative framework (zyada Pythonic)
• 3.4.05-Overfitting-Regularization - Hum Dropout, validation split kyun use karte hain
• 4.2.01-CNN-Architecture - Image models ke liye Keras (Conv2D layers)

#flashcards/ai-ml

TensorFlow mein tensor kya hota hai? :: Ek multi-dimensional array jisme uniform data type hoti hai, immutable, jo computational graphs mein flow karta hai. Scalars (rank 0), vectors (rank 1), matrices (rank 2) ko higher dimensions tak generalize karta hai.

Concept Map