Is My Understanding of This Autoencoder Architecture Correct?

[Surya-0823]

Hi everyone,

I'm trying to deeply understand the reasoning behind choosing the architecture below for an anomaly detection / fraud detection system.

35 → 24 → 12 → 6 → 12 → 24 → 35

• Input: 35 features
• Encoder: 35 → 24 → 12 → 6
• Latent (Bottleneck): 6 dimensions
• Decoder: 6 → 12 → 24 → 35
• Activation: LeakyReLU
• Batch Normalization
• Dropout = 0.2

I'm not looking for a simple "yes" or "no." I want to know whether my understanding is technically correct or where my reasoning is flawed.

My Current Understanding

From what I understand:

1. The encoder gradually compresses the original 35-dimensional feature vector into a 6-dimensional latent representation.

2. The bottleneck should ideally capture only the essential information needed to reconstruct normal data.

3. The decoder then reconstructs the original 35 features from those 6 latent values.

4. During inference, if the reconstruction error is high, the sample is considered anomalous.

---

My Main Questions

1. Why was this specific architecture chosen?

Why choose

35 → 24 → 12 → 6

instead of something like

35 → 28 → 14 → 8

Is there any mathematical principle, empirical rule, or engineering guideline behind selecting these hidden layer sizes?

---

2. Is a 6-dimensional bottleneck appropriate for a 35-feature input?

How do we determine whether 6 latent dimensions are sufficient?

Should the bottleneck size be based on:

• intrinsic dimensionality?
• PCA explained variance?
• manifold learning?
• empirical experimentation?
• domain knowledge?
• reconstruction performance on validation data?

How do experienced ML engineers make this decision?

---

3. Does a smaller bottleneck always improve anomaly detection?

My assumption is that:

• Smaller bottleneck

  • more compression
  • harder reconstruction
  • potentially better anomaly separation

But if the bottleneck becomes too small:

• normal samples may also reconstruct poorly
• reconstruction error variance increases
• false positives increase

Is this understanding correct?

---

4. How does Dropout interact with a small bottleneck?

Suppose Dropout = 0.2 is applied before the bottleneck.

Does the same Dropout behave differently in

35 → 24 → 12 → 6

versus

35 → 28 → 14 → 8

because the wider network has more redundant information?

Or is my reasoning incorrect?

---

5. Does a larger bottleneck necessarily reduce false positives?

Suppose I compare these two models.

Model A

35 → 24 → 12 → 6 → 12 → 24 → 35

Model B

35 → 28 → 14 → 8 → 14 → 28 → 35

If Model A produces many false positives while Model B performs better,

is it reasonable to conclude that:

• Model A is over-compressing the data
• the latent space is too restrictive
• normal samples cannot be reconstructed accurately
• reconstruction error for normal data becomes more variable
• anomaly threshold becomes harder to calibrate

Or could there be completely different reasons?

---

6. Is the wider architecture actually better?

If the wider model reduces false positives,

does that necessarily mean

35 → 28 → 14 → 8

is the better architecture?

Or could it simply be overfitting the normal data?

How would you verify that experimentally?

---

7. How would you choose the architecture in a production system?

If you were designing a production anomaly detection pipeline, how would you decide between:

35 → 24 → 12 → 6

and

35 → 28 → 14 → 8

What metrics would you compare?

For example:

• Validation reconstruction loss
• Precision
• Recall
• False Positive Rate
• False Negative Rate
• ROC-AUC
• PR-AUC
• Latent space visualization
• Threshold stability
• Generalization on unseen data

Would you use additional techniques such as PCA, intrinsic dimensionality estimation, or latent space analysis before fixing the architecture?

---

My Goal

I'm trying to understand the engineering reasoning behind selecting encoder and bottleneck sizes rather than treating them as arbitrary hyperparameters.

If any of my assumptions above are incorrect, I'd really appreciate corrections along with the reasoning behind them.

I'm especially interested in learning how experienced ML engineers justify these architecture decisions during real-world design reviews.

[CarterPerez-dev]

Hi everyone,

I'm trying to deeply understand the reasoning behind choosing the architecture below for an anomaly detection / fraud detection system.

35 → 24 → 12 → 6 → 12 → 24 → 35

• Input: 35 features

• Encoder: 35 → 24 → 12 → 6

• Latent (Bottleneck): 6 dimensions

• Decoder: 6 → 12 → 24 → 35

• Activation: LeakyReLU

• Batch Normalization

• Dropout = 0.2

I'm not looking for a simple "yes" or "no." I want to know whether my understanding is technically correct or where my reasoning is flawed.

My Current Understanding

From what I understand:

1. The encoder gradually compresses the original 35-dimensional feature vector into a 6-dimensional latent representation.

2. The bottleneck should ideally capture only the essential information needed to reconstruct normal data.

3. The decoder then reconstructs the original 35 features from those 6 latent values.

4. During inference, if the reconstruction error is high, the sample is considered anomalous.

---

My Main Questions

1. Why was this specific architecture chosen?

Why choose

35 → 24 → 12 → 6

instead of something like

35 → 28 → 14 → 8

Is there any mathematical principle, empirical rule, or engineering guideline behind selecting these hidden layer sizes?

---

2. Is a 6-dimensional bottleneck appropriate for a 35-feature input?

How do we determine whether 6 latent dimensions are sufficient?

Should the bottleneck size be based on:

• intrinsic dimensionality?

• PCA explained variance?

• manifold learning?

• empirical experimentation?

• domain knowledge?

• reconstruction performance on validation data?

How do experienced ML engineers make this decision?

---

3. Does a smaller bottleneck always improve anomaly detection?

My assumption is that:

• Smaller bottleneck

  • more compression

  • harder reconstruction

  • potentially better anomaly separation

But if the bottleneck becomes too small:

• normal samples may also reconstruct poorly

• reconstruction error variance increases

• false positives increase

Is this understanding correct?

---

4. How does Dropout interact with a small bottleneck?

Suppose Dropout = 0.2 is applied before the bottleneck.

Does the same Dropout behave differently in

35 → 24 → 12 → 6

versus

35 → 28 → 14 → 8

because the wider network has more redundant information?

Or is my reasoning incorrect?

---

5. Does a larger bottleneck necessarily reduce false positives?

Suppose I compare these two models.

Model A

35 → 24 → 12 → 6 → 12 → 24 → 35

Model B

35 → 28 → 14 → 8 → 14 → 28 → 35

If Model A produces many false positives while Model B performs better,

is it reasonable to conclude that:

• Model A is over-compressing the data

• the latent space is too restrictive

• normal samples cannot be reconstructed accurately

• reconstruction error for normal data becomes more variable

• anomaly threshold becomes harder to calibrate

Or could there be completely different reasons?

---

6. Is the wider architecture actually better?

If the wider model reduces false positives,

does that necessarily mean

35 → 28 → 14 → 8

is the better architecture?

Or could it simply be overfitting the normal data?

How would you verify that experimentally?

---

7. How would you choose the architecture in a production system?

If you were designing a production anomaly detection pipeline, how would you decide between:

35 → 24 → 12 → 6

and

35 → 28 → 14 → 8

What metrics would you compare?

For example:

• Validation reconstruction loss

• Precision

• Recall

• False Positive Rate

• False Negative Rate

• ROC-AUC

• PR-AUC

• Latent space visualization

• Threshold stability

• Generalization on unseen data

Would you use additional techniques such as PCA, intrinsic dimensionality estimation, or latent space analysis before fixing the architecture?

---

My Goal

I'm trying to understand the engineering reasoning behind selecting encoder and bottleneck sizes rather than treating them as arbitrary hyperparameters.

If any of my assumptions above are incorrect, I'd really appreciate corrections along with the reasoning behind them.

I'm especially interested in learning how experienced ML engineers justify these architecture decisions during real-world design reviews.