Skip to content

fix(antivirus | backport): introduce a default max scan size for the full example deployment#620

Merged
fschade merged 1 commit into
opencloud-eu:stable-2.0from
fschade:backport-antivirus-max-size-fix
Apr 10, 2025
Merged

fix(antivirus | backport): introduce a default max scan size for the full example deployment#620
fschade merged 1 commit into
opencloud-eu:stable-2.0from
fschade:backport-antivirus-max-size-fix

Conversation

@fschade

@fschade fschade commented Apr 8, 2025

Copy link
Copy Markdown
Member

Description

The current full example Docker deployment lacks a maximum scan size when the virus scanner is enabled. As a result, all files are scanned regardless of size. However, since Clamav has limits set in the default setup, this combination results in orphaned files that get stuck in post-processing.

The problem has already been fixed in the current main branch. Since an additional scan mode has been integrated here, a full backport is not possible. Therefore, I have only fixed the bare minimum to resolve the bug in these installations.

Related Issue

Motivation and Context

Quite simply, the basic installation from our example deployment should run error-free out of the box

How Has This Been Tested?

  • local test deployment

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation added

@fschade

fschade commented Apr 8, 2025

Copy link
Copy Markdown
Member Author

@micbar I'm totally undecided about whether we should also set a default in the code... the advantage: even non-Docker example deployments would run out of the box with the default clamav config... on the other hand, we would be able to outsmart installations that deliberately have no limit (0) and a correspondingly configured scanner, skipping files larger than the "new" default...

e.g.

hellllp :)

@fschade fschade requested a review from micbar April 8, 2025 15:48
@fschade fschade changed the title fix(antivirus): introduce a default max scan size for the full example deployment fix(antivirus | backport): introduce a default max scan size for the full example deployment Apr 8, 2025
@fschade fschade mentioned this pull request Apr 10, 2025
15 tasks

@dragotin dragotin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Keep changes small for stable branch, ie. doc changes here are almost too much.

@fschade fschade merged commit 1bea4b7 into opencloud-eu:stable-2.0 Apr 10, 2025
@openclouders openclouders mentioned this pull request Apr 10, 2025
1 task

Scanner Scanner
MaxScanSize string `yaml:"max-scan-size" env:"ANTIVIRUS_MAX_SCAN_SIZE" desc:"The maximum scan size the virus scanner can handle. Only this many bytes of a file will be scanned. 0 means unlimited and is the default. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB." introductionVersion:"1.0.0"`
MaxScanSize string `yaml:"max-scan-size" env:"ANTIVIRUS_MAX_SCAN_SIZE" desc:"The maximum scan size the virus scanner can handle. 0 means unlimited. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB." introductionVersion:"1.0.0"`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
MaxScanSize string `yaml:"max-scan-size" env:"ANTIVIRUS_MAX_SCAN_SIZE" desc:"The maximum scan size the virus scanner can handle. 0 means unlimited. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB." introductionVersion:"1.0.0"`
MaxScanSize string `yaml:"max-scan-size" env:"ANTIVIRUS_MAX_SCAN_SIZE" desc:"The maximum size the virus scanner is allowed to scan per file. Bigger files are scanned partially. 0 means unlimited. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB." introductionVersion:"1.0.0"`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, ok, just fix it in main ;-)

@v-scharf

Copy link
Copy Markdown
Contributor

I tested it on my local setup
stable-2.0
opencloud binary. also tried with docker setup (clamav image: image: mkodockx/docker-clamav:alpine)

PROXY_ENABLE_BASIC_AUTH=true \
ANTIVIRUS_SCANNER_TYPE="clamav" \
ANTIVIRUS_CLAMAV_SOCKET="tcp://host.docker.internal:3310" \
POSTPROCESSING_STEPS="virusscan" \
ANTIVIRUS_MAX_SCAN_SIZE=500MB OC_ASYNC_UPLOADS=true \
LOG_LEVEL=debug OC_ADD_RUN_SERVICES="antivirus" \

uploading small image, pdf, txt, move 10mb files works ✅️
finds and deletes virus files ✅️

But: I cannot download:

  • move file 80Mb (screen recording on my mac)
  • zip file ~ 70Mb
  • other files 100Mb, 200Mb 1Gb (mkfile 100m testfile100mb.bin)

all files hang in postprocessing state with log:
{"level":"error","service":"antivirus","error":"write tcp 127.0.0.1:55547->127.0.0.1:3310: write: broken pipe","uploadid":"213952e6-819c-42f5-8345-541bcc62c6fd","time":"2025-04-11T11:51:56+02:00","message":"error scanning file"}

Screenshot 2025-04-11 at 11 36 25

@fschade

fschade commented Apr 11, 2025

Copy link
Copy Markdown
Member Author

ANTIVIRUS_MAX_SCAN_SIZE=500MB

You set the size to 500mb, have you also adjusted the clamav configuration? If not it will fail! The changes in the back port only affect the docker compose setup and does not change any behavior in the binary

@v-scharf

Copy link
Copy Markdown
Contributor

You set the size to 500mb, have you also adjusted the clamav configuration? If not it will fail! The changes in the back port only affect the docker compose setup and does not change any behavior in the binary

Thanks Flo) I figure out how to run opencloud with clamav localy on mac and can confirm that your fix.
test docs is quite nice. I updated it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants