Batch Span Processor does not work with uwsgi

[owais]

Recently support for forking web servers landed in Batch Span Processor in #2242

I tested this and it works with gunicorn but does not work with uwsgi.

Given a file called myapp.py:

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World"

The following command works reports spans

opentelemetry-instrument gunicorn -b 127.0.0.1:8000 --threads 2 --workers 4 myapp:app

While as this does not

opentelemetry-instrument uwsgi --http 127.0.0.1:8000 --wsgi-file myapp.py --callable app --master --processes 2 --enable-threads

I added some debug statements to the fork hook and it seems the hook is called with gunicorn but not with uwsgi.

We should look into how uwsgi works and confirm whether the solution work with it or not.

[owais]

//cc @lonewolf3739

[owais]

Also tested with Celery and it does not work when worker's --pool CLI option is set to threads or gevent.

[srikanthccv]

That change wasn't effective for uwsgi since it's worker proc management implementation is in C. I might have overlooked it when verifying, my apologies. We can probably make use of pid at the time of instantiation and processing to reinitialise. Can you share the more details about how you tested celery? I tried to set the option to given values but was able to see the spans emitted.

[owais]

No worries at all. I just wanted to document what works and what doesn't somewhere so we can track it and patch it at some point.

[owais]

On celery:

tasks.py

from celery import Celery

app = Celery('tasks', broker='redis://localhost/1')

@app.task
def add(x, y):
    return x + y

producer.py

from tasks import add

add.delay(4, 4)

Producing jobs always works obviously:

opentelemetry-instrument python producer.py

Consuming messages works with the following command:

opentelemetry-instrument celery -A tasks worker --loglevel=INFO -c 4

But does not work with either of the following:

opentelemetry-instrument celery -A tasks worker --loglevel=INFO -c 4 --pool gevent

opentelemetry-instrument celery -A tasks worker --loglevel=INFO -c 4 --pool threads

[owais]

@lonewolf3739 My bad. Celery actually works but only if I remove this special case from the sitecustomize file: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py#L128-L141