V8 CI is not Python 3 compatible

[targos]

Job: https://ci.nodejs.org/job/node-test-commit-v8-linux
It is now permanently red for the master branch.

There are at least two issues:

1. The Python version installed on centos7-ppcle hosts doesn't include bzip2 support:
   build: ongoing list of actions for Python 3 compatibility #25789 (comment)
   https://stackoverflow.com/questions/12806122/missing-python-bz2-module
   https://ci.nodejs.org/job/node-test-commit-v8-linux/2592/nodes=centos7-ppcle,v8test=v8test/console Fixed

2. gclient doesn't yet support Python 3
   Dependency v8 is not Python 3 compatible #24512 (comment)
   https://ci.nodejs.org/job/node-test-commit-v8-linux/2592/nodes=benchmark-ubuntu1604-intel-64,v8test=v8test/console

[targos]

/cc @nodejs/platform-ppc for 1.

/cc @nodejs/build for 2.

Is Python 3 the default Python on test-nearform_intel-ubuntu1604-x64-1 ?

[rvagg]

Is Python 3 the default Python on test-nearform_intel-ubuntu1604-x64-1 ?

It shouldn't be. I believe python is always Python 2 on Linuxes, you have to opt-in to python3. There's very few places where we're forcing python3.

21:48:44 + ./configure
21:48:46 Node configure: Found Python 3.5.2...

I don't know why it's selecting that, my understanding of configure is that if python2.7 is found that it'll use that:

$ ssh test-nearform_intel-ubuntu1604-x64-1 which python2.7
/usr/bin/python2.7

There's also something wrong with this:

21:48:46 WARNING: C++ compiler (CXX=g++, 5.5.0) too old, need g++ 6.3.0 or clang++ 8.0.0

select-compiler.sh was supposed to fix that last week but it doesn't seem to be working: https://github.com/nodejs/build/blob/74a34e2d9ee5dc4fb945e7c9a5eff388e78227a6/jenkins/scripts/select-compiler.sh#L136-L141

I think it's to do with the switch to case and the match needs to allow for preceding characters. Probably *ubuntu1604-*64*. The $nodes in this instance should be benchmark-ubuntu1604-intel-64 I believe but ubuntu1604-64 on node-test-commit-linux. Can someone get that sanity check that and get a PR up? I'm off for the day.

[rvagg]

@nodejs/python FYI ^ "gclient doesn't yet support Python 3", yay

(EDIT: Just an FYI, this isn't a critical dependency, we just need to be able to run it in our infra for v8 builds)

[sam-github]

FYI @targos I'm working on rebuilding python-3.7 with bzip2 support on the *-osuosl-centos7-ppc64-le-* hosts.

[richardlau]

@AshCripps has re-ansibled the test-osuosl-centos7-ppc64-le-* hosts and the bzip2 issue looks to be resolved. https://ci.nodejs.org/job/node-test-commit-v8-linux/nodes=centos7-ppcle,v8test=v8test/2622/console is now failing with a different error:

Details

09:49:49 Failed to fetch file gs://chromium-clang-format/942fc8b1789144b8071d3fc03ff0fcbe1cf81ac8 for v8/buildtools/linux64/clang-format. [Err: Traceback (most recent call last):
09:49:49   File "/home/iojs/build/workspace/node-test-commit-v8-linux/nodes/centos7-ppcle/v8test/v8test/deps/v8/_depot_tools/gsutil.py", line 190, in <module>
09:49:49     sys.exit(main())
09:49:49   File "/home/iojs/build/workspace/node-test-commit-v8-linux/nodes/centos7-ppcle/v8test/v8test/deps/v8/_depot_tools/gsutil.py", line 186, in main
09:49:49     clean=args.clean)
09:49:49   File "/home/iojs/build/workspace/node-test-commit-v8-linux/nodes/centos7-ppcle/v8test/v8test/deps/v8/_depot_tools/gsutil.py", line 134, in run_gsutil
09:49:49     gsutil_bin = ensure_gsutil(force_version, target, clean)
09:49:49   File "/home/iojs/build/workspace/node-test-commit-v8-linux/nodes/centos7-ppcle/v8test/v8test/deps/v8/_depot_tools/gsutil.py", line 113, in ensure_gsutil
09:49:49     target_zip_filename = download_gsutil(version, instance_dir)
09:49:49   File "/home/iojs/build/workspace/node-test-commit-v8-linux/nodes/centos7-ppcle/v8test/v8test/deps/v8/_depot_tools/gsutil.py", line 70, in download_gsutil
09:49:49     u = urllib.urlopen(url)
09:49:49   File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen
09:49:49     return opener.open(url, data, timeout)
09:49:49   File "/usr/local/lib/python3.7/urllib/request.py", line 525, in open
09:49:49     response = self._open(req, data)
09:49:49   File "/usr/local/lib/python3.7/urllib/request.py", line 548, in _open
09:49:49     'unknown_open', req)
09:49:49   File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
09:49:49     result = func(*args)
09:49:49   File "/usr/local/lib/python3.7/urllib/request.py", line 1387, in unknown_open
09:49:49     raise URLError('unknown url type: %s' % type)
09:49:49 urllib.error.URLError: <urlopen error unknown url type: https>
09:49:49 ]

[cclauss]

$ python3
>>> import urllib
>>> urllib.urlopen('https://nodejs.org/en/').read() # does that print HTML?

>>> import ssl # does that work?

[richardlau]

On test-osuosl-centos7-ppc64_le-1:

$ `python3`
>>> `import urllib`
>>> `urllib.urlopen('https://nodejs.org/en/').read()` # does that print HTML?

>>> import urllib
>>> urllib.urlopen('https://nodejs.org/en/').read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'urllib' has no attribute 'urlopen'
>>>

>>> `import ssl` # does that work?

>>> import ssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
>>>

[richardlau]

ModuleNotFoundError: No module named '_ssl'

So given we're building Python 3.7 from source on centos7-ppc64 this is most likely missing openssl-devel (in a similar fashion to missing bzip2-devel) based on https://stackoverflow.com/questions/5128845/importerror-no-module-named-ssl?

cc @AshCripps @sam-github

[sam-github]

Seems reasonable, @AshCripps can you try adding openssl-devel to the deps, right after bzip2-devel?

[AshCripps]

Ive added openssl-devel to the playbooks nodejs/build#2044 and its now installing on the machine, I just need someone to disable the machines on jenkins so I can rebuild python on them