Recipe for Python part of Apache Arrow by wesm · Pull Request #1732 · conda-forge/staged-recipes

wesm · 2016-10-07T20:41:33Z

Requires #1729 and #992

conda-forge-linter · 2016-10-07T20:41:38Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/pyarrow) and found some lint.

Here's what I've got...

For recipes/pyarrow:

Failed to even lint the recipe (might be a conda-smithy bug) 😢

chrisaycock · 2016-10-10T15:30:37Z

+  version: {{ version }}
+
+source:
+  git_url: https://github.com/apache/arrow.git


Does the feedback from @ocefpaf on arrow-cpp here also apply?

Yes. If you could make those changes it will make this clearer.

done, will wait for #992 to become feedstock then get this to a green build

chrisaycock · 2016-10-10T15:36:16Z

The build and dependencies LGTM.

wesm · 2016-10-11T03:09:53Z

This is failing because of PARQUET-747. I will update the parquet-cpp build and report back when this build is green

chrisaycock · 2016-10-11T13:56:54Z

Link: https://issues.apache.org/jira/browse/PARQUET-747

wesm · 2016-10-11T19:51:22Z

OK, the parquet-cpp conda-forge package has been updated, looks like we're good to go whenever the OS X build passes

ocefpaf · 2016-10-11T19:59:54Z

+  number: 0
+  rpaths:
+    - lib                                                        # [unix]
+    - lib/python{{environ.get('PY_VER')}}/site-packages/pyarrow  # [unix]


The linter is failing here. Would {{ environ['PY_VER'] + '*' }} work instead? That is known to pass.

conda-forge-linter · 2016-10-11T20:23:45Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/pyarrow) and found it was in an excellent condition.

wesm · 2016-10-11T22:18:16Z

@ocefpaf nope, that broke my rpath. trying again without the '*'

ocefpaf · 2016-10-11T22:53:56Z

@ocefpaf nope, that broke my rpath. trying again without the '*'

No problem. We should probably fix the linter and not the recipe anyways.

wesm · 2016-10-12T02:33:10Z

This is good to go once the build is green

wesm · 2016-10-12T03:23:32Z

Argh, well we have an rpath issue on OS X now. Looking.

wesm · 2016-10-12T03:55:28Z

@ocefpaf I'm pretty bewildered by the OS X rpath stuff, do you have any advice? Our package builds on our Travis CI are working for OS X, e.g. https://travis-ci.org/apache/arrow/jobs/166942076

The issue here is that there is a C++ shared library nested under pyarrow/ and this is being hidden (?) by the install_name_tool. I can't tell what's different about the conda forge environment that would make this fail. It's XCode 6.0 versus 6.1, but not sure if that's enough to screw things up.

ocefpaf · 2016-10-12T09:20:40Z

@ocefpaf I'm pretty bewildered by the OS X rpath stuff, do you have any advice?

Sorry, never used a Mac in my life. Maybe @msarahan can help here.

Our package builds on our Travis CI are working for OS X, e.g. https://travis-ci.org/apache/arrow/jobs/166942076

Are you building with conda there? Note that the xcode is different too, but I don't think that is the issue.

The issue here is that there is a C++ shared library nested under pyarrow/ and this is being hidden (?) by the install_name_tool. I can't tell what's different about the conda forge environment that would make this fail. It's XCode 6.0 versus 6.1, but not sure if that's enough to screw things up.

I maintain this recipe where we had to manually change paths with install_name_tool. Maybe that is the case here?

wesm · 2016-10-12T14:29:21Z

I'm trying to resolve this in apache/arrow#171, setting this to build from that PR

ocefpaf · 2016-10-12T14:31:34Z

@wesm Linux is OK and you are already skipping Windows. Do you want to skip OS X for now, get this into a feedstock, and experiment there? It would make your tests more agile and you would free AppVeyor's queue. (Even when skipping AppVeyor will start. You can force skip it by adding skip appveyor in the commit message though.)

wesm · 2016-10-12T14:35:13Z

Let me see if this commit fixed it, and if not I'll make this Linux only to go to feedstock

wesm · 2016-10-17T21:04:42Z

@msarahan @ocefpaf @jakirkham this problem appears to be localized to the old version of XCode being used in conda-forge -- I left a comment on the bigger discussion about OS X architecture

wesm · 2016-10-19T03:16:08Z

@ocefpaf I disabled all but Linux for now, I can re-enable OS X when the XCode toolchain issues are resolved. thanks!

ocefpaf · 2016-10-19T08:51:51Z

Linux is no longer 😒

import: 'pyarrow'
Traceback (most recent call last):
  File "/staged-recipes/build_artefacts/test-tmp_dir/run_test.py", line 26, in <module>
    import pyarrow
  File "/opt/conda/envs/_test/lib/python3.5/site-packages/pyarrow/__init__.py", line 20, in <module>
    import pyarrow.config
ImportError: /opt/conda/envs/_test/lib/python3.5/site-packages/pyarrow/../../../libparquet_arrow.so: undefined symbol: _ZN5arrow16PrimitiveBuilderINS_11BooleanTypeEE6FinishEv
TESTS FAILED: pyarrow-0.1.post-0

wesm · 2016-10-19T11:27:55Z

Visibility is such a pain! I will fix

ocefpaf · 2016-10-19T11:33:10Z

I can re-enable OS X when the XCode

@wesm this kind of changes needs more discussion. In principle I am 👍 to that but we need to consensus as this change should happen everywhere. (In fact, for my local builds, I use homebrew's gcc so I am not tied to OS X specific stuff.)

It would be nice if you, or someone in your group, could participate some of our meetings and request that item to be discussed. See https://conda-forge.hackpad.com/conda-forge-meetings-2YkV96cvxPG

Maybe we could start by putting together a conda-forge-enhancement-proposal to move to a modern XCode. See https://github.com/conda-forge/conda-forge-enhancement-proposals

wesm · 2016-10-19T11:57:52Z

Oh, actually this is because the parquet-cpp package is out of date, whoops.

…ds but Linux until OS X toolchain issues resolved

wesm · 2016-10-19T20:25:12Z

@ocefpaf looks like something is broken in the conda forge toolchain now?

ocefpaf · 2016-10-19T20:29:33Z

@ocefpaf looks like something is broken in the conda forge toolchain now?

Good old "turn it off and on again" seems to fix it 😄

I re-started CircleCI and cancelled AppVeyor and Travis-CI since you are only building Linux for now.

wesm · 2016-10-19T20:31:19Z

thank you!

mariusvniekerk mentioned this pull request Oct 9, 2016

Add build recipe for Arrow HEAD commit, 0.1 pre-release #1729

Merged

chrisaycock reviewed Oct 10, 2016

View reviewed changes

wesm force-pushed the pyarrow branch 5 times, most recently from daaffca to 0b7593d Compare October 10, 2016 23:48

wesm force-pushed the pyarrow branch from 0b7593d to c165bec Compare October 11, 2016 19:46

ocefpaf reviewed Oct 11, 2016

View reviewed changes

wesm force-pushed the pyarrow branch from c165bec to 8094a22 Compare October 11, 2016 20:23

This was referenced Oct 13, 2016

Help with RPATH on OS X with cmake conda/conda#3624

Closed

ARROW-334: [Python] Remove INSTALL_RPATH_USE_LINK_PATH apache/arrow#171

Closed

Travis CI OS X Platform support/OS X Compatibility conda-forge/conda-forge.github.io#249

Closed

wesm force-pushed the pyarrow branch from 377b0bf to 9b8e52a Compare October 19, 2016 03:15

Recipe for pyarrow, the Python part of Apache Arrow. Disable all buil…

b019c27

…ds but Linux until OS X toolchain issues resolved

wesm force-pushed the pyarrow branch from 9b8e52a to b019c27 Compare October 19, 2016 20:09

ocefpaf merged commit 90eba6c into conda-forge:master Oct 19, 2016

wesm deleted the pyarrow branch October 19, 2016 20:31

This was referenced Oct 19, 2016

[C++] Symbol visibility for PrimitiveBuilder subclasses is not uniform across platforms toddfarmer/arrow-migration#300

Closed

[C++] Symbol visibility for PrimitiveBuilder subclasses is not uniform across platforms toddfarmer/arrow-migration#2330

Closed

Uh oh!

Uh oh!

Conversation

wesm commented Oct 7, 2016

Uh oh!

conda-forge-linter commented Oct 7, 2016

Uh oh!

chrisaycock Oct 10, 2016

Choose a reason for hiding this comment

Uh oh!

ocefpaf Oct 10, 2016

Choose a reason for hiding this comment

Uh oh!

wesm Oct 10, 2016

Choose a reason for hiding this comment

Uh oh!

chrisaycock commented Oct 10, 2016

Uh oh!

wesm commented Oct 11, 2016

Uh oh!

chrisaycock commented Oct 11, 2016

Uh oh!

wesm commented Oct 11, 2016

Uh oh!

ocefpaf Oct 11, 2016

Choose a reason for hiding this comment

Uh oh!

wesm Oct 11, 2016

Choose a reason for hiding this comment

Uh oh!

conda-forge-linter commented Oct 11, 2016

Uh oh!

wesm commented Oct 11, 2016

Uh oh!

ocefpaf commented Oct 11, 2016

Uh oh!

wesm commented Oct 12, 2016

Uh oh!

wesm commented Oct 12, 2016

Uh oh!

wesm commented Oct 12, 2016

Uh oh!

ocefpaf commented Oct 12, 2016

Uh oh!

wesm commented Oct 12, 2016

Uh oh!

ocefpaf commented Oct 12, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wesm commented Oct 12, 2016

Uh oh!

wesm commented Oct 17, 2016

Uh oh!

wesm commented Oct 19, 2016

Uh oh!

ocefpaf commented Oct 19, 2016

Uh oh!

wesm commented Oct 19, 2016

Uh oh!

ocefpaf commented Oct 19, 2016

Uh oh!

wesm commented Oct 19, 2016

Uh oh!

wesm commented Oct 19, 2016

Uh oh!

ocefpaf commented Oct 19, 2016

Uh oh!

wesm commented Oct 19, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ocefpaf commented Oct 12, 2016 •

edited

Loading