Python 在10.9 Mavericks上安装Scrapy时出现问题

我正在尝试使用我的自制python版本安装Scrapy。尽管所有东西都是最新的，安装在我的路径中，但我不能使用它

以下是我的操作顺序：

检查brew：

$ brew doctor
Your system is ready to brew.

检查python和pip：

$ which python
/usr/local/bin/python
$ python --version
Python 2.7.6
$ which pip
/usr/local/bin/pip
$ pip -- version
pip 1.5.4 from /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)

（这正常吗？看起来有两份，可能有冲突）

我已经安装了scrapy，但当我再次尝试时，我得到：

$ pip install scrapy
Requirement already satisfied (use --upgrade to upgrade): scrapy in /usr/local/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): Twisted>=10.0.0 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): queuelib in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0 in /usr/local/lib/python2.7/site-packages (from Twisted>=10.0.0->scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg (from zope.interface>=3.6.0->Twisted>=10.0.0->scrapy)
Cleaning up...

这对我来说很奇怪，因为我希望我的模块安装在

/usr/local/share/python

中，而不是

/usr/local/lib/python2.7/site包中

果然，在

/usr/local/lib/python2.7/site包中没有零碎的东西，但在
/usr/local/share/python
中有

我在我的路径中添加了
/usr/local/share/python/scrapy
，如下所述：

但是
哪个scrapy
返回空白

最有趣的部分是当我尝试使用
pip安装“更新”scrapy时——升级scrapy
：

~ ∮ pip install --upgrade scrapy
Requirement already up-to-date: scrapy in /usr/local/lib/python2.7/site-packages
Requirement already up-to-date: Twisted>=10.0.0 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: w3lib>=1.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: queuelib in /usr/local/lib/python2.7/site-packages (from scrapy)
Downloading/unpacking lxml from https://pypi.python.org/packages/source/l/lxml/lxml-3.3.2.tar.gz#md5=a3ea7bf74b718ecb46d9fd5198eec92d (from scrapy)
  Downloading lxml-3.3.2.tar.gz (3.5MB): 3.5MB downloaded
  Running setup.py (path:/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/lxml/setup.py) egg_info for package lxml
    /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.3.2.
    Building without Cython.
    Using build configuration of libxslt 1.1.28

    warning: no previously-included files found matching '*.py'
Downloading/unpacking pyOpenSSL from https://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.14.tar.gz#md5=8579ff3a1d858858acfba5f046a4ddf7 (from scrapy)
  Downloading pyOpenSSL-0.14.tar.gz (128kB): 128kB downloaded
  Running setup.py (path:/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/pyOpenSSL/setup.py) egg_info for package pyOpenSSL

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    no previously-included directories found matching 'doc/_build'
Requirement already up-to-date: cssselect>=0.9 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: six>=1.5.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: zope.interface>=3.6.0 in /usr/local/lib/python2.7/site-packages (from Twisted>=10.0.0->scrapy)
Downloading/unpacking cryptography>=0.2.1 (from pyOpenSSL->scrapy)
  Downloading cryptography-0.2.1.tar.gz (13.8MB): 13.8MB downloaded
  Running setup.py (path:/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py) egg_info for package cryptography
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found

    Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/cffi-0.8.1-py2.7-macosx-10.9-x86_64.egg
    Searching for pycparser
    Reading https://pypi.python.org/simple/pycparser/
    Best match: pycparser 2.10
    Downloading https://pypi.python.org/packages/source/p/pycparser/pycparser-2.10.tar.gz#md5=d87aed98c8a9f386aa56d365fe4d515f
    Processing pycparser-2.10.tar.gz
    Writing /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/setup.cfg
    Running pycparser-2.10/setup.py -q bdist_egg --dist-dir /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/egg-dist-tmp-pniw4C
    zip_safe flag not set; analyzing archive contents...

    Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/pycparser-2.10-py2.7.egg

    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 113, in <module>
        "build": cffi_build,
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
        dist.run_commands()
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "<string>", line 15, in replacement_run
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 186, in find_sources
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 246, in run
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 282, in add_defaults
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/sdist.py", line 167, in add_defaults
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
        self.finalize_options()
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/build_py.py", line 26, in finalize_options
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options
        ('force', 'force'))
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options
        src_cmd_obj.ensure_finalized()
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
        self.finalize_options()
      File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 52, in finalize_options
        from cryptography.hazmat.primitives import constant_time, padding
      File "cryptography/hazmat/primitives/constant_time.py", line 18, in <module>
        import six
    ImportError: No module named six
    Complete output from command python setup.py egg_info:
    Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found



Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/cffi-0.8.1-py2.7-macosx-10.9-x86_64.egg

Searching for pycparser

Reading https://pypi.python.org/simple/pycparser/

Best match: pycparser 2.10

Downloading https://pypi.python.org/packages/source/p/pycparser/pycparser-2.10.tar.gz#md5=d87aed98c8a9f386aa56d365fe4d515f

Processing pycparser-2.10.tar.gz

Writing /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/setup.cfg

Running pycparser-2.10/setup.py -q bdist_egg --dist-dir /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/egg-dist-tmp-pniw4C

zip_safe flag not set; analyzing archive contents...



Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/pycparser-2.10-py2.7.egg

running egg_info

creating pip-egg-info/cryptography.egg-info

writing requirements to pip-egg-info/cryptography.egg-info/requires.txt

writing pip-egg-info/cryptography.egg-info/PKG-INFO

writing top-level names to pip-egg-info/cryptography.egg-info/top_level.txt

writing dependency_links to pip-egg-info/cryptography.egg-info/dependency_links.txt

writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found



Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 113, in <module>

    "build": cffi_build,

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup

    dist.run_commands()

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands

    self.run_command(cmd)

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command

    cmd_obj.run()

  File "<string>", line 15, in replacement_run

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 186, in find_sources

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 246, in run

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 282, in add_defaults

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/sdist.py", line 167, in add_defaults

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command

    cmd_obj.ensure_finalized()

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized

    self.finalize_options()

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/build_py.py", line 26, in finalize_options

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options

    ('force', 'force'))

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options

    src_cmd_obj.ensure_finalized()

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized

    self.finalize_options()

  File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 52, in finalize_options

    from cryptography.hazmat.primitives import constant_time, padding

  File "cryptography/hazmat/primitives/constant_time.py", line 18, in <module>

    import six

ImportError: No module named six

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_myAccount/cryptography
Storing debug log for failure in /Users/myAccount/.pip/pip.log
~ ∮ 

看起来像是我的brew安装的python和不正确的链接/多个版本？许多pip导致模块安装在一个地方，而我的python安装在其他地方（这解释了为什么导入scrapy失败，但不是为什么导入scrapy失败）

一篇关于破损python模块的类似帖子建议运行
xcode-select--install
，我已经确认它是最新的

也许是虚拟时代

提前谢谢
 我在安装
virtualenv
时遇到了类似的问题：

这里的解决方案也很有效，尽管我仍然不明白问题为什么会首先出现，以及这个解决方案为什么有效。但确实如此

卸载并重新安装scrapy：

$pip uninstall scrapy
$pip install scrapy 

这将从
/usr/local/lib/python2.7/站点包中删除模块
并将它们安装在
/usr/local/bin/

问题是，根据自制python文档，pip应该安装在
/site包中
，它们应该可以工作。
它不仅在
/site-packages
中不能使用，而且pip默认安装在
/usr/local/bin/
中

这种变通方法是有效的，尽管我真的很想解释一下它为什么有效，为什么这是一个问题。那么，这两个问题是否可以合并
 以下是对我有效的方法：

由于您使用自制软件作为软件包管理器，因此通过自制软件安装的程序将设置为
/usr/local/bin
，而不是开箱即用应用程序的标准
/usr/bin
路径设置。您需要确保
/usr/local/bin
位于
/etc/path
中文件的顶部。该文件应被锁定，因此您可能需要在vi编辑器中使用以下命令保存更改：

:w !sudo tee %


$which python
应该返回
/usr/local/bin/python
，否则：

`$ brew unlink python`
`$ brew install python`
`$ brew linkapps`


$哪个pip
应该返回
/usr/local/bin/pip
，否则

`$ brew install pip`


然后

`$ sudo pip uninstall Scrapy`
`$ pip install Scrapy`
`$ pip install --upgrade Scrapy`
`$ pip install --upgrade lxml`
`$ scrapy startproject tutorial`


您需要让您的安装环境能够访问与OpenSSL捆绑的加密库。这是对我有效的解决方案

env ARCHFLAGS=“-arch x86_64”LDFLAGS=“-L/usr/local/opt/openssl/lib”CFLAGS=“-I/usr/local/opt/openssl/include”

请记住将此更改为更新的OpenSSL包的位置。
其他帖子对我不起作用，但这确实起到了作用：

sudo ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install scrapy

以上这些在OS X 10.10 Yosemite中对我都不起作用

但我找到了解决办法

它正在使用pkg配置：

brew install pkg-config libffi
export PKG_CONFIG_PATH=/usr/local/Cellar/libffi/3.0.13/lib/pkgconfig/
pip install cffi

您必须根据您的libffi版本更改路径。
是的，virtualenv可能会对这些问题进行排序。。。我已经做到了这一点，没有使用自制。。。只要做一个
sudo-pip-install-scrapy
（sudo因为通常mac权限都是乱七八糟的），它就可以工作了。。。除非您的计算机出现问题。@aj8uppal我得到的结果与我使用
pip install scrapy
时得到的结果相同，我有另一台没有自制的笔记本电脑，而且您的建议有效（我得到了命令行函数，但没有python导入）但是，在这台计算机上，scrapy在命令行上无法识别，python也无法识别它。导入scrapy
@hd1我尝试过安装
virtualenv
，效果与
scrapy
大致相同。我跟着：跟着第一个答案：
~∮ 已满足pip安装virtualenv的要求（使用--升级到升级）：virtualenv in/usr/local/lib/python2.7/site-packages Cleaning up…~∮ virtualenv test-bash:virtualenv:command未找到
执行时是否使用scrapy的完整路径？如果没有，试试看。这对我来说在10.9小牛队也很有效。使用
查找libffi.pc
查找我的：
/usr/local/ceral/libffi/3.0.13/lib/pkgconfig/libffi.pc
brew install pkg-config libffi
export PKG_CONFIG_PATH=/usr/local/Cellar/libffi/3.0.13/lib/pkgconfig/
pip install cffi